moooeeeep moooeeeep - 1 month ago 13
Python Question

Interactive plot fails to initialize with input piped from another program

I have written a Python script that is supposed to run a loop which reads 3d coordinates from standard input and display the most recently read coordinates as a scatter plot on a

mpl_toolkits.mplot3d.Axes3D
.

It works when I enter the coordinates manually via the prompt (it gives some deprecation warning though):

$ python plot3d.py
.5 .5 .5
/usr/lib/pymodules/python2.7/matplotlib/backend_bases.py:2407: MatplotlibDeprecationWarning: Using default event loop until function specific to this GUI is implemented
warnings.warn(str, mplDeprecation)


It also works when I feed coordinates from a file into the program:

$ head generated
0.56 0.40 0.55
0.61 0.49 0.60
0.48 0.39 0.48
0.39 0.33 0.39
0.32 0.28 0.32
0.35 0.31 0.35
0.50 0.47 0.50
0.40 0.38 0.40
0.37 0.35 0.37
0.51 0.50 0.51
$ python plot3d.py < generated


But it doesn't work when I pipe the output of the generating script directly into the plotting script, and the generating script performs a
time.sleep()
after each iteration:

$ python generate3d.py | python plot3d.py


Working behavior is, the figure window is opened, which displays a scatter plot showing one dot. Not working behavior is, the figure window is opened, but it displays only the grey background, no axes or dots.

Here is the code for the plotting script:

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt

plt.ion()

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

plotted_items = None
while True:
if plotted_items is not None:
plotted_items.remove()
try:
x,y,z = map(float, raw_input().split())
except:
break
else:
plotted_items = ax.scatter([x],[y],[z])
ax.set_xlim(-5, 5)
ax.set_ylim(-5, 5)
ax.set_zlim(-5, 5)
plt.pause(0.1)


Here is the code for the generating script:

from random import random
import time

last = None
while True:
if last is None:
last = [random(), random(), random()]
offset = random()
for i in range(len(last)):
last[i] += 0.25 * (offset - last[i])
print "%.2f %.2f %.2f" % tuple(last)
# the value of this sleep duration influences how long it takes to initialize
# the plot in the other script!?
time.sleep(0.5)


I observed that the impact of the sleeptime in the generating script might be proportional to the time needed for the initialization of the axes. With a sleeptime close to zero, almost no initialization time is needed. With a sleeptime at 0.1 it already takes ages for the chart to appear.
I have not yet explored about the latency of the displayed data when it finally is there.

Can anyone reproduce this behavior? Can anyone help me understand and fix this? Am I doing something wrong?

Potentially helpful information:

$ lsb_release -d
Description: Ubuntu 14.04.5 LTS
$ python --version
Python 2.7.6

>>> matplotlib.__version__
'1.3.1'

Answer

I can reproduce with matplotlib version 1.4.3 and python 2.7. The reason seems to be due to python stdin reading functions blocking until the pipe closes (described here),

The origin of this problem is in the way these reading mechanisms are implemented in Python (See the discussion on this issue from Python's issue tracker). In Python 2.7.6, the implementation relies on C's stdio library.

The solution they use in the link is to run geneate3d.py as a subprocess and set the nonblocking flag,

from subprocess import Popen, PIPE
import time
from fcntl import fcntl, F_GETFL, F_SETFL
from os import O_NONBLOCK, read

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import sys

plt.ion()

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# run the shell as a subprocess:
p = Popen(['python', 'generate3d.py'],
        stdin = PIPE, stdout = PIPE, stderr = PIPE, shell = False)
# set the O_NONBLOCK flag of p.stdout file descriptor:
flags = fcntl(p.stdout, F_GETFL) # get current p.stdout flags
fcntl(p.stdout, F_SETFL, flags | O_NONBLOCK)
# issue command:
p.stdin.write('command\n')
# let the shell output the result:
time.sleep(0.1)

plotted_items = None
while True:
    if plotted_items is not None:
        try:
            plotted_items.remove()
        except ValueError:
            print(plotted_items)
            pass
    try:
        x,y,z = map(float, read(p.stdout.fileno(), 1024).split(' '))
    except OSError:
        time.sleep(0.5) 
    except:
        raise
    else:
        plotted_items = ax.scatter([x],[y],[z])
        ax.set_xlim(-5, 5)
        ax.set_ylim(-5, 5)
        ax.set_zlim(-5, 5)
        plt.pause(0.1)
        plt.draw()

you need to add sys.stdout.flush() after the print in generate3d.py.

It seems like it should have been possible to set the O_NONBLOCK flag in sys.stdin with something like flags = fcntl(sys.stdin, F_GETFL) and fcntl(sys.stdin, F_SETFL, flags | O_NONBLOCK). However, this doesn't work for me. I think this is no longer a problem in python 3.0 according to the bug tracker.

Comments