Sergey Dylda Sergey Dylda - 2 months ago 10
Python Question

Python: How to get an output of subrocess everytime it's outputting and not on terminating?

I have two scripts.
(Code is listed below). First script is just printing numbers with a delay.

Second script is supposed to start the first one as a subprocess, then checking in the background (!) if the first script outputting a number, and printing it if so.

But what I get is that the second script printing all of the numbers at once when first one is terminated (after 20 seconds) and not one number at each moment.
Output is like:

START
0
1
2
3
4
5
6
7
8
9
16:05:51
OVER


But what I expect is something like:

START
0
16:05:51
1
16:05:53
2
16:05:55
3
16:05:57
4
16:05:59
5
16:06:01
6
16:06:03
7
16:06:05
8
16:06:07
9
16:06:09
OVER


How can I solve this?

Code:

1.py:



import time
for i in range(10):
print(i)
sys.stdout.flush()
time.sleep(2)


test.py:



from subprocess import Popen, PIPE, STDOUT
from time import strftime
import threading, sys

stream = Popen(['python', '1.py'], stdout=PIPE, stderr=STDOUT)
def a():
while True:
try:
out = stream.communicate()[0].decode("utf-8")
print(out, strftime("%H:%M:%S"))
sys.stdout.flush()
except ValueError:
print('OVER')
break
except:
raise
t = threading.Thread(target=a,)
t.start()
print('START')

Answer

Try calling

sys.stdout.flush()

after each printout!

You are basically writing into a buffer, and those writes don't become "flushed" out until the writer process is done! So, if you don't want to wait until the python interpreter (or the OS underneath) decides "it is time to flush" ... you have to enforce that step "manually"!

See that other answer for more background information!

EDIT: as this is not working with stream as you say: probably you have to other ways to invoke that second script. You could try using the pexpect module instead. That one exists for the "sole" purpose of "interacting" with stdout output!