Hodaya Beer Hodaya Beer - 3 years ago 211
Python Question

python popen sdtout doesn't get all the output when exe fails

I'm new to python, and trying to run a exe software from python in windows.
I wrote the following code:

from subprocess import STDOUT, Popen, PIPE


with open('test.log', 'w') as f:
p = subprocess.Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
for c in iter(lambda: p.stdout.read(1), ''):

The exe program have some running errors, and I need to get the output of the program in order to fix the params file in order to prevent the errors.

the problem is that by using the above code I don't get the full output of the exe (when comparing to the os.system() command). the error message window of the exe pops out before the completion of the output writing, and I don't know where is the problem.

can you please help me...

Answer Source

stderr=PIPE redirects the error stream to p.stderr, and you're not reading that (note that using p.communicate allows to get both stream results, but reading them separately can lead to deadlocks).

Anyway, if you don't care about merging both out & err streams, you could change that to:


so both out & err use the same stream p.stdout

Also: don't use shell=True, you don't need it.

If that doesn't fix it in your case, it means that the underlying program crashed while not flushing its output. Output flush works differently when output is not redirected, which may explain why you get more output when running it without redirection with os.system (more about this issue: forcing a program to flush its standard output when redirected)

One lead yet to be explored would be to use winpty which is an equivalent of unbuffer on Windows: What is the equivalent of unbuffer program on Windows?. Something like:

cmd = ["winpty.exe","-Xallow-non-tty","-Xplain","TLWMA-0.09.exe"]
p = subprocess.Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=STDOUT)
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download