David Heyman David Heyman - 2 months ago 10
Python Question

Cannot successfully redirect stdout from Popen to temp file

The title says it all, really. Currently running Python 2.7.8, using

subprocess32
. Minimum reproducing snippet, copied directly from the REPL:

Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> import subprocess32 as subp
>>> tfil = tempfile.TemporaryFile()
>>> tfil
<open file '<fdopen>', mode 'w+b' at 0x7f79dcf158a0>
>>> proc = subp.Popen(['ls', '-al'], stdout=tfil, stderr=subp.STDOUT)
>>> proc
<subprocess32.Popen object at 0x7f79dcf04a50>
>>> proc.wait()
0
>>> tfil.read()
''
>>>


Changes to the code that made no difference in the result, in any combination:


  • Using
    subprocess
    instead of
    subprocess32

  • Using
    NamedTemporaryFile
    instead of
    TemporaryFile

  • Setting
    bufsize=0
    on either or both of
    tfil
    and
    proc

  • If I try
    subp.Popen(['ls', '-al'], stdout=sys.stdout, stderr=subp.STDOUT)
    , the output redirects to the terminal just fine - but I need to capture it somewhere else, I can't just mix it together with actual terminal output. Redirecting
    sys.stdout
    to the temp file, and then setting it back after the process is done, doesn't work either.



I need to do this because the command I want to use in the actual non-minimal-example program (
qacct -j
) has an output that's too big for
subprocess.PIPE
, raising a
MemoryError
(no further details).

The most relevant existing question I've found is this, but that didn't actually help.

Answer

The file is properly written to, but your tfil.read() command will start reading at the point where the last write happened, so it will return nothing. You need to call tfil.seek(0) first.

Comments