user1777667 user1777667 - 4 months ago 11
Python Question

How to handle stdout from command like "with open() as f:" in python

I am trying to stream large files over ssh, and can currently stream raw files just fine; as in:

with open('somefile','r') as f:
tx.send(filepath='somefile',stream=f.read())


tx is a higher level class instance that I have that can stream just fine this way, but I want to be able to use commands like
pv
,
dd
, and
tar
to stream as well. What I need is:

with run_some_command('tar cfv - somefile') as f:
tx.send(filepath='somefile',stream=f.read())


This would take stdout as a stream and write to a remote file.
I've tried doing something like:

p = subprocess.Popen(['tar','cfv','-','somefile'], stdout=subprocess.PIPE)
tx.send(filepath='somefile',stream=p.stdout.readall())


but to no avail...
I've been googling for a while trying to find an example, but no luck so far.
Any help would be much appreciated!

Answer

I think that the only issue is the .readall() method, that does not exist.

You can use p.stdout.read() to read the entire contents of stdout:

p = subprocess.Popen(['tar','cfv','-','somefile'], stdout=subprocess.PIPE)
tx.send(filepath='somefile',stream=p.stdout.read())
Comments