python how to use subprocess pipe with linux shell

Question

I have a python script search for logs, it continuously output the logs found and I want to use linux pipe to filter the desired output. example like that:

$python logsearch.py | grep timeout

The problem is the sort and wc are blocked until the logsearch.py finishes, while the logsearch.py will continuous output the result.

sample logsearch.py:

p = subprocess.Popen("ping google.com", shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

for line in p.stdout:

    print(line)

UPDATE:

figured out, just change the stdout in subprocess to sys.stdout, python will handle the pipe for you.

p = subprocess.Popen("ping -c 5 google.com", shell=True, stdout=**sys.stdout**)

Thanks for all of you help!

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Answer 1

And why use grep? Why don't do all the stuff in Python?

from subprocess import Popen, PIPE
p = Popen("ping google.com", shell=True, stdin=PIPE, stdout=PIPE)
for line in p.stdout:
    if 'timeout' in line.split():
        # Process the error
        print("Timeout error!!")
    else:
        print(line)

python how to use subprocess pipe with linux shell

Sign up