user_loser user_loser - 5 months ago 21
Python Question

Python's os.system command and redirection

I am trying to experiment with redirection in a BASH shell, though this make work in a Windows Command Prompt shell too, with Python's

os.system
function. My problem is I was trying to redirect output from a Python script to a text file and the Python script would not do that for whatever strange syntactical reason. So, I wrote a small meaningless program to try and understand this issue. Below is my source code:

import os

for i in range(10):
os.system('ping -c 1 localhost')
print("finishing pinging the localhost - %d" % i)


When I run this in the shell like so:$python scriptName.py the program prints:

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.080/0.080/0.080/0.000 ms
finishing pinging the localhost - 0
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.073 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.073/0.073/0.073/0.000 ms
finishing pinging the localhost - 1
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.191 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.191/0.191/0.191/0.000 ms
finishing pinging the localhost - 2
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.076 ms


I only show the first 3 lines of output from the BASH shell above. However when I run the program with some type of redirection I receive this different output, more undesirable output from my view also. So if I run the program like so:
$python myScript > out.txt
. I receive the following output:

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.076/0.076/0.076/0.000 ms
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.077 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.077/0.077/0.077/0.000 ms
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.082 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.082/0.082/0.082/0.000 ms
finishing pinging the localhost - 0
finishing pinging the localhost - 1
finishing pinging the localhost - 2
finishing pinging the localhost - 3
finishing pinging the localhost - 4
finishing pinging the localhost - 5
finishing pinging the localhost - 6
finishing pinging the localhost - 7
finishing pinging the localhost - 8
finishing pinging the localhost - 9


I redacted the output from the file to only show the bottom portion. The bottom portion shows that the line that is printed, "finishing pinging the localhost - #" is printed after all the results of the
ping
program. This is completely different than the logic of the original program and the output when the script is run in the shell and not redirected.

Does anyone know why the output is different when adding redirection?

Answer

The output from your Python print statement is getting buffered, and written to the file when the script is finished (or when the buffer is full, in the case of a larger amount of printed text), whereas the output from the external call is getting written immediately. This occurs when the output is getting piped to something other than the terminal.

You need to flush the buffer to fix this:

import os

for i in range(10):
    os.system('ping -c 1 localhost')
    print("finishing pinging the localhost - %d" % i, flush=True)

If you're not using Python >3.3, then you will need to do:

import os
import sys

for i in range(10):
    os.system('ping -c 1 localhost')
    print("finishing pinging the localhost - %d" % i)
    sys.stdout.flush()