stackofpancakes stackofpancakes - 6 months ago 13
Python Question

TypeError while looping through a variable list of files with subprocess commands

As background, I have created a list with elements of different filenames with their full path

(/.../filea.dat)
called
fileList
, it is of variable length. It has the form
fileList = ['/../filea.dat', '/../fileb.dat']
.

I would like to execute a subprocess command on every file in that list of files, then later analyze the components of each of the files (and generated new files) separately.

for i, elem in enumerate(fileList):
hexed = fileList[i]
subprocess.Popen("hexdump " + hexed + " > hexed.dat", shell=True)

with open("hexed.dat", "r") as f:
for line in f:
if "statement" in line:
value = "examplevalue"
if value == "examplevalue"
other subprocess statements that create a file that will again be used later


Right now I have a
TypeError: cannot concatenate 'str' and 'list' objects
. Let me know if I'm on the right track with this method approach as well.

Please let me know if I need to provide additional clarification; I tried to simplify to the basics as the other details aren't important to the issue.

Answer

You are close. You got the type error because Popen requires that you also set shell=True when passing in a string instead of a list. But there is another problem: Popen doesn't wait for the process to complete so when you read the file, there isn't anything useful in it yet. A different strategy is to skip file redirection and read the output stream directly. Note also that you don't need to use enumerate... the for loop gets the values in the list already. And we can skip the shell and pass the command as a list.

for hexed in fileList:
    proc = subprocess.Popen(["hexdump", hexed], stdout=subprocess.PIPE,
        stderr=open(os.devnull, 'w'))
    for line in proc.stdout:
        if "statement" in line:
            value = "examplevalue"
    proc.wait()
    if proc.returncode != 0:
        print('error') # need less lame error handling!   
    if value == "examplevalue"
        other subprocess statements that create a file that will again be