BobPeterson BobPeterson - 2 months ago 12x
Python Question

Bypassing size limitation on a subprocess command

I need your help to solve my problem. I want to launch a powershell command using subprocess. However the command used will be very long, so it will be bigger than the size of the buffer. I have done many tests and right now I'm not able to find a good solution.
I would like to split my big buffer (it's a base64 content) into small buffers but I don't know how to do using subprocess.

For example, here is my subprocess call:

a = "a" * 32716
command = ["powershell.exe", "/c", "$base64='%s'\n" % a, "Write-Host $base64"]
process = subprocess.Popen(command, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, universal_newlines=True)

It works if the size of the variable "a" is less than 32717, if it's bigger (ex: equals to 32717) I have the following error: WindowsError: [Error 87] The parameter is incorrect. If it's a lot bigger, I have this error: WindowsError: [Error 206] The filename or extension is too long.

I tried to split the "command" variable into small buffers but the content of the buffer is not by cell, it's a concatenation of the entire tab, so it doesn't change the problem.

command = ["powershell.exe", "/c", "$base64=" + small_buffer1, "$base64=$base64+" small_buffer2, etc.]

Another idea was to use different subprocess calls. The problem is I cannot concatenate a variable because it will not be defined on the same powershell instance. So between subprocess, I will lose the "base64" content.

subprocess.Popen(["powershell.exe", "/c", "$base64='%s'\n" % a], stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
# $base64 will be empty
subprocess.Popen(["powershell.exe", "/c", "Write-Host $base64"], stderr=subprocess.STDOUT, stdin=subprocess.PIPE)

If anyone has an idea, it will be very nice. I think, it would be possible to define one instance of powershell and add buffers into it as an interactive shell.

Thank you very much for your help.


The solution that I've found is easy to implement. The goal is to keep the subprocess open and to write the buffer in stdin. Here is an example:

# keep the pipe open
p = subprocess.Popen(cmd, stdout=PIPE, stderr=PIPE, stdin=PIPE, shell=True)

# initialize the powershell value

# split the buffer to small buffers
size = 25000
tab = [veryLongLine[i:i+size] for i in range(0, len(veryLongLine), size)]
for t in tab:
    # concatenate small buffers to obtain the "veryLongLine" value
    p.stdin.write("$data+=\"%s\"\n" % t)

    # flush the buffer every time (the $data value is not flushed)        

# write the output
p.stdin.write("Write-Host $data\n")

Hope it will help !