lining lining - 1 month ago 6
Python Question

How to convert shell script in one line and execute it from Python subprocess?

I have my below simple shell script which I am trying to convert in one single line so that I can execute it from my Python program using subprocess -

#!/bin/bash
set -e

COUNT=60 #number of 10 second timeouts in 10 minutes
SUM_SYNCS=0
SUM_SYNCS_BEHIND=0
HOSTNAME=$hostname

echo $HOSTNAME

while [[ $COUNT -ge "0" ]]; do

#send the request, put response in variable
DATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)

#grep $DATA for syncs and syncs_behind
SYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')
SYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')

echo $SYNCS
echo $SYNCS_BEHIND

#verify conditionals
if [[ $SYNCS -gt "8" && $SYNCS_BEHIND -eq "0" ]]; then exit 0; fi

#decrement the counter
let COUNT-=1

#wait another 10 seconds
sleep 10

done


I converted the above script in one line like this -

jsonStr = {"script": "#!/bin/bash\nset -e\n\nCOUNT=60 #number of 10 second timeouts in 10 minutes\nSUM_SYNCS=0\nSUM_SYNCS_BEHIND=0\nHOSTNAME=$hostname\t\n\necho $HOSTNAME\n\nwhile [[ $COUNT -ge \"0\" ]]; do\n\n#send the request, put response in variable\nDATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)\n\n#grep $DATA for syncs and syncs_behind\nSYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')\nSYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')\n\necho $SYNCS\necho $SYNCS_BEHIND\n\n#verify conditionals\nif [[ $SYNCS -gt \"8\" && $SYNCS_BEHIND -eq \"0\" ]]; then exit 0; fi\n\n#decrement the counter\nlet COUNT-=1\n\n#wait another 10 seconds\nsleep 10\n\ndone\n"}


And I am trying to execute this using Python subprocess but everytime I am getting an error -

File "C:\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer


Below is my full Python code which I am trying out -

jsonStr = {"script": "#!/bin/bash\nset -e\n\nCOUNT=60 #number of 10 second timeouts in 10 minutes\nSUM_SYNCS=0\nSUM_SYNCS_BEHIND=0\nHOSTNAME=$hostname\t#\n\necho $HOSTNAME\n\nwhile [[ $COUNT -ge \"0\" ]]; do\n\n#send the request, put response in variable\nDATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)\n\n#grep $DATA for syncs and syncs_behind\nSYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')\nSYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')\n\necho $SYNCS\necho $SYNCS_BEHIND\n\n#verify conditionals\nif [[ $SYNCS -gt \"8\" && $SYNCS_BEHIND -eq \"0\" ]]; then exit 0; fi\n\n#decrement the counter\nlet COUNT-=1\n\n#wait another 10 seconds\nsleep 10\n\ndone\n"}

j = json.loads(jsonStr)

print "start"
proc = subprocess.Popen(j['script'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/bin/bash')
(stdout, stderr) = proc.communicate()
print stdout
print stderr
print "end"


What is wrong in my shell script which I have converted in one line? Any thoughts?

NOTE: All my other shell scripts are working which are also in one liner from the same Python subprocess call, only this is having some problem.

Answer

The error you are getting is from json.loads(jsonStr). json.loads require a str or unicode not a dict. jsonStr is already a dict you don't need json.loads