BarFooBar BarFooBar - 3 months ago 42
Python Question

Running an AWS cli command from a local python script?

I'm trying to run this aws s3 ls command:

aws s3 ls s3://path/to/my/bucket/12434 --recursive --human-readable --summarize

with this python:

command = 'aws s3 ls s3://path/to/my/bucket/12434 --recursive --human-readable --summarize'
s3_folder_data = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
print s3_folder_data

But it's failing with this error:

subprocess.CalledProcessError: Command 'aws s3 ls s3://path/to/my/bucket/12434 --recursive --human-readable --summarize' returned non-zero exit status 1

The command itself works when I run it. The python script is being called by the same user on the same machine. What gives?


As suggested by others, use Boto3 S3 library to get what you want. But if you insist on subprocess, try:

subprocess.check_output(['aws', 's3', 'ls', 's3://path/to/my/bucket/12434', '--recursive', '--human-readable', '--summarize'])

or['aws', 's3', 'ls', 's3://path/to/my/bucket/12434', '--recursive', '--human-readable', '--summarize'])

and build on it.