conner.xyz conner.xyz - 1 month ago 20
Bash Question

Calling rsync with pexpect: glob string not working

I'm attempting to

rsync
some files with
pexpect
. It appears the glob string argument I'm providing to identify all the source files is not working.

The gist of it is something like this...

import pexpect
import sys

glob_str = (
"[0-9]" * 4 + "-" +
"[0-9]" * 2 + "-" +
"[0-9]" * 2 + "-" +
"[A-B]" + "*"
)

SRC = "../data/{}".format(glob_str)
DES = "user@host:" + "/path/to/dest/"

args = [
"-avP",
SRC,
DES,
]

print "rsync" + " ".join(args)

# Execute the transfer
child = pexpect.spawn("rsync", args)
child.logfile_read = sys.stdout # log what the child sends back
child.expect("Password:")
child.sendline("#######")
child.expect(pexpect.EOF)


Fails with this...

building file list ...
rsync: link_stat "/Users/U6020643/git/ue-sme-query-logs/code/../data/[0-9][0-9][0-9][0-9]\-[0-9][0-9]\-[0-9][0-9]\-[A-B]*" failed: No such file or directory (2)
0 files to consider
...


The same command run in the shell works just fine

rsync -avP ../data/[0-9][0-9][0-9][0-9]\-[0-9][0-9]\-[0-9][0-9]\-[A-B].csv username@host:/path/to/dest/


The pexpect documentation mentions this


Remember that Pexpect does NOT interpret shell meta characters such as redirect, pipe, or wild cards (>, |, or *). This is a common mistake. If you want to run a command and pipe it through another command then you must also start a shell.


But doing so...

...
args = [
"rsync",
"-avP",
SRC,
DES,
]
...
child = pexpect.spawn("/bin/bash", args) # have to use a shell for glob expansion to work
...


Runs into a permissions issue

/usr/bin/rsync: /usr/bin/rsync: cannot execute binary file

Answer

To run rsync with bash you have to use bash -c "cmd...":

args = ["-c", "rsync -avP {} {}".format(SRC, DES)]
child = pexpect.spawn('/bin/bash', args=args)