ZSG ZSG - 3 months ago 6x
Bash Question

Running commands in subdirectories with bash

I have a sequence of directories that I need to run various shell commands on and I've made a short script called dodirs.sh to simplify running the command in each directory:

echo "Running in each directory: $@"
for d in ./*/; do
cd "$d"
eval "$@"

This is fine for many simple commands, but some have trouble, such as:

grep "free energy TOTEN" OUTCAR | tail -1

which looks for a string in a file located in each directory.

It seems that the pipe and/or the quotes is the trouble since if I say:

dodirs.sh grep "free energy TOTEN" OUTCAR

I get a sensible (if waaaay to long output) along the lines of:

Running in each directory: grep free energy TOTEN OUTCAR
OUTCAR: free energy TOTEN = -888.53122906 eV
OUTCAR: free energy TOTEN = -888.53132396 eV
OUTCAR: free energy TOTEN = -888.531324 eV

I notice the result of the echo loses the quotes, so that is a bit odd. On the other hand, if I say:

dodirs.sh grep "free energy TOTEN" OUTCAR | tail -1

then I get the nonsensical:

grep: energy: No such file or directory
grep: TOTEN: No such file or directory

Notice the echo doesn't echo at all now and it is clearly misinterpreting the line.

Is there some way I have to escape characters, or package the parameters inside my dodirs.sh script?

And maybe someone knows of a better approach altogether?




# use printf %q to generate a command line identical to what we're actually doing
printf "Running in each directory: " >&2
printf '%q ' "$@" >&2
echo >&2

# use && -- we don't want to execute the command if cd into a given directory failed!
for d in ./*/; do
    (cd "$d" && echo "$PWD" >&2 && "$@")

This is much more predictable: It passes exact argument lists through, so for a general command you can just quote it naturally. (This is the exact same behavior as you get with find -exec or other tools which call execv*-family calls with a literal, passed-through argument list; thus, it means you get identical behavior to sudo, chpst, chroot, setsid, etc).

For a single command, invocation looks like what you'd expect:

dodirs grep "free  energy   TOTEN" OUTCAR

To execute shell directives, such as pipelines, explicitly execute a shell:

dodirs sh -c 'grep "free  energy   TOTEN" OUTCAR | tail -n 1'
#      ^^ ^^

...or, if you're willing to let callers rely on implementation details (such as the fact that this is implemented with a shell, and exactly which shell it's implemented with), use eval:

dodirs eval 'grep "free  energy   TOTEN" OUTCAR | tail -n 1'
#      ^^^^

This may be slightly more work, but it puts you in line with standard UNIX conventions, and avoids risking shell injection vulnerabilities if callers fail to quote their arguments to be eval-safe.