Haravikk Haravikk - 8 days ago 5
Bash Question

How to Iterate Null Separated Results in non-Bash Shell

Okay, so I have a script processing the null-separated output of

find
, and I can easily process this using a bash shell like so:

#!/bin/sh
find "$1" -print0 | while read -rd '' path; do echo "$path"; done


Fairly silly example since it just converts the results to new-lines anyway, but it's just to give you an idea of what I'm looking to do. This basic method works great, and avoids potential issues due to files possibly containing new-lines on various file-systems.

However, I need to do the same thing on a non-bash shell, which means I lose support for
read -d
. So, without resorting to bash (or other shell) specific features, is there a way that I can process null-separated results similarly to the above?

If not, what is the best to protect myself against new-lines in results? I was thinking I could perhaps use the
-exec
option of
find
to replace new-lines in file names with some kind of escaped value, but I'm not sure of the best way to find and replace the new-lines (I can't use
tr
for example) or what replacement to use, which is why null-characters are the best option if available.

Answer

See How can I find and safely handle file names containing newlines, spaces or both?.

You can e.g. use find -exec:

find [...] -exec <command> {} \;

or xargs -0:

find [...] -print0 | xargs -r0 <command>

Note that in your above example you still need to set IFS or you will trim off leading/trailing whitespace:

while IFS= read -rd '' file; do
   do_something_with "${file}"
done

You are right, it's a real bummer that this read only properly works in bash. I usually don't give a damn about possible newlines in filenames and just make sure that otherwise portable code doesn't break if they occur (as opposed to ignoring the problem and your script exploding) which I believe suffices for most scenarios, e.g.

while IFS= read -r file; do
    [ -e "${file}" ] || continue # skip over truncated filenames due to newlines
    do_something_file "${file}"
done < <(find [...])

or use globbing (when possible) which behaves correctly:

for file in *.foo; do
    [ -e "${file}" ] || continue # or use nullglob
    do_something_file "${file}"
done
Comments