Rob Rob - 4 months ago 12
Bash Question

"for each in" standard output bash

I am looking to generate an output file that contains the result of the following command:

cd /path/to/files/; for each in *; do cat $each; echo "######_NEW_FILE_######"; done


I have tried:

cd /path/to/files/; for each in *; do cat $each; echo "######_NEW_FILE_######"; > output.txt; done


This generates the file, but it is blank. I have also tried:

cd /path/to/files/; for each in *; do cat $each; echo "######_NEW_FILE_######"; done; >output.txt


This generates a file that is way too big. This is not what I am looking for.
Thanks for the help.

Answer

If you want to have a redirection in place for a block of commands, without the performance impact and other side effects involved in a subshell, put it in braces:

{
  cd /path/to/files/; for each in *; do cat "$each"; echo "######_NEW_FILE_######"; done 
} >output.txt

...as a one-liner, be sure to put a ; before the closing brace:

{ cd /path/to/files/; for each in *; do cat "$each"; echo "######_NEW_FILE_######"; done; } >output.txt

By the way, note that it's cat "$each", not cat $each. If you have a file created with touch '*', you'd be doubling your output size otherwise.


That said, this is indeed very nearly identical to:

cd /path/to/files/; for each in *; do cat "$each"; echo "######_NEW_FILE_######"; done >output.txt

...for which the >output.txt applies to the for loop (and only the loop). The difference, then, is that the { ... } approach also redirects any stdout from the cd command (for which there should be none, unless you have it redefined with a shell function wrapper or similar).

That is to say:

# this does not redirect the header
cd /path/to/files; echo "header"; for each in *; do cat "$each"; done >output

# this does not redirect the header or the loop
cd /path/to/files; echo "header"; for each in *; do cat "$each"; done; echo footer >output

# this redirects everything
{ cd /path/to/files; echo "header"; for each in *; do cat "$each"; done; echo footer; } >output