Mangat Rai Modi Mangat Rai Modi - 5 months ago 47
Linux Question

Bash: Head & Tail behavior with bash script

Suppose I have following script:-

test.sh

#!/bin/bash
command1 #prints 5 lines
command2 #prints 3 lines


I run the script with
test.sh|head -n5


What will happen in this case? Will it run both the commands? or will it stop after command1?
What if I call it with -n1?

Background: I might be asking a very basic question, but I actually noticed something interesting. My script(different one) was processing 7,000 files and each file produces 1 line of output. It takes 7 minutes to run the script completely but doing head -n1 gave me prompt immediately like the script has terminated after processing first file only

Edit:
Following is my script

for i in $(ls filepath);do
echo "$i" # issue here
python mySript "$i" > "/home/user/output/""$i"".out"
fi
done


Removing echo above enables the script to run full 7 minute with head -n1, but with echo it just prints first line then exit.

Answer

This is a fairly interesting issue! Thanks for posting it!

I assumed that this happens as head exits after processing the first few lines, so SIGPIPE signal is sent to the running the script when it tries to echo $x next time. I used RedX's script to prove this theory:

#!/usr/bin/bash
rm x.log
for((x=0;x<5;++x)); do
    echo $x
    echo $x>>x.log
done

This works, as You described! Using t.sh|head -n 2 it writes only 2 lines to the screen and to x.log. But trapping SIGPIPE this behavior changes...

#!/usr/bin/bash
trap "echo SIGPIPE>&2" PIPE
rm x.log
for((x=0;x<5;++x)); do
    echo $x
    echo $x>>x.log
done

Output:

$ ./t.sh |head -n 2
0
1
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE

The write error occurs as stdout is already closed as the other end of the pipe is closed. And any attempt to write to the closed pipe causes a SIGPIPE signal, which terminates the program by default (see man 7 signal). The x.log now contains 5 lines.

This also explains why /bin/echo solved the problem. See the following script:

rm x.log
for((x=0;x<5;++x)); do
    /bin/echo $x
    echo "Ret: $?">&2
    echo $x>>x.log
done

Output:

$ ./t.sh |head -n 2
0
Ret: 0
1
Ret: 0
Ret: 141
Ret: 141
Ret: 141

Decimal 141 = hex 8D. Hex 80 means a signal was received, hex 0D is for SIGPIPE. So when /bin/echo tried to write to stdout it got a SIGPIPE and it was terminated (as default behavior) instead of the running the script.