Tom Tom - 3 months ago 8x
Perl Question

bash: remove last character of previous line under condition

I would be very grateful, if you could help me with the following problem:
Using bash, from the following piece of F90 code, I try to remove the last "&" if the next line begins with a "AA" (note the whitespace before AA).

F = 2 * 3 * a * b * 7&
& * 3 * b * c&
AA = ...

should become

F = 2 * 3 * a * b * 7&
& * 3 * b * c
AA = ...

There has been a suggestion on Bash - Remove the last character of the line this before? .
Based on this, I tried

perl -0pe 's/\&\n\s*AA/\nAA/g' $MYFILE

and also

sed -i 's/\&\n\s*AA/\nAA/g' $MYFILE

which does not create any errors but also does not change anything. I also tried without \s* .

Any help is very appreciated.



Using sed

Using GNU sed:

$ sed -z 's/&\n AA/\n AA/g' file
 F = 2 * 3 * a * b * 7&
& * 3 * b * c
 AA = ...

To keep this command simple, we use the -z option to read in the whole file at once. (Technically, -z reads in NUL-separated input. Since no valid Fortran file contains a NUL, this has the effect of reading in the whole file.)

s/&\n AA/\n AA/g does the substitution that we want. Any place where the file contains & followed by newline followed by space followed by AA, this substitution removes the &.

Reading the whole file in at once is not a good approach if the file is too big to fit in memory. This should not be a problem for Fortran files.

For non-GNU sed (BSD, OSX), we need to add code to replace the -z flag:

sed 'H;1h;$!d;x;  s/&\n AA/\n AA/g' file

Using awk

$ awk '{if (/^ AA/) sub(/[&]$/, "", last); if (NR>1) print last; last=$0} END{print last}' file
 F = 2 * 3 * a * b * 7&
& * 3 * b * c
 AA = ...

How it works:

This script uses one variable last which contains the contents of the previous line. If the current line starts with AA, then we remove, if present, the final & from last. In more detail:

  • if (/^ AA/) sub(/[&]$/, "", last)

    If the current line starts with AA, then remove the final & from the previous line.

  • if (NR>1) print last

    If we are not on the first line, then print the previous line.

  • last=$0

    Save the current line as last.

  • END{print last}

    After we reach the end of the file, print last.