Enjoy coding Enjoy coding - 1 month ago 6
Bash Question

How to Replace a line in the same file by SED in Unix Shell scripting?

In Reference to this question After getting the line identifier matching in first and second file I need to replace the line in first file with the line of second file.For that I am using SED as below. But Sed only replaces that line in a new file. How can I achieve the update in same file without temporary file (Because those are very huge files).

#!/bin/ksh
while read line
do
var=`echo $line|cut -c 3-25`
while read i
do
var1=`echo $i|cut -c 3-25`
if [ $var == $var1 ];
then
sed -i s/$line/$i/ t1
else

echo "not matched"
fi
done < t2
done < t1


Even without
-i
option also I am not getting the result. Please help me.

Edit: Or else suggest me any other optimal way of doing without temporary files or with any scripting languages(less preferred).

Answer

You shouldn't use sed like that. As your script now stands this is what it says:

  1. For each line in t1
  2. Step through all the lines in t2
  3. If one of the lines in t2 matches the current line in t1, then step through all the lines in t1 and replace the matches
  4. Go to the next line in t1 and repeat

That means that the entire t2 file is read each time one line is read from t1. This is incredibly inefficient.

It's not necessary to use echo and cut for substrings. In Bash and ksh you can do:

var=${line:3:23}

Note: cut uses character positions for the beginning and end of a range, while this shell construct uses starting position and character count so you have to adjust the numbers accordingly.

If t2 is a list of replacements to be made in t1, thus t2 is a "script" of sorts, then this might do what you need:

keystart=3
keylen=23
while read line
do
    var="${line:$keystart:$keylen}"
    if (( ${#var} == keylen ))    # need "$" then don't need "$"
    then
        sed -in "/^.\{$keystart\}$var/ c$line" t1    # use double-quote so vars get expanded
    fi
done < t2

This will find all the lines in t1 that each line in t2 matches and do the replacement.

If, however, t1 and t2 have a line-for-line correspondence and you want to make the substitution only where corresponding lines match, then this, using a temporary file, will be the most efficient:

tempfile=$(mktemp)
keystart=3
keylen=23
while read line1
do
    var1="${line1:$keystart:$keylen}"
    read line2 <&3    # use file descriptor 3 for input

    var2="${line2:$keystart:$keylen}"
    if [[ $var1 == $var2 && ${#var2} == $keylen ]]
    then
        echo "${line2}" >> $tempfile    # substitute when matched
    else
        echo "${line1}" >> $tempfile    # keep when not matched
    fi
done < t1 3< t2    # t1 is input on stdin, t2 is input on fd 3
mv t1 t1.bak && mv $tempfile t1