user180946 user180946 - 3 months ago 7
Bash Question

Search using reference file and print matching lines

have a folder structure as shown below ./all_files

-rwxrwxrwx reference_file.txt
drwxrwxrwx file1.txt
drwxrwxrwx file2.txt
drwxrwxrwx file3.txt


reference_file.txt has filenames as shown below

$cat reference_file.txt
file1.txt
file2.txt


data in file1.txt and file2.txt are as shown below:

$cat file1.txt
step_1
step_2
step_3


Now, I have to take particular step say step2 from each file

Note1: file name must present in reference_file.txt

Note2: step2 is not line no:2 always.

Note3: search should perform recursively.

I have used below script:

#!/bin/sh

for i in cat reference_file.txt;

do
find . -type f -name $i | grep -v 'FS*' | xargs grep -F 'step_2'
done<reference_file.txt


after using above code i got no output.

# bash -x script.sh
+ for i in cat reference_file.txt
+ find . -type f -name **cat**
+ xargs grep -F 'step_2'
+ for i in cat **reference_file.txt**
+ find . -type f -name reference_file.txt
+ xargs grep -F 'step_2'

Answer

I think this is what you are trying to achieve. Please let me know:

EDIT: my previous version did not search recursively. Further edits: Note that using process substitution for find means that this script MUST be run under bash and not sh.

Further edit for change in specification: note the change to target and the -E option to grep instead of -F.

#!/bin/bash

target='step_.*?_?2'

while read -r name
do
   while read -r fname
   do
       if [[ $fname != FS* ]]
       then
           # Display the filename (grep -H is not in POSIX)
           if out=$(grep -E "$target" "$fname")
           then
               echo "$fname: $out"
           fi
       fi
   done < <(find . -type f -name "$name")

done < reference_file.txt

Note that your trace (bash -x) uses bash but your #! line uses sh. They are different - you should be consistent with the shell you are using.

So, I have dropped the xargs, that reads strings standard input and executes a program using the strings as argument. Since we already have the argument strings for grep we don't need it.

Your grep -v 'FS*' probably doesn't do what you expect. The regular expression FS* means "F followed by zero or more S's". Not the same as a shell pattern matching (globbing). In my solution I have used FS* because I am using the shell, not grep.

Comments