Javia1492 Javia1492 - 1 month ago 6
Perl Question

Using grep to find matching patterns within files in different directories via perl

Im modifying an existing perl script which checks the status of some tests ran by going to their directory and parsing for pass, fail, or incomplete. I want to extend this script to generate a sublist containing the incompleted tests based on 3 strings:

"Error: Failure to checkout svverification license feature."
"Error: Unable to checkout verification license - required for testbench features"
"FATAL ERROR: License connection lost and unable to reconnect after multiple attempts."


Each test name has its own directory and inside the directory is a log file named
vsim.log
. I could do this directory with a
grep
, but I figured it would be nice to implement it into the script so I dont have to manually
grep
every time there are incompletes.

The script has a variable
$cur_test_name
which I was going to use to perform this since I have to make sure to go into the proper sub-directory.

chdir("/testout/$cur_test_name.1/") or die "Cannot chdir to /testout/$cur_test_name.t";
if(system(grep -f "Error: Failure to checkout svverification license feature." vsim.log) or
system(grep -f "Error: Unable to checkout verification license - required for testbench features" vsim.log) or
system(grep -f "# FATAL ERROR: License connection lost and unable to reconnect after multiple attempts." vsim.log))
{
open($incomplete_tests, '>'; 'incompletes.lst') or die "Cannot open file $!in write mode";
print {$incomplete_tests} $cur_Test_name . "\n";
close $incomplete_tests;
}
chdir("../../") or die "Cannot chdir to ../../";


So basically, I want to change directory to the subdirectory and check inside the vsim.log file of those patterns exist. If they create a new file via open (I think this is how its done?) and write the test name variable into the file. Then i close the file, and go back to the top level directory and repeat this for each test that is incomplete.

But this does not compile and Im new to perl so im having trouble figuring out how to use
system()
properly. I've seen different formats and I've seen some solutions using only
grep
without
system
so Im kind of stuck.

The errors i get are the following:

Bareword found where operator expected at bin/run_regress.pl line 1279, near ""Error: Failure to checkout svverification license feature." vsim"
(Missing operator before vsim?)
Bareword found where operator expected at bin/run_regress.pl line 1279, near ""Error: Unable to checkout verification license - required for testbench features" vsim"
(Missing operator before vsim?)
Bareword found where operator expected at bin/run_regress.pl line 1279, near ""# FATAL ERROR: License connection lost and unable to reconnect after multiple attempts." vsim"
(Missing operator before vsim?)
Not enough arguments for grep at bin/run_regress.pl line 1279, near ""Error: Failure to checkout svverification license feature." vsim"
syntax error at bin/run_regress.pl line 1279, near ""Error: Failure to checkout svverification license feature." vsim"
Global symbol "$cur_Test_name" requires explicit package name at bin/run_regress.pl line 1282.
syntax error at bin/run_regress.pl line 1287, near "}"

Answer Source

First off, there is no reason to go out to the system in order to search files for patterns; process files.

Next, if you were to do it system is a wrong tool. For one, it doesn't return the command's STDOUT but rather the exit status ($?). Here you only need to know whether there were matches and grep's exit does convey this, and is packaged into the high bits of $?. Thanks to ysth for a comment.

However, your code still won't see errors since any non-zero return is true under if. You'd need to analyze $?, and if $? >> 8 == 0 that means that there were matches, according to grep's man page. But then the utility of this approach is limited to grep.

Also, if the design ever changed and you need the output (lines that match for example ), you'd want qx (backticks), or one of a number of other tools/modules.

Here is a working program along the lines of what you want

use warnings;
use strict;
use feature 'say';
use Cwd;

my @dirs = qw(one two three);   # my tests, change to suitable names
my $logfile = 'log.txt';    
my $incomplete_tests = 'incomplete_tests.log';
my $curr_test_name   = 'current_test';

# regex to search with  
my $err_qr = qr/Error: Failure|Error: Unable|FATAL ERROR:/;  #/

# Open log file for append
open my $fh_incomp, '>>', $incomplete_tests 
    or die "Can't open to append $incomplete_tests: $!";

my $orig_dir = cwd;
foreach my $dir (@dirs)
{
    chdir $dir or die "Can't chdir to $dir: $!";
    my $updated_name = $curr_test_name . "_$dir";  # update as/if needed

    open my $fh, '<', $logfile  or do {
        warn "Can't open $logfile: $!";
        next;
    };

    while (<$fh>) {
        say $fh_incomp $updated_name if /$err_qr/;
    }   

    chdir $orig_dir or die "Can't chdir to $orig_dir: $!";
}
close $fh_incomp;

Since I don't understand your full problem this is basic, and with a few guesses. It goes through a list of directories, checking a log file with the same name in each and recording the "test name" each time a line matches the regex composed ahead of time with qr operator.

You can change it to record error-lines as well (add a print), or to record the test name only once per log file (skip to next directory once it matches).

Also note that there is no reason really for chdir, as you can easily use $dir to form proper paths. I use it in case your real code does more work and can actually benefit from being in $dir.

There are other ways to do this, in the first place by searching for the log-file across all directories and running the same code in each case. This can be done nicely with File::Find::Rule or Path::Tiny.