Rishi Rishi - 1 year ago 73
Perl Question

How loading of file into memory in perl

i have tried with some script for sorting a input text file in descending order and print only top usage customer.

input text file containts:

example :

and so on, this is very large file and i am trying to avoid loading file into memory.

I have tried with following script.

use warnings ;

use strict;

my %hash = ();
my $file = $ARGV[0] ;

open (my $fh, "<", $file) or die "Can't open the file $file: ";

while (my $line =<$fh>)
chomp ($line) ;
my( $name,$key,$ip) = split /,/, $line;

$hash{$key} = [ $name, $ip ];

my $count= 0 ;

foreach ( sort { $b <=> $a } keys %hash ){
my $value = $hash{$_};
print "$_ @{$value} \n" ;
last if (++$count == 5);

Output should be sorted based on usage and it will show the name and IP for respective usage. " `

Answer Source

I think you want to print the five lines of the file that have the highest value in the second column

That can be done by a sort of insertion sort that checks each line of the file to see if it comes higher than the lowest of the five lines most recently found, but it's easier to just accumulate a sensible subset of the data, sort it, and discard all but the top five

Here, I have array @top containing lines from the file. When there are 100 lines in the array, it is sorted and reduced to the five maximal entries. Then the while loop continues to add lines to the file until it reaches the limit again or the end of the file has been reached, when the process is repeated. That way, no more than 100 lines from the file are ever help in memory

I have generated a 1,000-line data file to test this with random values between 100 and 2,000 in column 2. The output below is the result

use strict;
use warnings 'all';

open my $fh, '<', 'usage.txt' or die $!;

my @top;

while ( <$fh> ) {

    push @top, $_;

    if ( @top >= 100 or eof ) {

        @top = sort {
            my ($aa, $bb) = map { (split /,/)[1] } ($a, $b);
            $bb <=> $aa;
        } @top;

        @top = @top[0..4];

print @top;


Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download