ddronz17 ddronz17 - 10 days ago 5x
Perl Question

Concatenating multiple lines of a text file together in perl

I'm working with some sizable data files - 90012 lines each. Each file contains weather data from 7,501 weather stations for each day of the year. There are 12 lines for each weather station, one for each month. A sample of the data is below (truncated to show only three days for each month).

I would like to write a perl script that concatenates all 12 lines for each weather station onto a single line for easier post processing. Any help is greatly appreciated.

AQW00061705 01 824C 824C 824C
AQW00061705 02 826C 826C 826C
AQW00061705 03 829C 829C 829C
AQW00061705 04 826C 826C 826C
AQW00061705 05 821C 821C 821C
AQW00061705 06 813C 813C 813C
AQW00061705 07 806C 805C 805C
AQW00061705 08 801C 801C 801C
AQW00061705 09 807C 807C 808C
AQW00061705 10 812C 812C 812C
AQW00061705 11 816C 816C 817C
AQW00061705 12 823C 823C 823C
CAW00064757 01 204Q 202Q 200Q

#!/usr/bin/env perl

use strict;
use warnings;

my %station;
while(my $line = <>) {
    my ($id, undef, @day_data) = split(' ', $line);
    push @{$station{$id}}, @day_data;

foreach my $sid (sort keys %station) {
    print $sid, ' ', join(' ', @{ $station{$sid} }), "\n";

And then

./script.pl your_wheather_file        

gives the following output:

AQW00061705 824C 824C 824C 826C 826C 826C 829C 829C 829C 826C 826C 826C 821C 821C 821C 813C 813C 813C 806C 805C 805C 801C 801C 801C 807C 807C 808C 812C 812C 812C 816C 816C 817C 823C 823C 823C
CAW00064757 204Q 202Q 200Q