user3008149 user3008149 - 14 days ago 5
Bash Question

Filter lines containing date between a range in csv file in shell

I am new to shell and need to filter files from

temp.csv
using a range of date.

Content of csv is below.

ABHA_BSC,11DPM12-1-7-C1,10/2/2016 21:00,10176244118,61948776719,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 21:15,8277521465,54472178563,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 21:30,8105902217,53136518653,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 21:45,8213431962,53183143647,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 22:00,7951867805,53442712809,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 22:15,7741104157,52587797047,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 22:30,7393727032,52228344686,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 22:45,7035552805,49033988457,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:00,6956202883,46914504782,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:15,6623893225,42756482355,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:30,6781639211,44625787536,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:45,6586403766,41882620412,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 21:00,11895958325,75139993057,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 21:15,11040804077,70012701080,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 21:30,10647919562,69074001433,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 21:45,10365456912,68592543420,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 22:00,10217309244,66630665557,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 22:15,10354867964,66097450466,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 22:30,10584852507,65666291858,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 22:45,9589181850,63763092294,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 23:00,9796008284,60538131757,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 23:15,8440733035,54114599426,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 23:30,8051347485,49383381691,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE

Answer

In awk:

$ cat program.awk
function mkdt(str) {                                             # functionize dt conversion
    split(str, a, "[/ ]")                                        # split dt
    return sprintf( "%s-%02d-%02d %s\n" ,a[3], a[2], a[1], a[4]) # zeropad and reorganize
} 
mkdt($3) > mkdt(start) && mkdt($3) < mkdt(end)                   # compare and print 

Run it:

$ awk -v start="10/2/2016 23:00" -v end="11/2/2016 20:45" -F, -f program.awk temp.csv
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:15,6623893225,42756482355,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:30,6781639211,44625787536,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C1,10/2/2016 23:45,6586403766,41882620412,Juniper_GBE_ABHA_BSC-1-7-C1_JIZAN-1-7-C1_JIZ1AH1-01 | (SOUTHERN_ABHA_ABH0027-MX480-1 TO SOUTHERN_JIZAN_JIZ0005-MX104-1),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 23:15,8440733035,54114599426,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE
ABHA_BSC,11DPM12-1-7-C11,10/2/2016 23:30,8051347485,49383381691,Juniper_GBE_ABHA_BSC-1-7-C11_JIZAN-1-7-C11_JIZ1AH1-03 | (SOUTHERN_ABHA_ABH0027-MX480-2 TO SOUTHERN_JIZAN_JIZ0005-MX104-2),1GbE

I only zeropad the day and month (1/1/2016 -> 2016-01-01), not the hours or minutes. There is no sanity checking for missing or distorted datetimes. Add = to comparisons if needed (ie. > -> >=).

Comments