Hjulle Hjulle - 2 months ago 7
Bash Question

How to generate a list of all dates in a range using the tools available in bash?

I want to download a bunch of files named with ISO-8601 dates. Is there a simple way to do this using bash+GNU coreutils? (Or some trick to make wget/curl to generate the list automatically, but I find that unlikely)

Similar to this question, but not restricted to weekdays: How to generate a range of nonweekend dates using tools available in bash?.
I guess that there is a simpler way to do it without that restriction.

Also related to How to generate date range for random data on bash, but not restricted to a single year.

Answer

Here's two ways you could do it. Both of them require GNU date.

You could do use a for loop:

for i in {1..5}; do echo $(date -I -d "2014-06-15 +$i days"); done
2014-06-16
2014-06-17
2014-06-18
2014-06-19
2014-06-20

Alternatively, you could use an until loop:

d="2014-06-29"
until [[ $d > "2014-07-03" ]]; do 
    echo "$d"
    d=$(date -I -d "$d + 1 day")
done

Output:

2014-06-29
2014-06-30
2014-07-01
2014-07-02
2014-07-03

For a more portable way to do the same thing, you could use a Perl script:

use strict;
use warnings;
use Time::Piece;
use Time::Seconds;    
use File::Fetch;

my ($t, $end) = map { Time::Piece->strptime($_, "%Y-%m-%d") } @ARGV; 

while ($t <= $end) {
    my $url = "http://www.example.com/" . $t->strftime("%F") . ".log";
    my $ff = File::Fetch->new( uri => $url );
    my $where = $ff->fetch( to => '.' );  # download to current directory
    $t += ONE_DAY;
}

Time::Piece, Time::Seconds and File::Fetch are all core modules. Use it like perl wget.pl 2014-06-29 2014-07-03.