hallettwx hallettwx - 1 year ago 86
Bash Question

Downloading Only Newest File Using Wget / Curl

How would I use wget or curl to download the newest file in a directory?

This seems really easy, however the filename won't always be predictable, and as new data comes in it'll be replaced with a random filename.

Specifically, the directory I wish to download data from has the following naming structure, where the last string of characters is a randomly generated timestamp:


The randomly generated timestamp is in the format of: {hour}{minute}{second}

The directory in question is here: http://mrms.ncep.noaa.gov/data/2D/RotationTrackML1440min/

Could it have to be something with something in the headers, where you'd use curl to sift through the last-modified timestamp?

Any help would be appreciated here, thanks in advance.


You can just run following command periodically:

wget -r -nc --level=1 http://mrms.ncep.noaa.gov/data/2D/RotationTrackML1440min/

It will download recursively whatever is new in the directory after last run.