[Bug-wget] Help or feature request - WGET -N option preventing file overwrite

[John Murrell]

Hello,

 

I have been trying to use WGET to download some files but have been unable
to find a combination of options to achieve what I need to do.

 

The site I am downloading from publishes a new version of a file every hour
but the exact time it is available is not specified. The replacement has the
same name.

 

I am running a script using cron to check the site every 10 minutes using
wget -N to download only the new versions of the file.

 

The problem is that WGET -N clobbers the previous version  and what I want
is to keep a copy of each version. I can't find any way of achieving this -
can anyone advise please.

 

I have tried both renaming and moving the file when WGET -N completes but
then WGET -N 'forgets' the file has already been downloaded and downloads it
again.

 

-nc only keeps a copy of the original version of the source file - I need
all the versions of the file.

 

If this is not possible I would like to suggest a new option that either:

 

*         Prevents clobbering files with different contents but duplicate
names this is a combination of the no option and -N option behaviour

*         Alternatively WGET should have an option to remember the filename
(+ timestamp and any other comparison information it requires) even though
the file is no longer in the destination directory - I call this the Ghost
option as the ghostly memory of the file exists.

2nd option would be useful if people want to rename or move files on the fly
but I can see problems on storing this information between WGET actuations.

 

Thanks

 

John Murrell