[Duplicity-talk] Incremental backup when data changes but timestamp does not

[Nate Eldredge]

Returning to a thread from many years ago 
(https://lists.gnu.org/archive/html/duplicity-talk/2013-07/msg00015.html), 
I am looking for a way to do an incremental backup involving files whose 
data has changed but the timestamp, permissions and size stayed the same.

This actually comes up in a real-life situation, not via some sort of 
deliberate timestamp abuse.  They're files from the same package in two 
different versions of Ubuntu.  I assume the packages were built 
simultaneously from the same source, but using different compiler 
versions, and the files for each one happened to be created within the 
same second.  So if you upgrade from one package to the other, the new 
version of the file is different, but has the same mtime and permissions 
and possibly even the same size.  Then `duplicity incremental` doesn't 
notice the change, and your backup stays with the old version.

At one time I worked around this by hacking in a command-line option which 
causes ROPath.__eq__ 
(https://gitlab.com/duplicity/duplicity/-/blob/main/duplicity/path.py#L331) 
to always return 0.  Then every file is treated as "changed", and so the 
changes in question are picked up.  For those files that haven't actually 
changed, the rdiff is trivial, and so the only practical impact is that 
the backup takes a long time and you get a big new-signatures file, which 
I can live with.  For me it usually only happens when I upgrade OS 
versions, so I would run an incremental with this option at those times. 
(Or, I would bite the storage-space bullet and run a full backup even if I 
didn't otherwise need one.)

It'd be nice to have something more efficient and robust, though.  One 
thought would be to check whether the ctime is newer than the date of the 
previous backup.  We could also check the birth time on filesystems that 
support it.  We would get false positives in cases like replacing a disk 
and `cp -a`ing over all the files (which normally would preserve mtime but 
not ctime), but it could still be useful as an option.

I'm curious if anyone has other suggestions, or tips on how / where to 
implement them.

Thanks!

--
Nate Eldredge
nate@thatsmathematics.com