Re: [Duplicity-talk] resume interrupted backup + changed files - are they saved properly?

[edgar . soldin]

hey Mark,

On 23.07.2018 11:33, Mark Diab wrote:
> Hi,
> 
> In the meantime I have tested it and it indeed seems to be the case :(
> 
> I ran an incremental backup right after the full backup, it didn't detect 
> changes. Then I tried to restore some files that were changed between 
> crash<>resume and while "duplicity list" shows the proper (new) dates/times, 
> duplicity restore restored old (at time of crash) files. What's worse, it now 
> thinks that those new files are backed up so it won't ever make incremental 
> backups until a new full backup is started.

just for clarification, which backup was interrupted and resumed. the full 
right?
 
> It means that it leaves an invisible "hole" in backups that is very tricky to 
> fix or detect. Files created in the interim period will not be saved unless 
> they are changed again. 
> 
> This could be fixed by either processing the previous (crashed) snapshot file 
> at recovery or simply NOT ignoring any files that were changed since the 
> first backup attempt (when recovery is fast forwarding to the last backed up 
> file). I am not sure how the system reacts if a backup archive has multiple 
> copies of the same file though so this may make implementation difficult.

yeah, resuming is a tricky beast, let's see what Ken has to say about. in the 
meanwhile, if you feel capable a patch to fix the issue would be highly 
appreciated.

..ede/duply.net
 
> thanks, 
> 
> Mark
> 
> 
> 
> 
> On Mon, Jul 23, 2018 at 11:16 AM <address@hidden <mailto:address@hidden>> 
> wrote:
> 
>     as Ken implemented that feature i guess he can come back to you w/ 
> implementation details.
> 
>     ..ede/duply.net <http://duply.net>
> 
>     On 23.07.2018 08:54, Mark Diab via Duplicity-talk wrote:
>     > Hi List,
>     >
>     > I'm running duplicity (0.7.10) to back up several servers and recently 
> I've hit the signature file >2GB bug when incrementals became too old and it 
> tried to make a new full backup, so I had to increase --max-blocksize. 
>     >
>     > Duplicity seems to be recovering OK, however, it made me wonder about 
> files that have changed in the meantime (between the crash and the resumed 
> recovery). It's quite a lot of files so I'd prefer not to start over from the 
> beginning of the full backup but resume what's already uploaded.
>     >
>     > Now, if i understand correctly resume is basically implemented as a 
> "fake" backup not doing any uploads up to the point of the last backed up 
> file. I can see that the SigTar file is regenerated. I assume that sigtar 
> contains the metadata to detect changes. However, this may lead to a 
> situation where checksums / file meta information contains different data 
> (time of resume) than the actual archive (backed before those files were 
> changed).
>     >
>     > My question is if anyone knows what happens to those files? Is this how 
> resume works? Does it mean that duplicity is never going to back up files 
> that were changes between crash<>resume  because their change went unnoticed 
> since the meta data contains newer files than the archive uploaded before the 
> crash?
>     >
>     > thanks,
>     >
>     > Mark
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > Duplicity-talk mailing list
>     > address@hidden <mailto:address@hidden>
>     > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >
>