Re: [rdiff-backup-users] rdiff-backup features?

[address@hidden]

On Wed, 11 Jul 2007, Joe Beda wrote:

> (Having code to move the mirror forward and backward in time might be
> interesting too at some point -- that way you could undo bad backups
> by moving the mirror back in time and removing forward diffs).

I rather like this strange option.  Can anyone think of a real-life use of 
this feature?  Could backups be made while the mirror was back in time, 
but had forward increments several day (weeks, months...) ahead?  It would 
be kind of cool to move the mirror forward and back in time.  If this is a 
side effect of your implementation, please do include an option for it!

I like Andrew's suggestion of "The user could, optionally, "tie off" that
backup as "incomplete" and move forward." for when a backup 
fails, but you need to apply the forward increments and move on without 
completing that particular backup (for some reason).

> The biggest challenge will be making sure that we deal with the case
> where we have an increment but we don't have the metadata for that
> increment.  That is the only unforeseen case that we'll want to be
> able to clean up.
>
> So -- the backup process would change to look like this:
> 1) Look for the presence of an update_mirror.<time> file.  If it
> exists, delete any forward_mirror file and skip to step 6 and finish
> the current_mirror update.

Is this part of the recovery if #6 is interrupted?

> After the initial implementation, I would also want to change the
> metadata system to flush after so many bytes are transferred instead
> of after so many files have been written.  This way we don't loose as
> much on interruption when the files are very large.  (I backup up my
> CDs as very large flac files and can only write a few a night)

Good idea.  Graceful recovery after out-of-disk and kill-9 (power failure) 
conditions would be nice too; perhaps this your forward increment idea 
would allow for this by deleting the forward increments and keeping a 
consistent backup tree at all times if this were to happen.  Currently, 
rdiff-backup does not always recover from a power failure/kill-9.

-Eric

> Joe
>
> On 7/11/07, address@hidden <address@hidden> wrote:
> > On Mon, 9 Jul 2007, Andrew Ferguson wrote:
> > >> Step A, Backup Sync:
> > >>    1. Write changes as diffs against the current repository into
> > >> something like $REPO/rdiff-backup-data/scratch/.
> > >>    2. Simultaneously, write changes as rdiffs against the will-be new
> > >> version for placement in $REPO/rdiff-backup-data/increments/.
> > >
> > > What do you do when the link goes down in the middle of Step A? The
> > > file's you've already written scratch changes for could have changed, so
> > > you'll have to recompare anyway.
> >
> > True, but the new data wouldn't need to be retransfered.
> >
> > > Maybe the verdict is we should keep NEW files in a 'scratch area' in
> > > case the link goes down? Then, in the backup after a failed backup,
> > > rdiff-backup could inspect the scratch area for already transferred new
> > > files? Those could then be the basis for a diff transfer.
> >
> > Yes, but in the scratch area we should hold diffs to the changed files,
> > not the changed files themselves for space saving concerns.
> >
> > -Eric
> >
> >
> > _______________________________________________
> > rdiff-backup-users mailing list at address@hidden
> > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> > Wiki URL: 
> > http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki