Re: [Duplicity-talk] idea on gpg zombies

[Ed Blackman]

On Sat, Jul 02, 2011 at 09:38:41AM -0400, Nate Eldredge wrote:
> I agree with Edgar.  I think there are situations where this is a 
> perfectly reasonable thing to want to do.  Imagine a large dataset 
> with frequent small changes that overwrite one another.  It may be 
> important to save past backups for a long time, so that one can 
> recover the data as it was at any time in the past.  However, there 
> may not be enough backup space to store lots of full backups.  In this 
> case the only good option is a single full backup plus lots of 
> incrementals.

I'm not familiar with the internals of duplicity, but is it possible to 
rebase an incremental chain?

In other words, set some limit (fixed? configurable?) for the length of 
a chain of incrementals.  For the purposes of discussion, say it's X.  
Like today, incremental 1 is based off of the full, and incrementals 2 
through X would be based on the one before it.  But incremental X+1 
would be an incremental from the full again.  Incrementals X+2 through 
2X would still be based on the one before it again, incremental 2X+1 
would be based on the full again, and so on.

Currently incremental chains require more and more resources to process 
as they get longer and longer.  Additionally, the longer a current 
incremental chain is, the higher the chance becomes that some part of 
the chain will become corrupt, causing problems for all the incrementals 
after it.  Rebasing the chain on the full backup after X incrementals 
limits the resources required to X*(resources for single incremental), 
and limits the damage done by a corrupted incremental backup to X*(time 
period between incrementals).

This would come at the cost of X+1 incrementals being bigger than they 
are now, since they would, in effect, roll up all the changes from the 
previous incrementals into one delta.  But my guess is that in most use 
cases it would be significantly smaller than a full backup.

Again, I have no idea if this is feasible, but I thought it might be a 
good idea for discussion.

--
Ed Blackman

signature.txt
Description: Digital signature