Re: [Duplicity-talk] incremental backups only?

[Ed Blackman]

On Mon, Mar 20, 2017 at 03:50:50PM +0100, Raphael Bauduin via Duplicity-talk 
wrote:
> The data represents a big chunk of our backed up data, and doing a full
> backup will double the data usage of the jenkins backup, possibly putting
> us over quota if we keep multiple full backups....
> 
> What is the best backup strategy for such a dataset? Is incremental-only a
> viable option? What are the costs of incrementatl only regarding data
> safety and restore/verification speed?

Others have written about how a corrupted incremental backup will make 
all the incrementals backups that came after it useless for restoration, 
meaning that long chains of incrementals a risky proposition.

There was some talk a few years ago about adding an option to "rebase" 
the incremental chain after every N incrementals, so that incrementals 1 
through N-1 are regular incrementals, but creating incremental N would 
ignore all the previous incremental backups, such that N is the 
difference between the full backup and the current state, starting a new 
chain.  And so on until 2N, etc.

That talk never resulted in any code that I saw, but it would be 
possible to simulate it by moving files on your storage backend.  You'd 
need to add some scripting to move the incremental files on your backend 
into eg a subdirectory, so that duplicity won't see them the next time 
it goes to create an incremental.  It won't find any, so will create a 
new chain.

You'll need to script things so you're not moving files while a backup 
is running, and it will be annoying if you have to restore from one of 
the moved incrementals, but I thought I'd offer it as a possibility.

Ed

-- 
Ed Blackman