Re: [Duplicity-talk] Sliding window backup strategy?

[Serge Wroclawski]

On Sun, Oct 05, 2008 at 04:46:03PM +0200, Lars Marowsky-Bree wrote:

> However, one idea I did have (maybe it is not original, but I couldn't
> readily find it elsewhere) was what I call "sliding window
> full-backups".
> 
> The closest I found in the archives here is
> http://lists.gnu.org/archive/html/duplicity-talk/2008-08/msg00075.html
> (a discussion between Rik v. A and Colin Ryan), but merging changes back
> into encrypted chunks is of course hard.

Look at AMANDA. Thier whole system is designed around a concept called 
"Schedule Balancing".

They examine every machine, when the last full was done, how many data chamges, 
how frequenly data changes on a particular 
machine, etc. then balances out the deltas and fulls across the pool of 
machines.

> For example, if I set n = 5%, I would only require the last 20d daily
> backups to restore a full snapshot. And every incremental run would be
> bounded and about the same time - I'd never have to pay the penalty of a
> full sync.

With AMANDA, you say "I want to be sure to get a full backup of every machine 
in n days". In that time, it figures out which to 
do when based on the above.

http://man.chinaunix.net/network/amanda/topten.html   "the friday-tape-question"

This is especially important for AMANDA because it is meant for tapes, so the 
issue of fulls vs incrementals is a significant 
one since it could mean needing many tapes to be loaded for a restore.

AMANDA has been around since the late 90s, so they have thought a lot about 
this and other scheduling issues.

If you want to go down this road, I suggest you look there first for 
inspiration.

- Serge