Re: [Duplicity-talk] Feature request/discussion: Store identical files only once

[Peter Schuller]

Catching up on older traffic... [quoting a bit extra since it's old context]

> > I considered in the past implementing an exclusively hash-based backup
> > where the backup store would just be a refcounted set of
> > checksum->content mappings, and each backup set a tree of meta-data with
> > checksums.
>
> I was thinking about that too.
>
> > I generally like the idea. The problem I see is security.
>
> I see another problem:
>
> 1) either you pack your backed up files in archives just like duplicity
> does, and then you can't delete an archive as long as it contains at
> least one file which is still referenced
>
> 2) or you have one backup file for each original file, which causes an
> information leak (gives much more information to an attacker about your
> backup), and also causes performance issues when backuping many files
> (eg. full system backup).
>
> I would go for 1), but it requires to address the mentioned issue with
> still referenced files...

In my case I was going to store files in dedicated files. So, I was not 
attempting to reach the level of security of encrypted dumps; in fact I was 
not going to encrypt at all. This was more like an alternative to 
rdiff-backup which also happens to translate more easily to remove backups.

I was not worrying too much about performance either since you would never 
ever have to re-do full backups (and I feel a non-crappy local file system is 
an acceptable requirement). That said, of course traversing the full backup 
will be slow; to me, in the initial design, that was an acceptable trade-off.

I also had a toy implementation of part of the stuff that backed up to a 
PostgreSQL database ;)

Another alternative would be to store files above a certain threshold in 
dedicated files, but lump smaller files together. But then we are sliding 
into complexity again.

-- 
/ Peter Schuller

PGP userID: 0xE9758B7D or 'Peter Schuller <address@hidden>'
Key retrieval: Send an E-Mail to address@hidden
E-Mail: address@hidden Web: http://www.scode.org

signature.asc
Description: This is a digitally signed message part.