Re: [Duplicity-talk] --archive-dir is growing in size, how can I reduce it?

[Chris Poole]

On Fri, Dec 9, 2011 at 6:43 PM, Alex Cartwright <address@hidden> wrote:
> I think the idea of having to use a new location to resolve this is
> opening up quite a big flaw in how Duplicity works.

I wouldn't call it a flaw, really. If all your stuff is encrypted elsewhere, and
you're encrypting at a filesystem level (virtual or otherwise), then you need a
local cache of metadata.

> Surely the majority of this metadata can be stored at the backup location.

It is. On the remote, there's all your actual data (encrypted), and all your
metadata (also encrypted). The metadata is simply used so that it knows where
old versions of files are when you come to restore, and what parts of which
files have changed when you run the next incremental backup.

However, when you do come to running an incremental backup, you'd then have to
pull those metadata files from your remote host, decrypt (and possibly check the
signature(s)), before actually reading the metadata.

To save time, this metadata is also stored locally, unencrypted. By default it's
in `~/.cache/`.

If this is taking up too much space for you, feel free to remove it. Just
remember that on next backup, the relevant metadata files will have to be pulled
from the remote host before the backup can be made.

If the time and data transferred this requires isn't unreasonable for you
(perhaps it's a backup done at night), then simply write a quick script to
remove the local cache directory after the backup.

> I can't really use a new location because I have scripts to restore from these
> backups, having multiple would be a nightmare to work out actually where my
> backups are.

I don't understand what you mean. Multiple cache directories? You could just
change the cache directory to point to a local, secondary drive, if you have one
with sufficient space.

> My current AS3 bucket is ~50GB in size, I do an incremental backup nightly and
> a full backup after 1 month. Backups are removed once they are older than 6
> months.

How big is each full backup, and the average incremental backup after that?

> The /root/.cache/duplicity dir is 1.5GB in size.

That doesn't sound unreasonable to me, really. I mean, 1TB drives are fairly
cheap these days.


Cheers


Chris Poole
[PGP BAD246F9]