Timothee Besset schrieb:
I'm finding that some systems that were backed up by duplicity are
running out of disk space after I upgraded the duplicity version. More
specifically, I am seeing 19GB worth of data in a ~/.cache/duplicity folder?
I can't afford to have 20GB of disk occupied by my backup software. I
haven't found much information on the nature of that directory, except
that it's an addition in 0.6? Should it really be growing that big? Is
there a limit to how much it'll grow? Is it a case of backup biting it's
own tail like we had backups of /tmp trying to backup duplicity's own
temporary files at one point?
The problem seems to be, that duplicity does not cleanup signature-files
when removing old backups, so the signatures occupy more and more space.
To reproduce, I did some backups (locally), some with a forced full
backup and some incementals. Afterwards the backup looks like this:
$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93
Found 2 secondary backup chains.
Secondary chain 1 of 2:
-------------------------
Chain start time: Wed Feb 10 16:50:16 2010
Chain end time: Wed Feb 10 16:50:44 2010
Number of contained backup sets: 4
Total number of contained volumes: 4
Type of backup set: Time: Num volumes:
Full Wed Feb 10 16:50:16 2010 1
Incremental Wed Feb 10 16:50:36 2010 1
Incremental Wed Feb 10 16:50:41 2010 1
Incremental Wed Feb 10 16:50:44 2010 1
-------------------------
Secondary chain 2 of 2:
-------------------------
Chain start time: Wed Feb 10 16:51:01 2010
Chain end time: Wed Feb 10 16:51:39 2010
Number of contained backup sets: 4
Total number of contained volumes: 4
Type of backup set: Time: Num volumes:
Full Wed Feb 10 16:51:01 2010 1
Incremental Wed Feb 10 16:51:36 2010 1
Incremental Wed Feb 10 16:51:38 2010 1
Incremental Wed Feb 10 16:51:39 2010 1
-------------------------
Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
Full Wed Feb 10 16:51:46 2010 1
Incremental Wed Feb 10 16:52:04 2010 1
-------------------------
No orphaned or incomplete backup sets found.
$ ls /tmp/duptest/
duplicity-full.20100210T155016Z.manifest.gpg
duplicity-full.20100210T155016Z.vol1.difftar.gpg
duplicity-full.20100210T155101Z.manifest.gpg
duplicity-full.20100210T155101Z.vol1.difftar.gpg
duplicity-full.20100210T155146Z.manifest.gpg
duplicity-full.20100210T155146Z.vol1.difftar.gpg
duplicity-full-signatures.20100210T155016Z.sigtar.gpg
duplicity-full-signatures.20100210T155101Z.sigtar.gpg
duplicity-full-signatures.20100210T155146Z.sigtar.gpg
duplicity-inc.20100210T155016Z.to.20100210T155036Z.manifest.gpg
duplicity-inc.20100210T155016Z.to.20100210T155036Z.vol1.difftar.gpg
duplicity-inc.20100210T155036Z.to.20100210T155041Z.manifest.gpg
duplicity-inc.20100210T155036Z.to.20100210T155041Z.vol1.difftar.gpg
duplicity-inc.20100210T155041Z.to.20100210T155044Z.manifest.gpg
duplicity-inc.20100210T155041Z.to.20100210T155044Z.vol1.difftar.gpg
duplicity-inc.20100210T155101Z.to.20100210T155136Z.manifest.gpg
duplicity-inc.20100210T155101Z.to.20100210T155136Z.vol1.difftar.gpg
duplicity-inc.20100210T155136Z.to.20100210T155138Z.manifest.gpg
duplicity-inc.20100210T155136Z.to.20100210T155138Z.vol1.difftar.gpg
duplicity-inc.20100210T155138Z.to.20100210T155139Z.manifest.gpg
duplicity-inc.20100210T155138Z.to.20100210T155139Z.vol1.difftar.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.manifest.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.vol1.difftar.gpg
duplicity-new-signatures.20100210T155016Z.to.20100210T155036Z.sigtar.gpg
duplicity-new-signatures.20100210T155036Z.to.20100210T155041Z.sigtar.gpg
duplicity-new-signatures.20100210T155041Z.to.20100210T155044Z.sigtar.gpg
duplicity-new-signatures.20100210T155101Z.to.20100210T155136Z.sigtar.gpg
duplicity-new-signatures.20100210T155136Z.to.20100210T155138Z.sigtar.gpg
duplicity-new-signatures.20100210T155138Z.to.20100210T155139Z.sigtar.gpg
duplicity-new-signatures.20100210T155146Z.to.20100210T155204Z.sigtar.gpg
Now I remove the old backups:
$ PASSPHRASE="foo" duplicity remove-all-but-n-full 1 --force
file:///tmp/duptest
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Feb 10 16:51:46 2010
Deleting backup sets at times:
Wed Feb 10 16:50:16 2010
Wed Feb 10 16:50:36 2010
Wed Feb 10 16:50:41 2010
Wed Feb 10 16:50:44 2010
Wed Feb 10 16:51:01 2010
Wed Feb 10 16:51:36 2010
Wed Feb 10 16:51:38 2010
Wed Feb 10 16:51:39 2010
Warning, found the following local orphaned signature files:
duplicity-new-signatures.20100210T155016Z.to.20100210T155036Z.sigtar.gz
duplicity-new-signatures.20100210T155036Z.to.20100210T155041Z.sigtar.gz
duplicity-new-signatures.20100210T155041Z.to.20100210T155044Z.sigtar.gz
duplicity-new-signatures.20100210T155101Z.to.20100210T155136Z.sigtar.gz
duplicity-new-signatures.20100210T155136Z.to.20100210T155138Z.sigtar.gz
duplicity-new-signatures.20100210T155138Z.to.20100210T155139Z.sigtar.gz
Note the "locally orphaned files".
When I look at the status:
$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20100210T155016Z.sigtar to local cache.
Copying duplicity-full-signatures.20100210T155101Z.sigtar to local cache.
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93
Found 0 secondary backup chains.
Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
Full Wed Feb 10 16:51:46 2010 1
Incremental Wed Feb 10 16:52:04 2010 1
-------------------------
No orphaned or incomplete backup sets found.
Note that duplicity did fetch the signatures of the 2 old (deleted!)
full-backups
When looking at the target, it becomes clear that "duplicity remove..."
did only remove the data (*vol*difftar-files), not the obsoleted signatures:
$ ls /tmp/duptest/
duplicity-full.20100210T155146Z.manifest.gpg
duplicity-full.20100210T155146Z.vol1.difftar.gpg
duplicity-full-signatures.20100210T155016Z.sigtar.gpg
duplicity-full-signatures.20100210T155101Z.sigtar.gpg
duplicity-full-signatures.20100210T155146Z.sigtar.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.manifest.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.vol1.difftar.gpg
duplicity-new-signatures.20100210T155016Z.to.20100210T155036Z.sigtar.gpg
duplicity-new-signatures.20100210T155036Z.to.20100210T155041Z.sigtar.gpg
duplicity-new-signatures.20100210T155041Z.to.20100210T155044Z.sigtar.gpg
duplicity-new-signatures.20100210T155101Z.to.20100210T155136Z.sigtar.gpg
duplicity-new-signatures.20100210T155136Z.to.20100210T155138Z.sigtar.gpg
duplicity-new-signatures.20100210T155138Z.to.20100210T155139Z.sigtar.gpg
duplicity-new-signatures.20100210T155146Z.to.20100210T155204Z.sigtar.gpg
These files are also present (in decrypted form) in the cache-dir. And
they stay there (or are refetched from the backup-target).
Only workaround: delete the unneeded signature-files on the target, then
duplicity also removes them from the cache (but be carefull to only
remove old and unneeded files):
$ cd /tmp/duptest/
:/tmp/duptest$ rm duplicity-full-signatures.20100210T155016Z.sigtar.gpg
duplicity-full-signatures.20100210T155101Z.sigtar.gpg
duplicity-new-signatures.20100210T1550*
duplicity-new-signatures.20100210T155101Z.to.20100210T155136Z.sigtar.gpg
duplicity-new-signatures.20100210T15513*
:/tmp/duptest$ ls
duplicity-full.20100210T155146Z.manifest.gpg
duplicity-full.20100210T155146Z.vol1.difftar.gpg
duplicity-full-signatures.20100210T155146Z.sigtar.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.manifest.gpg
duplicity-inc.20100210T155146Z.to.20100210T155204Z.vol1.difftar.gpg
duplicity-new-signatures.20100210T155146Z.to.20100210T155204Z.sigtar.gpg
:/tmp/duptest$ cd -
~$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Synchronizing remote metadata to local cache...
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-full-signatures.20100210T155016Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-full-signatures.20100210T155101Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155016Z.to.20100210T155036Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155036Z.to.20100210T155041Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155041Z.to.20100210T155044Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155101Z.to.20100210T155136Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155136Z.to.20100210T155138Z.sigtar.gz
(not authoritative at backend).
Deleting local
/home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93/duplicity-new-signatures.20100210T155138Z.to.20100210T155139Z.sigtar.gz
(not authoritative at backend).
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/.cache/duplicity/dfd6cdfea9f4dc13e6e57c08d55d2a93
Found 0 secondary backup chains.
Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
Full Wed Feb 10 16:51:46 2010 1
Incremental Wed Feb 10 16:52:04 2010 1
-------------------------
No orphaned or incomplete backup sets found.
(Of course, one has to do full-backups from time to time in the first
place, when you only ever do incrementals, ALL signature-files are still
needed)
cu
Tim
_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk