[Duplicity-talk] problems running duplicity

[Jelle de Jong]

Hello everybody,

I got a list of issues on my duplicity systems. The backup site has
been turned off for a few days, and now the remote machine is running
again, but duplicity on the main machine is having some troubles.

Please see the attachments for the duplicity commands and its output.

- Remote file or directory does not exist in command...
I got this error every run and I don't know why duplicity is not
recovering from the event?

- GPGError: GPG Failed, see log below...
One of the backup commands is failing because it wants the secret
master key. I setup my system that the secret master key is not
available on any system for security! It should only be used for
recovery of data from the backups. I tested this and it worked fine
until now.

Does somebody know what I am doing wrong? Any help is appreciated.

Thanks in advance,

Kind regards,

Jelle

phase 0: remove-older-than: home:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Mar  8 23:30:25 2010
No old backup sets found, nothing deleted.
phase 1: remove-older-than: vmail:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
duplicity-new-signatures.20100208T223035Z.to.20100209T223042Z.sigtar.gpg
duplicity-new-signatures.20100209T223042Z.to.20100210T223033Z.sigtar.gpg
duplicity-new-signatures.20100210T223033Z.to.20100211T223040Z.sigtar.gpg
duplicity-new-signatures.20100211T223040Z.to.20100212T223033Z.sigtar.gpg
duplicity-new-signatures.20100212T223033Z.to.20100213T223037Z.sigtar.gpg
duplicity-new-signatures.20100213T223037Z.to.20100214T223033Z.sigtar.gpg
duplicity-new-signatures.20100214T223033Z.to.20100215T223037Z.sigtar.gpg
duplicity-new-signatures.20100215T223037Z.to.20100216T223037Z.sigtar.gpg
duplicity-new-signatures.20100216T223037Z.to.20100217T223037Z.sigtar.gpg
duplicity-new-signatures.20100217T223037Z.to.20100218T223039Z.sigtar.gpg
duplicity-new-signatures.20100218T223039Z.to.20100219T230500Z.sigtar.gpg
duplicity-new-signatures.20100219T230500Z.to.20100311T171314Z.sigtar.gpg
duplicity-new-signatures.20100311T171314Z.to.20100311T172255Z.sigtar.gpg
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part]
Last full backup date: Fri Mar 12 23:30:52 2010
No old backup sets found, nothing deleted.
phase 2: remove-older-than: root:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part]
Last full backup date: Mon Feb  1 21:42:18 2010
No old backup sets found, nothing deleted.
phase 3: cleanup: home:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Mar  8 23:30:25 2010
1268087425 1268087425 1268692502 1268692502
1268087425 1268087425 1268692502 1268692502
No extraneous files found, nothing deleted in cleanup.
phase 4: cleanup: vmail:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
duplicity-new-signatures.20100208T223035Z.to.20100209T223042Z.sigtar.gpg
duplicity-new-signatures.20100209T223042Z.to.20100210T223033Z.sigtar.gpg
duplicity-new-signatures.20100210T223033Z.to.20100211T223040Z.sigtar.gpg
duplicity-new-signatures.20100211T223040Z.to.20100212T223033Z.sigtar.gpg
duplicity-new-signatures.20100212T223033Z.to.20100213T223037Z.sigtar.gpg
duplicity-new-signatures.20100213T223037Z.to.20100214T223033Z.sigtar.gpg
duplicity-new-signatures.20100214T223033Z.to.20100215T223037Z.sigtar.gpg
duplicity-new-signatures.20100215T223037Z.to.20100216T223037Z.sigtar.gpg
duplicity-new-signatures.20100216T223037Z.to.20100217T223037Z.sigtar.gpg
duplicity-new-signatures.20100217T223037Z.to.20100218T223039Z.sigtar.gpg
duplicity-new-signatures.20100218T223039Z.to.20100219T230500Z.sigtar.gpg
duplicity-new-signatures.20100219T230500Z.to.20100311T171314Z.sigtar.gpg
duplicity-new-signatures.20100311T171314Z.to.20100311T172255Z.sigtar.gpg
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part]
Last full backup date: Fri Mar 12 23:30:52 2010
1265103633 1268433052 1265103633 1268692531
1265103633 1268433052 1268346661 1268692531
1268433052 1268433052 1268692531 1268692531
1268433052 1268433052 1268692531 1268692531
Deleting these files from backend:
duplicity-full-signatures.20100202T094033Z.sigtar.gz
duplicity-new-signatures.20100202T094033Z.to.20100203T223035Z.sigtar.part
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gz
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gz
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gz
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gz
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gz
duplicity-new-signatures.20100208T223035Z.to.20100209T223042Z.sigtar.gz
duplicity-new-signatures.20100209T223042Z.to.20100210T223033Z.sigtar.gz
duplicity-new-signatures.20100210T223033Z.to.20100211T223040Z.sigtar.gz
duplicity-new-signatures.20100211T223040Z.to.20100212T223033Z.sigtar.gz
duplicity-new-signatures.20100212T223033Z.to.20100213T223037Z.sigtar.gz
duplicity-new-signatures.20100213T223037Z.to.20100214T223033Z.sigtar.gz
duplicity-new-signatures.20100214T223033Z.to.20100215T223037Z.sigtar.gz
duplicity-new-signatures.20100215T223037Z.to.20100216T223037Z.sigtar.gz
duplicity-new-signatures.20100216T223037Z.to.20100217T223037Z.sigtar.gz
duplicity-new-signatures.20100217T223037Z.to.20100218T223039Z.sigtar.gz
duplicity-new-signatures.20100218T223039Z.to.20100219T230500Z.sigtar.gz
duplicity-new-signatures.20100219T230500Z.to.20100311T171314Z.sigtar.gz
duplicity-new-signatures.20100311T171314Z.to.20100311T172255Z.sigtar.gz
duplicity-new-signatures.20100311T172255Z.to.20100311T173524Z.sigtar.part
duplicity-new-signatures.20100311T173524Z.to.20100311T223101Z.sigtar.part
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part
duplicity-full-signatures.20100202T094033Z.sigtar.gpg
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
duplicity-new-signatures.20100208T223035Z.to.20100209T223042Z.sigtar.gpg
duplicity-new-signatures.20100209T223042Z.to.20100210T223033Z.sigtar.gpg
duplicity-new-signatures.20100210T223033Z.to.20100211T223040Z.sigtar.gpg
duplicity-new-signatures.20100211T223040Z.to.20100212T223033Z.sigtar.gpg
duplicity-new-signatures.20100212T223033Z.to.20100213T223037Z.sigtar.gpg
duplicity-new-signatures.20100213T223037Z.to.20100214T223033Z.sigtar.gpg
duplicity-new-signatures.20100214T223033Z.to.20100215T223037Z.sigtar.gpg
duplicity-new-signatures.20100215T223037Z.to.20100216T223037Z.sigtar.gpg
duplicity-new-signatures.20100216T223037Z.to.20100217T223037Z.sigtar.gpg
duplicity-new-signatures.20100217T223037Z.to.20100218T223039Z.sigtar.gpg
duplicity-new-signatures.20100218T223039Z.to.20100219T230500Z.sigtar.gpg
duplicity-new-signatures.20100219T230500Z.to.20100311T171314Z.sigtar.gpg
duplicity-new-signatures.20100311T171314Z.to.20100311T172255Z.sigtar.gpg
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #1)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #2)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #3)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #4)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #5)
Giving up trying to execute 'sftp  -oServerAliveInterval=15 
-oServerAliveCountMax=1 address@hidden' after 5 attempts
BackendException: Error running 'sftp  -oServerAliveInterval=15 
-oServerAliveCountMax=1 address@hidden'
phase 5: cleanup: root:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part]
Last full backup date: Mon Feb  1 21:42:18 2010
1265056938 1265056938 1268332500 1268332500
1265056938 1265056938 1268346796 1268332500
Deleting these files from backend:
duplicity-full-signatures.20100201T204218Z.sigtar.gz
duplicity-new-signatures.20100201T204218Z.to.20100203T052717Z.sigtar.gz
duplicity-new-signatures.20100203T052717Z.to.20100205T061530Z.sigtar.gz
duplicity-new-signatures.20100205T061530Z.to.20100206T033316Z.sigtar.gz
duplicity-new-signatures.20100206T033316Z.to.20100207T033328Z.sigtar.gz
duplicity-new-signatures.20100207T033328Z.to.20100208T033259Z.sigtar.gz
duplicity-new-signatures.20100208T033259Z.to.20100209T033250Z.sigtar.gz
duplicity-new-signatures.20100209T033250Z.to.20100210T033255Z.sigtar.gz
duplicity-new-signatures.20100210T033255Z.to.20100210T223257Z.sigtar.gz
duplicity-new-signatures.20100210T223257Z.to.20100211T223410Z.sigtar.gz
duplicity-new-signatures.20100211T223410Z.to.20100212T223344Z.sigtar.gz
duplicity-new-signatures.20100212T223344Z.to.20100213T223310Z.sigtar.gz
duplicity-new-signatures.20100213T223310Z.to.20100214T223628Z.sigtar.gz
duplicity-new-signatures.20100214T223628Z.to.20100215T223754Z.sigtar.gz
duplicity-new-signatures.20100215T223754Z.to.20100216T223351Z.sigtar.gz
duplicity-new-signatures.20100216T223351Z.to.20100217T223321Z.sigtar.gz
duplicity-new-signatures.20100217T223321Z.to.20100218T223349Z.sigtar.gz
duplicity-new-signatures.20100218T223349Z.to.20100219T230952Z.sigtar.gz
duplicity-new-signatures.20100219T230952Z.to.20100311T173524Z.sigtar.gz
duplicity-new-signatures.20100311T173524Z.to.20100311T183500Z.sigtar.gz
duplicity-new-signatures.20100311T183500Z.to.20100311T184715Z.sigtar.part
duplicity-new-signatures.20100311T184715Z.to.20100311T223316Z.sigtar.part
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #1)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #2)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #3)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #4)
Remote file or directory does not exist in command='rm 
duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=1 address@hidden' 
failed (attempt #5)
Giving up trying to execute 'sftp  -oServerAliveInterval=15 
-oServerAliveCountMax=1 address@hidden' after 5 attempts
BackendException: Error running 'sftp  -oServerAliveInterval=15 
-oServerAliveCountMax=1 address@hidden'
phase 6: full-if-older-than: home:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Mar  8 23:30:25 2010
--------------[ Backup Statistics ]--------------
StartTime 1268778913.10 (Tue Mar 16 23:35:13 2010)
EndTime 1268778921.03 (Tue Mar 16 23:35:21 2010)
ElapsedTime 7.94 (7.94 seconds)
SourceFiles 9616
SourceFileSize 7934971109 (7.39 GB)
NewFiles 4
NewFileSize 108881 (106 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 4
RawDeltaSize 100689 (98.3 KB)
TotalDestinationSizeChange 98062 (95.8 KB)
Errors 0
-------------------------------------------------

phase 7: full-if-older-than: vmail:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
duplicity-new-signatures.20100208T223035Z.to.20100209T223042Z.sigtar.gpg
duplicity-new-signatures.20100209T223042Z.to.20100210T223033Z.sigtar.gpg
duplicity-new-signatures.20100210T223033Z.to.20100211T223040Z.sigtar.gpg
duplicity-new-signatures.20100211T223040Z.to.20100212T223033Z.sigtar.gpg
duplicity-new-signatures.20100212T223033Z.to.20100213T223037Z.sigtar.gpg
duplicity-new-signatures.20100213T223037Z.to.20100214T223033Z.sigtar.gpg
duplicity-new-signatures.20100214T223033Z.to.20100215T223037Z.sigtar.gpg
duplicity-new-signatures.20100215T223037Z.to.20100216T223037Z.sigtar.gpg
duplicity-new-signatures.20100216T223037Z.to.20100217T223037Z.sigtar.gpg
duplicity-new-signatures.20100217T223037Z.to.20100218T223039Z.sigtar.gpg
duplicity-new-signatures.20100218T223039Z.to.20100219T230500Z.sigtar.gpg
duplicity-new-signatures.20100219T230500Z.to.20100311T171314Z.sigtar.gpg
duplicity-new-signatures.20100311T171314Z.to.20100311T172255Z.sigtar.gpg
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T173524Z.to.20100311T223101Z.manifest.part]
Last full backup date: Fri Mar 12 23:30:52 2010
--------------[ Backup Statistics ]--------------
StartTime 1268778937.84 (Tue Mar 16 23:35:37 2010)
EndTime 1268778946.91 (Tue Mar 16 23:35:46 2010)
ElapsedTime 9.07 (9.07 seconds)
SourceFiles 16374
SourceFileSize 1565322699 (1.46 GB)
NewFiles 143
NewFileSize 5820763 (5.55 MB)
DeletedFiles 14
ChangedFiles 27
ChangedFileSize 29655691 (28.3 MB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 184
RawDeltaSize 12705751 (12.1 MB)
TotalDestinationSizeChange 9151956 (8.73 MB)
Errors 0
-------------------------------------------------

phase 8: full-if-older-than: root:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following orphaned backup file:
[duplicity-inc.20100311T184715Z.to.20100311T223316Z.manifest.part]
Last full backup date: Mon Feb  1 21:42:18 2010
GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: encrypted with 2048-bit RSA key, ID 038F355D, created 2010-01-30
"example B.V. Backup System (Master Encryption Key) <address@hidden>"
gpg: decryption failed: secret key not available
===== End GnuPG log =====

phase 9: diskfree: home:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb1             917G   27G  844G   4% /home

export PASSPHRASE=""

echo "phase 0: remove-older-than: home: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/home

echo "phase 1: remove-older-than: vmail: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/vmail

echo "phase 2: remove-older-than: root: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/root

echo "phase 3: cleanup: home: "
duplicity cleanup --extra-clean --force \
scp://address@hidden/backup/home

echo "phase 4: cleanup: vmail: "
duplicity cleanup --extra-clean --force \
scp://address@hidden/backup/vmail

echo "phase 5: cleanup: root: "
duplicity cleanup --extra-clean --force \
scp://address@hidden/backup/root

echo "phase 6: full-if-older-than: home: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 30D \
--exclude /home/management/backup \
--exclude /home/vmail \
--exclude /home/system01-backup \
--scp-command "scp -C -c blowfish -l 320" \
/home scp://address@hidden/backup/home

echo "phase 7: full-if-older-than: vmail: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 38D \
--exclude /home/vmail/system/backup/example.com \
--scp-command "scp -C -c blowfish -l 320" \
/home/vmail scp://address@hidden/backup/vmail

echo "phase 8: full-if-older-than: root: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 46D \
--exclude /proc --exclude /mnt --exclude /tmp --exclude /sys \
--exclude /dev --exclude /home \
--scp-command "scp -C -c blowfish -l 320" \
/ scp://address@hidden/backup/root

echo "phase 9: diskfree: home: "
ssh address@hidden df -h /home

exit