duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Duplicity-talk] failed to upload before termination, restarting, restor


From: Maximilian Bloch
Subject: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch
Date: Fri, 17 Apr 2015 12:46:19 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0

Hi there,

I'm new here, nice to meet everyone. First off, I am pretty happy with
duplicity, which has been working for me for the past two years.

I have a current issue though and no solution. I searched through the
mailing-list archive and couldn't find anything to help me. Sorry if
this has been asked before.

I am doing automated daily backups:

duplicity \
    --full-if-older-than 14D \
    --encrypt-key ${GPG_KEY} \
    --sign-key ${GPG_KEY} \
    --exclude-regexp '\/te?mp\/' \
    --exclude-regexp '\/cache\/' \
    --include '/var/www' \
    --exclude '**' \
    / s3+http://xxxx-var-www


My last full backup(s) seem to have failed (Backup taking longer than
24h and conflicting with the next backup script?), then resumed the next
day with further errors, resumed/restarted on the third day with no
errors. Restoring the backup is failing due to a corrupted volume (SHA1
hash mismatch for file ..vol907.difftar.gpg). I have noticed similar
issues with my last two full backups. My backup consists of 1043 volumes
à 25MB.

Does anybody have a solution to get the next backups to work properly?
bigger volume sizes? More time until script is run again?

Here are some details of duplicity output:

Full backup take 1:
Sun Apr 12, Sorry, no output, I must have trashed it because it
contained no errors.

Full backup take 2:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sun Apr 12 07:12:11 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Sun Apr 12 07:12:11 2015
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
RESTART: Volumes 907 to 908 failed to upload before termination.
         Restarting backup at volume 907.
Restarting after volume 906, file var/www/xxxx.png, block 13
File duplicity-full.20150412T051211Z.vol914.difftar.gpg was corrupted
during upload.


Full backup take 3 (note it takes longer than 24h):

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 28 07:09:23 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 28 07:09:23 2015
Last full backup is too old, forcing full backup
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
--------------[ Backup Statistics ]--------------
StartTime 1428815542.46 (Sun Apr 12 07:12:22 2015)
EndTime 1428911965.18 (Mon Apr 13 09:59:25 2015)
ElapsedTime 96422.72 (26 hours 47 minutes 2.72 seconds)
SourceFiles 150833
SourceFileSize 28580154097 (26.6 GB)
NewFiles 150833
NewFileSize 28580154097 (26.6 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 150833
RawDeltaSize 28490687563 (26.5 GB)
TotalDestinationSizeChange 27385520785 (25.5 GB)
Errors 0


Restoring attempt:
[...]
Processed volume 908 of 1048
Invalid data - SHA1 hash mismatch for file:
 duplicity-full.20150412T051211Z.vol907.difftar.gpg
 Calculated hash: 3b332923613018d50d4b73d0b2831c72c36f4d74
 Manifest hash: f9f8fa5e8bd5a07bb17228c83e13cf9f6f2adcdb

^ the restore script hangs here


Here is some output of my previous to last full backup attempt, which
also failed:

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 14 06:29:20 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 14 06:29:20 2015
Last full backup is too old, forcing full backup
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
Upload
's3+http://xxxx-var-www/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
failed (attempt #1, reason: S3ResponseError: S3ResponseError: 400 Bad
Request
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did
not match what we
received.</Message><ExpectedDigest>oKyY+wDSBIKvBXzNFupGIQ==</ExpectedDigest><CalculatedDigest>zHtMZsbKINhRo9flUUjXPA==</CalculatedDigest><RequestId>4177190AB50613BA</RequestId><HostId>+hN7JpYX6auLQ63LegVOHtpxBk9ZHFfYU/92HucGXhFuIRKyScdWruXujslIFIb5dh1tfoeFcp8=</HostId></Error>)
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1404, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1397, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1367, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 506, in full_backup
    sig_outfp.to_remote()
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line
184, in to_remote
    globals.backend.move(tgt) address@hidden
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
364, in move
    source_path.delete()
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567,
in delete
    util.ignore_missing(os.unlink, self.name)
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116,
in ignore_missing
    fn(filename)
OSError: [Errno 2] No such file or directory:
'/root/.cache/duplicity/16e0b913754120e86533128ea399d494/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'


Thanks for any help.

Best,
Max



reply via email to

[Prev in Thread] Current Thread [Next in Thread]