Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch

[edgar . soldin]

On 17.04.2015 12:46, Maximilian Bloch wrote:
> Hi there,
> 
> I'm new here, nice to meet everyone. First off, I am pretty happy with
> duplicity, which has been working for me for the past two years.
> 
> I have a current issue though and no solution. I searched through the
> mailing-list archive and couldn't find anything to help me. Sorry if
> this has been asked before.
> 
> I am doing automated daily backups:
> 
> duplicity \
>     --full-if-older-than 14D \
>     --encrypt-key ${GPG_KEY} \
>     --sign-key ${GPG_KEY} \
>     --exclude-regexp '\/te?mp\/' \
>     --exclude-regexp '\/cache\/' \
>     --include '/var/www' \
>     --exclude '**' \
>     / s3+http://xxxx-var-www
> 
> 
> My last full backup(s) seem to have failed (Backup taking longer than
> 24h and conflicting with the next backup script?), then resumed the next
> day with further errors, resumed/restarted on the third day with no
> errors. Restoring the backup is failing due to a corrupted volume (SHA1
> hash mismatch for file ..vol907.difftar.gpg). I have noticed similar
> issues with my last two full backups. My backup consists of 1043 volumes
> à 25MB.
> 
> Does anybody have a solution to get the next backups to work properly?
> bigger volume sizes? More time until script is run again?
> 
> Here are some details of duplicity output:
> 
> Full backup take 1:
> Sun Apr 12, Sorry, no output, I must have trashed it because it
> contained no errors.
> 
> Full backup take 2:
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sun Apr 12 07:12:11 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup left a partial set, restarting.
> Last full backup date: Sun Apr 12 07:12:11 2015
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> RESTART: Volumes 907 to 908 failed to upload before termination.
>          Restarting backup at volume 907.
> Restarting after volume 906, file var/www/xxxx.png, block 13
> File duplicity-full.20150412T051211Z.vol914.difftar.gpg was corrupted
> during upload.
> 
> 
> Full backup take 3 (note it takes longer than 24h):
> 
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 28 07:09:23 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 28 07:09:23 2015
> Last full backup is too old, forcing full backup
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> --------------[ Backup Statistics ]--------------
> StartTime 1428815542.46 (Sun Apr 12 07:12:22 2015)
> EndTime 1428911965.18 (Mon Apr 13 09:59:25 2015)
> ElapsedTime 96422.72 (26 hours 47 minutes 2.72 seconds)
> SourceFiles 150833
> SourceFileSize 28580154097 (26.6 GB)
> NewFiles 150833
> NewFileSize 28580154097 (26.6 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 150833
> RawDeltaSize 28490687563 (26.5 GB)
> TotalDestinationSizeChange 27385520785 (25.5 GB)
> Errors 0
> 
> 
> Restoring attempt:
> [...]
> Processed volume 908 of 1048
> Invalid data - SHA1 hash mismatch for file:
>  duplicity-full.20150412T051211Z.vol907.difftar.gpg
>  Calculated hash: 3b332923613018d50d4b73d0b2831c72c36f4d74
>  Manifest hash: f9f8fa5e8bd5a07bb17228c83e13cf9f6f2adcdb
> 
> ^ the restore script hangs here
> 
> 
> Here is some output of my previous to last full backup attempt, which
> also failed:
> 
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 14 06:29:20 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 14 06:29:20 2015
> Last full backup is too old, forcing full backup
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> Upload
> 's3+http://xxxx-var-www/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
> failed (attempt #1, reason: S3ResponseError: S3ResponseError: 400 Bad
> Request
> <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did
> not match what we
> received.</Message><ExpectedDigest>oKyY+wDSBIKvBXzNFupGIQ==</ExpectedDigest><CalculatedDigest>zHtMZsbKINhRo9flUUjXPA==</CalculatedDigest><RequestId>4177190AB50613BA</RequestId><HostId>+hN7JpYX6auLQ63LegVOHtpxBk9ZHFfYU/92HucGXhFuIRKyScdWruXujslIFIb5dh1tfoeFcp8=</HostId></Error>)
> Traceback (most recent call last):
>   File "/usr/bin/duplicity", line 1404, in <module>
>     with_tempdir(main)
>   File "/usr/bin/duplicity", line 1397, in with_tempdir
>     fn()
>   File "/usr/bin/duplicity", line 1367, in main
>     full_backup(col_stats)
>   File "/usr/bin/duplicity", line 506, in full_backup
>     sig_outfp.to_remote()
>   File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line
> 184, in to_remote
>     globals.backend.move(tgt) address@hidden
>   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 364, in move
>     source_path.delete()
>   File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567,
> in delete
>     util.ignore_missing(os.unlink, self.name)
>   File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116,
> in ignore_missing
>     fn(filename)
> OSError: [Errno 2] No such file or directory:
> '/root/.cache/duplicity/16e0b913754120e86533128ea399d494/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
> 
> 

what's your duplicity version? ..ede/duply.net