[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Duplicity-talk] failed to upload before termination, restarting, restor
From: |
Maximilian Bloch |
Subject: |
[Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch |
Date: |
Fri, 17 Apr 2015 12:46:19 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 |
Hi there,
I'm new here, nice to meet everyone. First off, I am pretty happy with
duplicity, which has been working for me for the past two years.
I have a current issue though and no solution. I searched through the
mailing-list archive and couldn't find anything to help me. Sorry if
this has been asked before.
I am doing automated daily backups:
duplicity \
--full-if-older-than 14D \
--encrypt-key ${GPG_KEY} \
--sign-key ${GPG_KEY} \
--exclude-regexp '\/te?mp\/' \
--exclude-regexp '\/cache\/' \
--include '/var/www' \
--exclude '**' \
/ s3+http://xxxx-var-www
My last full backup(s) seem to have failed (Backup taking longer than
24h and conflicting with the next backup script?), then resumed the next
day with further errors, resumed/restarted on the third day with no
errors. Restoring the backup is failing due to a corrupted volume (SHA1
hash mismatch for file ..vol907.difftar.gpg). I have noticed similar
issues with my last two full backups. My backup consists of 1043 volumes
à 25MB.
Does anybody have a solution to get the next backups to work properly?
bigger volume sizes? More time until script is run again?
Here are some details of duplicity output:
Full backup take 1:
Sun Apr 12, Sorry, no output, I must have trashed it because it
contained no errors.
Full backup take 2:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sun Apr 12 07:12:11 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Sun Apr 12 07:12:11 2015
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
RESTART: Volumes 907 to 908 failed to upload before termination.
Restarting backup at volume 907.
Restarting after volume 906, file var/www/xxxx.png, block 13
File duplicity-full.20150412T051211Z.vol914.difftar.gpg was corrupted
during upload.
Full backup take 3 (note it takes longer than 24h):
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 28 07:09:23 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 28 07:09:23 2015
Last full backup is too old, forcing full backup
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
--------------[ Backup Statistics ]--------------
StartTime 1428815542.46 (Sun Apr 12 07:12:22 2015)
EndTime 1428911965.18 (Mon Apr 13 09:59:25 2015)
ElapsedTime 96422.72 (26 hours 47 minutes 2.72 seconds)
SourceFiles 150833
SourceFileSize 28580154097 (26.6 GB)
NewFiles 150833
NewFileSize 28580154097 (26.6 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 150833
RawDeltaSize 28490687563 (26.5 GB)
TotalDestinationSizeChange 27385520785 (25.5 GB)
Errors 0
Restoring attempt:
[...]
Processed volume 908 of 1048
Invalid data - SHA1 hash mismatch for file:
duplicity-full.20150412T051211Z.vol907.difftar.gpg
Calculated hash: 3b332923613018d50d4b73d0b2831c72c36f4d74
Manifest hash: f9f8fa5e8bd5a07bb17228c83e13cf9f6f2adcdb
^ the restore script hangs here
Here is some output of my previous to last full backup attempt, which
also failed:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 14 06:29:20 2015
No old backup sets found, nothing deleted.
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Mar 14 06:29:20 2015
Last full backup is too old, forcing full backup
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
Upload
's3+http://xxxx-var-www/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
failed (attempt #1, reason: S3ResponseError: S3ResponseError: 400 Bad
Request
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did
not match what we
received.</Message><ExpectedDigest>oKyY+wDSBIKvBXzNFupGIQ==</ExpectedDigest><CalculatedDigest>zHtMZsbKINhRo9flUUjXPA==</CalculatedDigest><RequestId>4177190AB50613BA</RequestId><HostId>+hN7JpYX6auLQ63LegVOHtpxBk9ZHFfYU/92HucGXhFuIRKyScdWruXujslIFIb5dh1tfoeFcp8=</HostId></Error>)
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1404, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1397, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1367, in main
full_backup(col_stats)
File "/usr/bin/duplicity", line 506, in full_backup
sig_outfp.to_remote()
File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line
184, in to_remote
globals.backend.move(tgt) address@hidden
File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
364, in move
source_path.delete()
File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567,
in delete
util.ignore_missing(os.unlink, self.name)
File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116,
in ignore_missing
fn(filename)
OSError: [Errno 2] No such file or directory:
'/root/.cache/duplicity/16e0b913754120e86533128ea399d494/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
Thanks for any help.
Best,
Max
- [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch,
Maximilian Bloch <=
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, edgar . soldin, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, edgar . soldin, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/27
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Kenneth Loafman, 2015/04/27
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, edgar . soldin, 2015/04/27