[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Duplicity-talk] performance issue

From: Chris Chatelain
Subject: [Duplicity-talk] performance issue
Date: Thu, 31 Jul 2008 09:55:58 -0500
User-agent: Thunderbird (X11/20080505)

I've been using duplicity to back up to amazon S3 for about 3 months. I'm backing up nearly an entire RHEL5 server, excluding a few things. It used to take 70 minutes or so, and suddenly one day it jumped to taking 4 or 5 hours. The size of the files has only gone up from 32GB to about 36. Is there a way to see which part is taking long (ie. is it the amazon interaction taking so long, or is it collecting the information on the files, or is it the taring that's taking so long)? Now that I look at the output and back in my logs, it looks like it catches a socket error 3 times each run. Any other ideas what the problem might be?

My script looks like this:
export AWS_ACCESS_KEY_ID=#(insert your key_id)
export AWS_SECRET_ACCESS_KEY=#(insert access key)
echo "Backing up filesystem to Amazon S3"
time duplicity --exclude-device-files --exclude /sys --exclude /var/named/chroot/var/run --exclude /var/named/chroot/var/tmp --exclude /var/named/chroot/proc --exclude /var/named/chroot/dev --exclude /proc --exclude /tmp --no-encryption --allow-source-mismatch / s3+http://thewebshop-black-full
echo "Backup complete"
export AWS_ACCESS_KEY_ID= #clear out your id and key

and this is the output:

Thu Jul 31 05:31:15 CDT 2008

Backing up filesystem to Amazon S3
Caught a socket error, trying to recover
Caught a socket error, trying to recover
--------------[ Backup Statistics ]--------------
StartTime 1217501754.04 (Thu Jul 31 05:55:54 2008)
EndTime 1217511035.29 (Thu Jul 31 08:30:35 2008)
ElapsedTime 9281.25 (2 hours 34 minutes 41.25 seconds)
SourceFiles 859713
SourceFileSize 62674401579 (58.4 GB)
NewFiles 2509
NewFileSize 237279065 (226 MB)
DeletedFiles 1798
ChangedFiles 6062
ChangedFileSize 36115011699 (33.6 GB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 10369
RawDeltaSize 38133832 (36.4 MB)
TotalDestinationSizeChange 630377063 (601 MB)
Errors 0

real    179m44.075s
user    85m47.365s
sys     56m5.727s
Backup complete
Thu Jul 31 08:30:59 CDT 2008

reply via email to

[Prev in Thread] Current Thread [Next in Thread]