[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [rdiff-backup-users] extremely slow backups off OS X
From: |
Kurt Yoder |
Subject: |
Re: [rdiff-backup-users] extremely slow backups off OS X |
Date: |
Wed, 12 Jul 2006 12:00:24 -0400 |
On Jul 12, 2006, at 11:41 AM, Kurt Yoder wrote:
Hello all
I've seen some posts regarding slow rdiff-backups, but no
suggestions on speeding them up. I've been experimenting with this
program for the last week, and it looks extremely useful. However,
if I can't resolve the slowness problem I'm going to have to give
up on it.
I was previously backing up one of my Mac OS X boxen using using
straight rsync. Here is an example run, along with duration:
/var/backups/rsync_scripts/paprika
Mon Jul 10 01:48:10 EDT 2006
Number of files: 769005
Number of files transferred: 36932
Total file size: 27633230023 bytes
Total transferred file size: 398646044 bytes
Literal data: 33119070 bytes
Matched data: 593205243 bytes
File list size: 18580399
Total bytes sent: 2711991
Total bytes received: 34505287
sent 2711991 bytes received 34505287 bytes 16821.37 bytes/sec
total size is 27633230023 speedup is 742.48
end:
Mon Jul 10 02:25:02 EDT 2006
So using straight rsync, I was able to do a relatively large backup
in approximately 40 minutes. After switching to rdiff backup, the
initial backup took over 24 hours. The subsequent backups are
taking approximately 6 1/2 hours:
# less session_statistics.2006-07-11T11\:50\:58-04\:00.data
StartTime 1152633058.00 (Tue Jul 11 11:50:58 2006)
EndTime 1152656310.11 (Tue Jul 11 18:18:30 2006)
ElapsedTime 23252.07 (6 hours 27 minutes 32.07 seconds)
SourceFiles 734136
SourceFileSize 25594604054 (23.8 GB)
MirrorFiles 734139
MirrorFileSize 25597749516 (23.8 GB)
NewFiles 5
NewFileSize 63692 (62.2 KB)
DeletedFiles 8
DeletedFileSize 3415368 (3.26 MB)
ChangedFiles 109
ChangedSourceSize 404526962 (386 MB)
ChangedMirrorSize 404320748 (386 MB)
IncrementFiles 124
IncrementFileSize 43212042 (41.2 MB)
TotalDestinationSizeChange 40066580 (38.2 MB)
Errors 0
On the "pull" side, I'm running Debian Linux, with a "backported"
rdiff-backup version 1.1.5 (package 1.1.5-0bpo1). On the On the
"client" side, I'm running Mac OS 10.4.7 with rdiff-backup from
fink, version 1.1.5 (fink package rdiff-backup_1.1.5-1).
Can someone offer debugging suggestions so I can figure out why
this is taking so long? A 975% increase in backup duration for
rdiff-backup over rsync seems excessive.
I forgot to include the backup commands I was using. Here is the one
for rsync:
/usr/local/macrsync/bin/rsync -azE --stats --delete --delete-excluded
paprika.rg.loc:/ /backup/rsync/paprika/ \
--exclude "*.mov" \
--exclude "*.mp3" \
--exclude "*.mp4" \
--exclude "/private/tmp/*" \
--exclude "/private/var/run/*" \
--exclude "/private/var/spool/postfix/*" \
--exclude "/tmp/*" \
--exclude "/proc" \
--exclude "/dev/*" \
--exclude "/afs/*" \
--exclude "/cores/*" \
--exclude "*/.Trash" \
--exclude "/.vol/*" \
--exclude "/Network/*" \
--exclude "/automount/*" \
--exclude "/Volumes/*" \
--exclude "/Previous Systems.localized"
Here is the one for rdiff-backup:
/usr/bin/rdiff-backup -v0 --print-statistics \
--exclude "**.mov" \
--exclude "**.mp3" \
--exclude "**.mp4" \
--exclude "/private/tmp/*" \
--exclude "/private/var/run/*" \
--exclude "/private/var/spool/postfix/*" \
--exclude "/tmp/*" \
--exclude "/proc" \
--exclude "/dev/*" \
--exclude "/afs/*" \
--exclude "/cores/*" \
--exclude "**/.Trash" \
--exclude "/.vol/*" \
--exclude "/Network/*" \
--exclude "/automount/*" \
--exclude "/Volumes/*" \
--exclude "/Previous Systems.localized" \
paprika.rg.loc::/ /backup/rsync/paprika/