rdiff-backup-users
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] extremely slow backups off OS X


From: Kurt Yoder
Subject: Re: [rdiff-backup-users] extremely slow backups off OS X
Date: Wed, 12 Jul 2006 12:00:24 -0400


On Jul 12, 2006, at 11:41 AM, Kurt Yoder wrote:

Hello all

I've seen some posts regarding slow rdiff-backups, but no suggestions on speeding them up. I've been experimenting with this program for the last week, and it looks extremely useful. However, if I can't resolve the slowness problem I'm going to have to give up on it.

I was previously backing up one of my Mac OS X boxen using using straight rsync. Here is an example run, along with duration:

/var/backups/rsync_scripts/paprika
Mon Jul 10 01:48:10 EDT 2006

Number of files: 769005
Number of files transferred: 36932
Total file size: 27633230023 bytes
Total transferred file size: 398646044 bytes
Literal data: 33119070 bytes
Matched data: 593205243 bytes
File list size: 18580399
Total bytes sent: 2711991
Total bytes received: 34505287

sent 2711991 bytes  received 34505287 bytes  16821.37 bytes/sec
total size is 27633230023  speedup is 742.48
end:
Mon Jul 10 02:25:02 EDT 2006


So using straight rsync, I was able to do a relatively large backup in approximately 40 minutes. After switching to rdiff backup, the initial backup took over 24 hours. The subsequent backups are taking approximately 6 1/2 hours:

# less session_statistics.2006-07-11T11\:50\:58-04\:00.data
StartTime 1152633058.00 (Tue Jul 11 11:50:58 2006)
EndTime 1152656310.11 (Tue Jul 11 18:18:30 2006)
ElapsedTime 23252.07 (6 hours 27 minutes 32.07 seconds)
SourceFiles 734136
SourceFileSize 25594604054 (23.8 GB)
MirrorFiles 734139
MirrorFileSize 25597749516 (23.8 GB)
NewFiles 5
NewFileSize 63692 (62.2 KB)
DeletedFiles 8
DeletedFileSize 3415368 (3.26 MB)
ChangedFiles 109
ChangedSourceSize 404526962 (386 MB)
ChangedMirrorSize 404320748 (386 MB)
IncrementFiles 124
IncrementFileSize 43212042 (41.2 MB)
TotalDestinationSizeChange 40066580 (38.2 MB)
Errors 0


On the "pull" side, I'm running Debian Linux, with a "backported" rdiff-backup version 1.1.5 (package 1.1.5-0bpo1). On the On the "client" side, I'm running Mac OS 10.4.7 with rdiff-backup from fink, version 1.1.5 (fink package rdiff-backup_1.1.5-1).

Can someone offer debugging suggestions so I can figure out why this is taking so long? A 975% increase in backup duration for rdiff-backup over rsync seems excessive.


I forgot to include the backup commands I was using. Here is the one for rsync:

/usr/local/macrsync/bin/rsync -azE --stats --delete --delete-excluded paprika.rg.loc:/ /backup/rsync/paprika/ \
        --exclude "*.mov" \
        --exclude "*.mp3" \
        --exclude "*.mp4" \
        --exclude "/private/tmp/*" \
        --exclude "/private/var/run/*" \
        --exclude "/private/var/spool/postfix/*" \
        --exclude "/tmp/*" \
        --exclude "/proc" \
        --exclude "/dev/*" \
        --exclude "/afs/*" \
        --exclude "/cores/*" \
        --exclude "*/.Trash" \
        --exclude "/.vol/*" \
        --exclude "/Network/*" \
        --exclude "/automount/*" \
        --exclude "/Volumes/*" \
        --exclude "/Previous Systems.localized"

Here is the one for rdiff-backup:

/usr/bin/rdiff-backup -v0 --print-statistics \
        --exclude "**.mov" \
        --exclude "**.mp3" \
        --exclude "**.mp4" \
        --exclude "/private/tmp/*" \
        --exclude "/private/var/run/*" \
        --exclude "/private/var/spool/postfix/*" \
        --exclude "/tmp/*" \
        --exclude "/proc" \
        --exclude "/dev/*" \
        --exclude "/afs/*" \
        --exclude "/cores/*" \
        --exclude "**/.Trash" \
        --exclude "/.vol/*" \
        --exclude "/Network/*" \
        --exclude "/automount/*" \
        --exclude "/Volumes/*" \
        --exclude "/Previous Systems.localized" \
        paprika.rg.loc::/ /backup/rsync/paprika/





reply via email to

[Prev in Thread] Current Thread [Next in Thread]