[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] Severe performance degradation

From: Randy Syring
Subject: Re: [rdiff-backup-users] Severe performance degradation
Date: Tue, 21 Dec 2010 12:44:47 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101028 Thunderbird/3.1.6

Did you do testing on the local machine only ( i.e. do tests from one folder to another)? Doing comparisons on the older server and then the new server should show you if its just the server to blaim. If that proves solid, then I would look at network stuff. Maybe use a different NIC, etc. I know you said you did testing with --force that made it work faster and you therefore ruled out the network. But IMO, that isn't thorough enough, I want to remove the network completely from the equation. Your NIC could be having problems with something that only happens when doing the diff algorithm (or something else weird).

Maybe this isn't the right track at all, or maybe you have even tried this already, but its an idea anyway. :)

Randy Syring
Direct: 502-276-0459
Office: 502-212-9913

For the wages of sin is death, but the
free gift of God is eternal life in
Christ Jesus our Lord (Rom 6:23)

On 12/21/2010 12:25 PM, Maarten J H van den Berg wrote:
Hi there,

I'm looking for help with a very severe performance problem using
rdiff-backup. Basically we've bought a new server with more and faster
resources to replace a 4-year old one. However, rdiff-backup refuses to
perform on the new server.

Various tests show that generic disk accesses are much faster on the new
server, and it has more memory and faster CPU's.
Nevertheless we see a very severe slowdown when rdiff-backup is making
an incremental backup, up to four or even tenfold or more times slower
than it used to be before on the old server.

I gathered some numbers but they differ wildly depending on the source
material / dir. Maybe it is therefore better to leave specific numbers
for what they are for now and focus on the big picture: our old server
did an rdiff-backup of a remote storage server, worth some 300 GB, in
typically under an hour or so. The new server running with the same
source dataset typically starts at night and is still running the next
morning, into the afternoon even(!).

When we do trials on a tiny subset of the data we get varying results.
Some data takes eightfold the amount of time, some is within a +80%
margin. So that is not very dependable, alas.
Still, what is observable is that any initial backup run (with --force)
runs significantly faster on the new server. Any differential run
afterwards is slower than on the original server. I feel this proves
there are no performance bottlenecks in the network, disks, filesystems
etc of the server.

This is fully repeatable and a real time tail on the log file shows no
one file is to blame, it is just the overall speed that's slow.

The new server runs rdiff-backup 1.2.8, the old one 1.0.5. Downgrading
the new server to 1.0.5 makes things a bit interesting: that speeds it
up a bit, but still a fair bit slower than the original.

During investigation we experimented with different filesystems, testing
local versus remote backups, looking at compile flags and versions of
librsync and python, but we have had no success there.
All versions use librsync 0.9.7 All OS'es are Gentoo, 32 bit.

We did search for workarounds like spawning multiple parallel
rdiff-backup processes dealing each with separate directories so as to
fully use the eight CPU cores. Sadly even that speedup is still not
resulting in an acceptable overall speed. We compared compilation flags,
options and parameters but nothing obvious struck us in that regard.

I'm basically out of ideas. I was tearing my hair out over this until a
couple of days ago (yes, well, I'm bald now). I turn to this list as a
last resort. Can anyone help debugging this strange problem please ?

Regards, thanks for listening,


rdiff-backup-users mailing list at address@hidden
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

reply via email to

[Prev in Thread] Current Thread [Next in Thread]