Re: AW: [rdiff-backup-users] Severe performance degradation

[Maarten J H van den Berg]

D. Kriesel wrote:
> Hi maarten,
> 
> Some questions to help myself (and the list) helping:
> 
> 1) Which file systems on source and target did you use before and after the 
> server change? Which other ones did you try?
> 2) What kind of data do you have? Millions of dirs with millions of small 
> files or millions of files in just a few dirs or several very large files 
> (gigabytes) or all of them?

Okay, sorry, I should clarify a bit. By remote I meant it is coming over
the network, but it is not a client-server rdiff-backup setup. The
source comes from a Windows server 2008 share, mounted with cifs on the
two backup servers, the old one and the new one.
File systems on both backupserver are reiserfs. We had XFS as a test,
went for reiserfs in the end, to make both environments as equal as
possible.
Data is mostly 'office stuff' so anything ranging for spreadsheets,
photos, docs to your typical windows (profile) files. No source code, no
movies, no big drive snapshots. So basically both huge files and tiny
files are more or less scarce in this source tree. total number of files
runs in the high 6 digits, but definitely not more than a million.

>> Still, what is observable is that any initial backup run (with --force)
>> runs significantly faster on the new server.
> 
> Think so, too.
> 
>>  Any differential run
>> afterwards is slower than on the original server. I feel this proves
>> there are no performance bottlenecks in the network, disks, filesystems
>> etc of the server.
> 
> 
>> The new server runs rdiff-backup 1.2.8, the old one 1.0.5.
> 
> Did you upgrade the remote server too?

Sorry was not clear: no remote rdiff-backup, just a network-mounted share.

>>  Downgrading
>> the new server to 1.0.5 makes things a bit interesting: that speeds it
>> up a bit, but still a fair bit slower than the original.
> 
> As of Rdiff-backup 1.1.1, hashing is done with sha-1, maybe this slows down a 
> bit. Source: http://rdiff-backup.nongnu.org/CHANGELOG-stable

That's interesting.

>> During investigation we experimented with different filesystems, testing
>> local versus remote backups, looking at compile flags and versions of
>> librsync and python, but we have had no success there.
> 
> Did you try reiserfs (nice with lots and lots of small files)?

Certainly. It was our first choice, because of that.

Regards,
Maarten