[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Rdiff-backup-bugs] [bug #22183] Feature request - speed improvements at
From: |
Guru Evi |
Subject: |
[Rdiff-backup-bugs] [bug #22183] Feature request - speed improvements at cost of integrity for large systems |
Date: |
Wed, 30 Jan 2008 21:59:51 +0000 |
User-agent: |
Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/523.10.6 (KHTML, like Gecko) Version/3.0.4 Safari/523.10.6 |
URL:
<http://savannah.nongnu.org/bugs/?22183>
Summary: Feature request - speed improvements at cost of
integrity for large systems
Project: rdiff-backup
Submitted by: guruevi
Submitted on: Wednesday 01/30/2008 at 21:59
Category: None
Severity: 3 - Normal
Item Group: None
Status: None
Privacy: Public
Assigned to: None
Open/Closed: Open
Discussion Lock: Any
_______________________________________________________
Details:
Hi,
I just would like to know if it were possible, to implement a switch somehow
that would improve the speed of rdiff-backup significantly by not comparing
file-by-file by calculating SHA1 (I believe that is the approach you take for
comparing files).
I have rather large binary files (in the order of hundreds of megabytes) or
very much small (1000's of 16k files from a medical scanner) on a very large
file system (9 terabytes) and just a run through with 6 Gbyte of changed files
takes about 72 hours for rdiff-backup and takes up a whole CPU (not really
what we like). The metadata file which I believe contains all the SHA1 hashes
is a whopping 3 Gigabyte. The files itself however never change, they either
get created or removed.
If there were a switch that could do turn off sha1 hashing for something way
less expensive (md4) or something else that would save a lot since (depending
on your machine) md4 is about 10-200% faster.
If you have additional information or tips (maybe insert it into the
documentation) on how to speed up rdiff-backup, I would like to know. I'm
going to switch from SSH to straight NFS (currently pushing the ACL for the
backup user), I already tried turning off compression but that didn't seem to
help a whole lot (you probably do the compression after a complete backup?)
_______________________________________________________
Reply to this item at:
<http://savannah.nongnu.org/bugs/?22183>
_______________________________________________
Message sent via/by Savannah
http://savannah.nongnu.org/
- [Rdiff-backup-bugs] [bug #22183] Feature request - speed improvements at cost of integrity for large systems,
Guru Evi <=