Re: [rdiff-backup-users] rdiff-backup performance -- slow operation on initial backup?

[Greg Troxel]

"Derek Atkins" <address@hidden> writes:

>> I would up the send/receive socket buffers (because it's easy, not
>> because I think that's the problem), and watch disk/cpu on both sides,
>> and also run netstat to see if data is piling up in the transmit socket
>> buffer.
>
> Do you mean within rdiff-backup, or at some other level?

On NetBSD I mean bumping up these sysctls

net.inet.tcp.sendspace = 131072
net.inet.tcp.recvspace = 131072
net.inet6.tcp6.sendspace = 131072
net.inet6.tcp6.recvspace = 131072

and presumably that's similar on other systems.

But, if your socket buffers aren't full, that's probably not your
problem.

>> FWIW, I used to use rdiff-backup but found it to be nonrobust on
>> machines with limited (only a few GB) RAM and hundreds of GB of backup.
>> I have switched to bup.
>
> Unfortunately bup is not available on all my target platforms.

bup is python with a little C, and thus seems pretty portable.   Where
isn't it working?

There is also attic and borg which are similar to bup.

> Maybe I should consider amanda or bacula?

amanda is basically wrappers around dump and tar.  If you have 50
machines and want to do level 0/1/2 to tape and take tapes offsite, it
works great, after you pay for the LTO tape drive.

Two things to think about:

  do you care about deduplication?  bup does not only per-file but
  within-file deduplication, so if multiple boxes have the same data it
  doesn't take up extra space

  Do you really need to back up all platforms, or could you sync from
  some (Android?) to a machine with more disk and back that up?  I have
  been using syncthing, which seems to be pretty solid, for syncing
  among Android and regular computers (BSD and OS X).  (It is written in
  go so it's not in practice that portable.)

signature.asc
Description: PGP signature