Re: [rdiff-backup-users] rdiff-backup for email

[Ty! Boyack]

Iain Dooley wrote:
> On Fri, 10 Nov 2006, Blair Zajac wrote:
>
> > Michael Simms wrote:
> > > Hello,
> > >
> > > I'm new to the list, and to rdiff-backup. We're looking at 
> > > alternatives for backing up email. It's currently 3TB, and growing 
> > > quickly. Is anyone out there backing up email, or other 'mission 
> > > critical' data with rdiff-backup?
> >
> > I back up my entire system with rdiff-backup to a local drive, then 
> > rsync the entire rdiff-backup over to another box.  I backup email, 
> > Postgresql, Mysql, Apache, etc.
>
> that's exactly what i do :) i do it for 6 production servers
>
> cheers
>
> iain
I do the same for several TB of production data here as well, but have 
found that very active filesystems (like mail) can cause rdiff-backup to 
complain a bit.  As you can expect, it doesn't like it if you start 
backing up a (source) mailbox, and before it can finish doing all the 
reverse-diff/update of the (backup) mailbox you let the mail server 
stuff another message into the (source) mailbox.  I speak from the 
experience of mbox formatted mail - I don't know if other formats would 
have quite the same issues.

I've used rdiff-backup for nightly backups in a mail-server scenario and 
it works great.  What I found is that I would sometimes miss a single 
night's backup of certain mailboxes, but they would be caught the next 
night.  The randomness of which user's mailbox was updated at exactly 
the wrong time was enough that I had success for any given mailbox 
nearly 100% of the time, while in aggregate I had several mailboxes fail 
each night. 

If you can, I'd look into snapshots to make this problem go away.  If 
you are using LVM it is fairly easy to create a snapshot volume:
lvcreate --size 10G --snapshot --name snapshot /path/to/lvm/volume
(where /patch/to/lvm/volume is your own LVM volume), mount that new 
snapshot volume, and then do your backups from the snapshot.  Once you 
are done with the rdiff-backup, you can remove the snapshot with:
lvremove /path/to/lvm/snapshot-volume

For "mission critical" data, I also push a copy of my rdiff-backup 
archive to an offsite server using rsync.  (We're lucky enough to have a 
fat pipe to another site).  That way I have 3 disk copies, on 3 
different hardware devices, in 2 physical locations, where the data is 
within 24 hours of current.  (The live data, the local rdiff-backup 
copy, and the rsync'ed copy of the rdiff-backup copy).  I can bring up 
either of the two rdiff-backup versions as a primary repository in case 
of a nasty hardware failure, like a raid controller going up in smoke 
and taking out an entire disk array (like our last power failure...).  
This is particularly nice since the data is as current as tape backup, 
but your time to recovery is MUCH shorter.  I this case, under an hour 
to reconfig and bring back 3+TB of data on about a dozen logical volumes. 

True, the multiple copies are expensive, but for us it was cheaper than 
tape solutions.

Do keep in mind, though, that rdiff-backup does not give you easy (if 
clunky) archives like tape backups do.  If you are required to retain 
e-mail records, you will still need a tape solution or some other 
offline system.

Sorry if I'm pointing out the totally obvious when the merely obvious 
will do.

-Ty!