[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] About backups and increments

From: Robert Nichols
Subject: Re: [rdiff-backup-users] About backups and increments
Date: Mon, 22 Aug 2011 10:54:48 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110816 Red Hat/3.1.12-1.el6_1 Thunderbird/3.1.12

On 08/21/2011 05:27 PM, Yuri D'Elia wrote:
Is the backup procedure crash/interrupt safe? If I'm backing-up some data and
for some reason the program gets killed (say, manually), in what state the
backup will remain? Does the latest snapshot reflect the old state (as if no
backup was performed), a broken state (cannot reliably recover anything), or a
mixed state? Is this state ever recorded as an increment so that I can drop it?

Recovery from a failed or interrupted session is reliable, but time consuming.
The next time you try to do any operation on that archive, rdiff-backup will
insist on first performing a regression of the failed session back to the
previous state.  There is also the "--check-destination-dir" action, which
does only the regression, if one is needed.  For a large backup set, the
regression takes a *long* time.

About space requirements: I assume the space required for the backup is:

- the space of the source files themselves
- the space of all the increments
- extra space required to compute the increment?
* Is this space stored on the source or destination drive?  * This should be
the size of the file currently computed + it's increments right? So should I
assume that to backup the *second* increment of some space X (where X can
possibly be just one huge file) I need at least X * 2 space for the backup -
just for temporary files?
* This brings me back to my first question: what happens when the destination
is full?

I'm not aware of any extra space needed for computing the increment, but the
increment itself, of course, does need to be stored on the destination drive.
If the destination drive runs out of space, the rdiff-backup session will

About backup speed. rdiff-backup doesn't seem to support both backupping *and*
pruning the increments at the same time (yes, I've read the man page). Though
this sounds like a very sensible thing to do: knowing that you will prune
several old increments, you can avoid to calculate the reverse diffs. Has this
been considered?

There's not much point in combining those two, totally independent actions.
Computing the reverse diffs for session N vs. session N-1 is totally
independent of the existence (or lack thereof) of earlier sessions in the

A request: increments pruning. --remove-older-than is fine most of the times,
though I would like an extra knob:

--keep-increments N (where N is the number of most recent increments to keep,
irregardless of time).

You can already do that.  Though the manpage doesn't mention it, you can also
use a "B" suffix to specify the number of sessions to keep:

       rdiff-backup --remove-older-than 30B /path/to/archive

will retain the most recent 30 sessions.  (Yes, you'll probably need to
include "--force" with that.)

Let's say I want always to keep at all times at least 2 increments (or 2
months, if that matters), I have no way to do that directly (I could list the
increments and calculate the time myself, but that's ugly).

Hey!! Some of us scripting veterans really get off on "ugly"!

    # Each Sunday, delete backups older than the previous Sunday, but
    # always retaining at least 6 backups.
    if [ "$(date +%a)" = "Sun" ]; then
        Cut=$(date -d 'last week' "+increments.%Y-%m-%d")
        CutAt=$(rdiff-backup -l "$BkVolume" | \
            awk -v "Where=$Cut" '
    stderr = "/dev/stderr"
    Dmax = -1
FNR > 1 {
    bdate[10000+FNR] = $1
    if(Dmax < 0 && $1 > Where)  Dmax = FNR
    if(Dmax < 0 || Dmax > FNR - 5)  Dmax = FNR - 5
    if(Dmax > 2) {
        if(match(bdate[10000+Dmax], "^increments\\.....-..-..T..:..:..-")) {
            print substr(bdate[10000Dmax], 12, 10)
            exit 0
        else {
            print "Unrecognized increment: \"" bdate[10000+Dmax] "\"" >stderr
    exit 1
}' )
        [ $? = 0 -a -n "$CutAt" ] && rdiff-backup --remove-older-than $CutAt \
            --force "$BkVolume"

Bob Nichols     "NOSPAM" is really part of my email address.
                Do NOT delete it.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]