Re: [rdiff-backup-users] feature requests and notes

[Ben Escoto]

On Mon, 27 Oct 2003 03:42:45 -0500
address@hidden (Andrew K. Bressen) wrote:
> I have some feature request/suggestions:
> 
> (1) python errors are wonderfully informative compared to most
> environments these days, but not so easy to automatically parse.
> If rdiff-backup could spit some indicator at the beginning and
> end of each error stack, that would make writing log parsers
> much easier. I usually run at -v6; I don't know if running at
> a lower level would be more easily parseable. I have found this
> to be a useful level for diagnostics/bug reporting. 

Yes, at lower than 5 you don't get the traceback any more, and the
errors are supposed to be easy to parse---they fit on one line and
start with something regular like ListError or whatever.

Still, it's a good suggestion.  Perhaps you could show some examples
of actual output, and then edit them into something you would want to
see.  It shouldn't be very hard to fix once it's clear what the output
should look like.

> Actually, it comes to mind that perhaps outputting the log level
> of each message at the start of the line would be useful, so I could
> grep out the level one or two output to read and keep the lower
> level stuff in case it is needed later. 

Yes, definitely a good idea.  In the meantime you could use
--terminal-verbosity.  One complication is that log messages are
sometimes multi-line.

> (2) from reading the list, it seems that sometimes (as when a file
> in the backup dir is inadvertantly deleted) users may have to 
> manually dink with the metadata. It'd be cool if there were tools
> to do this programmatically, such as options to delete the latest 
> or all versions of a file from the archive, or to sync the metadata 
> to the archive contents.

Yes, also good.

> I'm unclear on if --check-destination-dir does anything besides 
> back out the failed increment attempt. 

It also updates the metadata of files in the mirror directory that
don't match their records in the metadata file.

> (3) in re space management by number of backup cycles instead of
> by dates, Ben said:
> 
>   "Yes, it's a good idea.  How about if a new time specification was
>    allowed, so that '--remove-older-than 4B' would remove everything
>    older than the 4th Backup session?  It could be useful when restoring
>    also, e.g. '--restore-as-of 2B'."

This has been added already as of 0.13.2.

> I thought of a new failure scenario that might complicate this syntax. 
> If my daily backup scripts do a --remove-older-than 7D and something
> goes wrong with backing up, but not the remove, for 8 days, I could be
> in trouble. Ideally, a way to tell --remove-older-than to retain a
> certain minimum number of copies would be a safety, and I don't think
> the above syntax quite allows for that. Similarly, if one has been 
> doing lots of tests and reruns one day, perhaps one would want to make sure
> to have several days worth of backups around. 
> 
> Perhaps a --retain-at-least that also took 
> <number> smhDWMY and <number> B args?

The new 'B' letter doesn't complicate the syntax (just the
implementation :)).  If anything it is safer because
--remove-older-than 5B will never delete all your history.  Addition
of --retain-at-least is still possible, as in:

    --remove-older-than 1M --retain-at-least 5B

or even

    --remove-older-than 2003-10-05 --retain-at-least 1M

>     [s]locate(1) functionality
>     md5 sums
>     file integrity (tripwire/integrit) functionality
>     retrieve increment history of a specific file

What do these mean?

>     situation on windows and samba time granularity unclear

rsync has a --modify-window for this, but I can't see how this
couldn't just be a samba bug.  Is it so hard just to return, say, only
even times?

> (5) Ben, a while back you said that a file mv was handled as a delete and 
> an add of the whole file. But I noticed in some later mail you talked about
> tracking changes via inodes vs. md5 sums. Does this mean that at some
> point one or both will be implemented? Does that in turn mean that 
> mv's could be detected?

I don't plan on detecting moves in rdiff-backup any time soon.  In the
long term, I was thinking about adding this capability to duplicity,
which has a simpler task in that it only deals with one file tree.  If
that is considered worthwhile perhaps it could then added to
rdiff-backup.

> (6) Documentation
> the man page linked from http://rdiff-backup.stanford.edu/docs.html
>   doesn't say what version it's for or when it was made. 

Looks like the groff2html thing I am using strips it out.  Should be
in the man page more visibly anyway, current convention is stupid.

> http://rdiff-backup.stanford.edu/format.html
>   claims to be current for 0.6, and I know metadata handling
>   has changed dramatically since then... 

On-disk format is pretty much the same, but still it should be
updated.

I will be very busy for the next month or so and probably won't be
doing much rdiff-backup work.  But there is some good stuff here and
it would be a shame if it got lost.  Perhaps you could add support
requests at

https://savannah.nongnu.org/support/?group=rdiff-backup

for many/all of these?  I don't like the interface much but a mailing
list isn't the best interface for this kind of stuff either.

What I would really like is a wiki but Savannah said they didn't have
the resources, and it would be too much of a hassle for me to run my
own.


-- 
Ben Escoto

pgpitaGVGVksm.pgp
Description: PGP signature