[rdiff-backup-users] file-by-file changes from one backup to the next

[Henrik]

Hi there,

I've just recently switched backup system from rsnapshot
to rdiff-backup for a part of my data. Mostly because
of the space saved when large files have small changes,
and thus the ability to retain more backups (possibly
going back all the way to the original version of a file
when it first entered the system).

All in all I am happy with rdiff-backup. However I don't
have the same level of confidence that I quickly developed 
when starting with rsnapshot.

The main reason is (as I perceive it) the lack of status
output, or maybe the wrong kind of status output.
Or maybe I just haven't found the right combination of
options to get what I need. I am after all pretty new
to rdiff-backup.

In order to get some more information I have switched
to "-v5" to get file by file output but I still hardly
see the changes that I am interested in, while the noise
level understandably goes up.

Look at this:
===========
Processing changed file 2011/05
Incrementing mirror file /backup/rdiff-backup/foto/2011/05
Processing changed file 2011/05/2011-05-14_233003_dsc02931.jpg
Incrementing mirror file 
/backup/rdiff-backup/foto/2011/05/2011-05-14_233003_dsc02931.jpg
...
Processing changed file 2011/05/2011-05-31_083803_img_8789.jpg
Incrementing mirror file 
/backup/rdiff-backup/foto/2011/05/2011-05-31_083803_img_8789.jpg
Processing changed file 2011/05/2011-05-31_235712_img_8790.jpg
Incrementing mirror file 
/backup/rdiff-backup/foto/2011/05/2011-05-31_235712_img_8790.jpg
===========

It tells that those files have changed, but not in what way,
and it takes two lines per file to do so (and the output is
very hard to read or rather browse).
Adding and removing files is similarly terse.


I dearly miss is the "--itemize-changes" output that I have rsnapshot
(or rsync underneath ) produce:

.d..t...... /etc/
>f.st...... /etc/rsnapshot.conf
.d..t...... /etc/cron.d/
>f+++++++++ /etc/cron.d/rdiff
.d...p.g... /home/data/static/foto/new/
.f....og... /home/data/static/foto/new/foo.txt
>f..tpog... /home/data/static/foto/2009/10/2009-10-31_082857_hla_0019.jpg
*deleting   home/data/static/foto/2002/12/2002-12-24_200200_xmas02.jpg

Here I see that 
"etc" directory timestamp has changed
"rsnapshot.conf" file check(s)um has changed (seems I have edited that file)
"rdiff" is new file. (ahh now i know why I have edited rsnapshot.conf)
"new" directory (p)ermission and (g)roup have changed.
"foo.txt" file has a new owner and group.
"2009-10-31_082857_hla_0019.jpg" has timestamp permission owner and group 
changes.
"2002-12-24_200200_xmas02.jpg" has been deleted.

The format of one character per type of change at a fixed
position is great for parsing the output and helps to see
what changed at a glance.

What neither rsnapshot not rdiff-backup provide is a
quantitative measure of changed data. rsnapshot does
a summary output of the data sent and received and thus
gives a rough estimate on how much has changed, but it
doesn't say so for each file.

If rdiff-backup was to adopt a similar kind of output
it would be great to have a measure on the volume of
changes, too.

(I know that --list-increment-sizes does a summary output
of the rdiff size but having some more detail there on
number of changed/added/deleted files might be great too.)

I hope I don't sound like I am looking for nits to pick.
I just feel that rdiff-backup could be even greater if
it gave the user some more/different output.

cheers
-henrik

PS: Just in case you wonder:
My particular use case is my digital pictures collection.
I'd love to tag my pictures and rate them and maybe edit
some color profile here and there but I haven't
done it yet because I don't trust any photo management
software not to completely screw up my pictures. And
since we are talking about several thousand pictures
for e.g. a 2 week trip up some south american mountain
I can't review and rebuild thumbnails after each bulk
exif tagging operation. Thus a bug that eats my pictures
might go unnoticed for enough time so that the original's
rsnapshot backup got rotated out into data nirvana.

Also having the output parseable might allow to quickly
find out when a picture was first added to the collection
and thus allow an automated copy of (mostly) unedited
files to a separate location.