Re: [rdiff-backup-users] file-by-file changes from one backup to the next

[Henrik]

On Tue, Jun 14, 2011 at 04:36:38PM -0700, Adrian Klaver wrote:
> On Tuesday, June 14, 2011 4:18:25 pm Robert Nichols wrote:
> 
> > 
> > You can get a little of that from the file_statistics.{timestamp}.data.gz
> > file that is created for each backup.  For each file or directory there is
> > a flag that indicates a change, plus the sizes of the source file, the file
> > in the mirror, and the increment.  Because of the format and the
> > possibility of file names with spaces, it gets a bit messy to parse that
> > file, but the following will show all of the lines with the "change" flag
> > set:
> > 
> >    zgrep -E '(# F)|(1 [^ ]+ [^ ]+ [^ ]+$)' \
> >         file_statistcs.{timestamp}.data.gz | less

Ok, so file_statistics.* is the place to start with and the format for my
perl conditioned mind would be something like this

 ^(.+) (0|1) (NA|\d+) (NA|\d+) (NA|\d+)$

Seems like the occurence of NA and 0 values can easily tell me about
deleted or new files.


new file or directory:
foo.data 1 1613908 NA NA

removed file:
foo.txt 1 4 NA 0

removed directories:
foo/2009 1 0 NA NA
foo/2009/12 1 0 NA NA

changed file: (grew from 4 bytes to 5 bytes)
foo.txt 1 4 5 126

changed directory (in this backup some files were added):
2011/05 1 0 0 NA


I guess I could thrown together a perl one liner to go through this but
what I'm missing is the information on the type of change .. like here:

2007/04/2007-04-28_095048_s5001204.jpg 1 176308 176308 131

I see that the size didn't change but I can't tell if it was meta data or
file content. To do that I'd probably have to through the two most recent
mirror_metadata.* to see what changed, or find out the format of
 
increments/2007/04/2007-04-28_095048_s5001204.jpg.2011-06-13T02\:20\:02+02\:00.diff

> > Alas, if you were using the "--null-separator" option for your backup,
> > that gets even worse:

I don't plan to use anthing outside the ascii range for filenames, but good
to know that I could :-)

> > Also, you can always see what versions of a file exist in the backup:
> > 
> >    rdiff-backup -l /backup_dir/path/to/some/filename

Oh, I didn't know --list-increments worked on files too. Thanks!

> > 
> > That's nothing like what you're used to, but at least it's something.

It's a good start.
 
> Maybe:
> 
> rdiff-backup --list-changed-since 1B  /some/rdiff-backup/dir
> 
> where 1B equals last backup before current.

I'll take a look at the code that handles this option. maybe it is easier
to hack my changes into the existing python code instead of writing my own
parser for the data.
... 
I've taken a look at the code and I guess to do it in a reasonably efficient
manner I'll have to stuff it into an IDE to see if the ListChangedSince method
in restore.py has access to the right kind of data, and where it's output is
used besides from ListChangedSince in Main.py.

Thank you both for the input!

cheers
-henrik