Re: [rdiff-backup-users] Weird behavior of rdiff-backup : founding diffs while there isn't

[Matthieu . Rioteau]

> From: address@hidden
> To: address@hidden
> Date: 19/11/2012 23:54
> Subject: Re: [rdiff-backup-users] Weird behavior
of rdiff-backup : 
> founding diffs while there isn't
> Sent by: address@hidden
> 
> On 11/19/2012 11:19 AM, address@hidden wrote:
> >
> > So I think here is the key. You can see that uname is once void
and once
> > filled with the user name.
> > It also appears that "faulty" changes can be of 2 sorts
:
> >   - Uname was void and is now filled
> >   - Uname was filled and is now void
> >
> > Now, running a "stat" command on a file where uname
is currently void (in
> > rdiff mirror) correctly returns the owner name.
> >
> > A strange thing remains : running a rdiff-backup --compare-at-time
2B (or
> > even 3B or 4B) on a folder chere files changed returns "No
changes found.
> > Directory matches archive data.".
> > Notice that --compare-hash-at-time or --compare-full-at-time
returns the
> > same.
> > So is it possible that documentation tells that --compare-at-time
do the
> > same as when a backup is runned, but that uname is actually used
only for
> > true backups and not for --compare option ?
> >
> > The last weird thing is that if I browse through the "increments/"
folder,
> > I will find a very small .diff file for each changed file.
> > Example with the first file listed above :
> > --------------------
> > hexdump -C VirtualBox.xml.2012-11-16T23\:50\:04+01\:00.diff
> > 00000000  72 73 02 36 46 00 04 35  00    
                  |rs.6F..5.|
> > --------------------
> >
> > But this is not the most important as preventing rdiff-backup
viewing
> > changes shall prevent also these increments to be stored.
> >
> > So question is : why are these "uname" once here and
once gone ?
> >
> > I see 4 main differences between the working system and this
one that
> > could lead to that :
> >   - Hard drive is a RAID 1 (working system doesn't have
RAID), but I don't
> > think it is the reason as it is pure hardware, and there is the
FS over.
> >   - Source partition is ext4 (working system is ext3).
> >   - Owners (i.e. uname) are LDAP users and LDAP server isn't
on the faulty
> > machine (it is on the working one). So users are not listed in
the
> > /etc/passwd file.
> >   - Kernel is 2.6 (working system is 3.2) and maybe one
of the above
> > characteristics (or another one) is badly managed by 2.6 kernels.
> 
> By default, rdiff-backup tries to preserve user _names_, not numeric
UIDs.
> This is obviously going to be a problem if one machine is able to
map
> UIDs to names and the other is not.  I'd look into why the faulty
machine
> is apparently unable to contact the LDAP server to do the mapping.
 You
> might see if the "--preserve-numerical-ids" option masks
the problem,
> but, frankly, I doubt it.
> 
> The tiny change files are basically saying, "There is no change."
 You
> will get that any time there is a metadata-only change.  That
file is
> required to exist if any change at all occurred.
> 
> -- 
> Bob Nichols     "NOSPAM" is really part of my
email address.
>                  Do NOT
delete it.
> 
> 
> _______________________________________________
> rdiff-backup-users mailing list at address@hidden
> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Hi Bob,

I also think problem is definitely LDAP, more precisely
that machine is not able to get access to LDAP server during a time lap.
And I think I found the reason (actually when waking
up on this morning (seems it was good to sleep on it). ;-)
The 2 machines are starting their respective backup
at the same hour and the one with the LDAP server is shutting it down during
backup (voluntarily) to prevent partial/corrupted data backup.
Thus the other machine is probably not able to resolve
user names during a small time.

So a good way to fix the issue will probably to postpone
one the backup to an hour where I'm sure that the other one is no more
running.

However I think that the "--preserve-numerical-ids"
option that you suggested can also do the job. So I have add it to the
rdiff-backup so it will be tested during next backup.
Then only I will time shift one of the backup, but
if the option is well working, that will be an extra data-safety feature.

When I will have test everything, I will keep you
updated so all your efforts finally get an answer (you can expect it by
the end of the week). :-)

Here I should say I'm a little bit confused to have
not think about that before posting on the mailing list (maybe it could
have preserve your time)... :-|

So I'm more than very thankful to you for all your
complete and helpful answers. ;-)

Thanks a lot, I will keep you updated very soon.

PS : one small thing I can share. During investigations,
I often have to analyze file_statistics....data.gz files. Thus I quickly
use this practical bash line :
---------------------------------------------------------------------------
address@hidden:~$ (gunzip -d | gawk -F" "
'{if ($0 !~ /NA$/) print $0;}' | less) < /BACKUP/FOLDER/rdiff-backup-data/file_statistics.THE_DATE.data.gz
---------------------------------------------------------------------------
That will list all lines where IncrementSize is not
"NA". If you remove the '~', you will get the opposite. Hope
it can help someone. ;-)