Re: [rdiff-backup-users] Failing on 11GB file - other large files working

[Ty! Boyack]

"Ty! Boyack" wrote:
> 
> Ben Escoto wrote:
> >
> > >>>>> "Ty!" == ty  <Ty!>
> > >>>>> wrote the following on Fri, 18 Jul 2003 12:10:26 -0600
> >
> >   Ty!> We've been doing nightly backups for a couple of weeks now
> >   Ty!> without any problems, and then a user throws another kink at
> >   Ty!> us.  I've attached a trace below of the entire backup run, but
> >   Ty!> the file it is failing on is an 11GB file, of which about 5GB
> >   Ty!> shows up on the backup mirror.
> >     ...
> >   Ty!> Any ideas on how to get large files like this (this file will
> >   Ty!> have very large block-level differences) to get backed up
> >   Ty!> successfully?
> >
> > Try 0.12.1.  The librsync block size is not set based on the file
> > size.  I just tested it successfully two totally random (made using dd
> > on /dev/urandom) 12GB files, although this operation still took 4
> > hours.
> 
> Ben, I'll be glad to try 12.1 - it sounds like a great move to have
> the blocksize adjust based on the file size.  However, we weren't
> seeing the long times you did - Our failures were occuring after
> around 15 minutes.  I don't know what the difference is, but it's just
> a little more information for you.  It could be I/O system
> differences, I'm running this on a dual Athlon 2000+ with a 14 disk
> ide-to-scsi raid, with 512MB cache.  It's possible that the large
> cache is really coming into effect here.
> 
> I'll try 12.1 today and let you know if it works.

Hmm...  unfortunatly 12.1 did not fix the problem.  Since you were
able to do it with 12.1 on two totally different randomized files, I'm
guessing that the problem is outside of rdiff-backup.  Which python
version are you running?  Do you know if there is any way to verify
that librsync got compiled with large file support (my compile, but
I've been known to goof builds once or twice...)? 

Also, for python and librsync, do you use package versions or do you
compile your own?  

Thanks again,

-Ty!


-- 
-===========================-
  Ty! Boyack
  NREL Unix Network Manager
  address@hidden
  (970) 491-1186
-===========================-