Re: [Duplicity-talk] Caching for pwd and grp operations

[Steve Atwell]

On Thu, Nov 8, 2012 at 1:58 AM,  <address@hidden> wrote:
> looks like extending grp and pwd and import these instead would be more 
> structured. obviously every change to tarfile must be documented carefully in 
> the source, so it can be reapplied in future versions.

I pushed what I have right now to
https://code.launchpad.net/~satwell/duplicity/caching.  It does modify
tarfile, but in a pretty limited way.  And still needs tests.

> before doing it, i'd also like to see a comparision of a backup with and w/o 
> the patch.

In my environment, the difference is pretty astounding.  I have about
12MB of passwd data, and 8MB of group data.  Most files are owned by
very large groups (generally 10k to 40k members).  I did an initial
full backup a git checkout of the Linux kernel (1.3GB, 43k files).
Then did an incremental backup with what's in head vs. my branch:

no caching:  272.57s user 27.83s system 99% cpu 5:00.79 total
with caching:  5.59s user 0.24s system 99% cpu 5.856 total

-- 
Steve Atwell <address@hidden>