Re: [Gnu-arch-users] Slow inventories on large source trees

[Aaron Bentley]

Mike Hearn wrote:

> On the WineHQ source tree, with an in-tree build, doing tla inventory
> --ids >/dev/null takes approximately 20 seconds on my athlon 1200 with
> 256mb of ram.
>
> This makes certain operations extremely slow, and this is problematic for
> us (the Wine developers thinking of adopting arch).

I'd love to see Wine adopt Arch.  It seems like a good fit for the way 
Wine's developed (from what I read on kerneltraffic).  Making it work 
well for you folks would make it work well for other large projects, and 
it would also raise some awareness.  Wins all around.

> I haven't tested with an out-of-tree build - while I suspect performance
> would be better, I'm really not sure we want to ask everyone to do
> out-of-tree builds to work around this slowness in arch.

If the object files are in a separate directory in the tree, that 
directory can be marked precious, and tla won't descend into it.

> Are there any ideas for how to optimize this? The CVS gateway tree is
> using names tagging, though a real switch to arch would be using explicit
> tags in future.

I know that there are performance differences between the various 
tagging methods, but I'm not familiar with the causes.

Changeset generation has an optimization for the case where the 
reference tree is hardlinked to the project tree, so "get --link" may 
help.

My backbuilder code has one optimization that would improve the speed of 
your initial get.

Generating a changeset should involve at most two stats per unchanged 
source file, plus the cost of handling changed files.

Sounds like we should do some profiling on your WINE tree.

Aaron
--
Aaron Bentley
Director of Technology
Panometrics, Inc.