bootstrap time (was: ensure $MV is set before each use [libtool--gary--1.0--patch-30])

[Ralf Wildenhues]

* Gary V. Vaughan wrote on Mon, Aug 29, 2005 at 12:55:13PM CEST:
> On 29 Aug 2005, at 07:33, Ralf Wildenhues wrote:
> 
> >[ BTW, could you get your mailer to do decent quote wrapping?   
> >Thanks ]
> 
> I'm at home and using Apple's crappy Mail.app.  I'll upgrade to  
> Thunderbird (like I use at work) when I have some spare time.  Sorry
> about that.

No problem at all.  I just figured you might not even be able to see it
do that.

> >I am NOT talking about the number of autoreconf invocations.  I am
> >talking about the time ONE autoreconf invocation takes.
> 
> Okay.  I hadn't noticed any significant difference between HEAD and
> branch-2-0, but I haven't used 1.5 in what seems like years.  Do you
> know which  step is taking all the time?  If it is autoconf, then
> either m4 or the macros  we are invoking are to blame.

See below, I've done rough measurements.

> >Almost all the time is spent in building the list of tag decls, I
> >believe.  I know it's at least quadratic in the number of decls,  
> >but it
> >may also be even higher, haven't looked at it closely.  So maybe it's
> >not M4 but ltsugar.m4 and parts of libtool.m4 that would need a  
> >rewrite.
> 
> Quite possibly, yes.  When I get back to m4 development, I plan to add
> some sort of instrumenting feature to help with this kind of
> profiling.  We can revisit the issue then if it hasn't been addressed
> already.

Just as an example, here's a comparison of the three branches on a fast
machine (sic!  The first is the whole branch-1-5 bootstrap, not just
top_srcdir!):

branch-1-5/libtool $ ./bootstrap
58.25user 4.40system 1:02.54elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+581639minor)pagefaults 0swaps

branch-2-0/libtool $ reconfdirs=. \time ./bootstrap
80.77user 3.39system 1:26.07elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+383895minor)pagefaults 0swaps

HEAD/libtool $ reconfdirs=. \time ./bootstrap
83.66user 3.37system 1:30.01elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+402730minor)pagefaults 0swaps

HEAD/libtool $ \time ./bootstrap
425.39user 18.49system 7:36.83elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1929876minor)pagefaults 0swaps


Now the individual steps when reconfiguring `.' from HEAD (not very
informative, I believe):

$ ACLOCAL=/usr/bin/time\ aclocal AUTOMAKE=/usr/bin/time\ automake \
  AUTOCONF=/usr/bin/time\ autoconf reconfdirs=. ./bootstrap
*snip*
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: /usr/bin/time aclocal --force -I libltdl/m4
16.68user 0.61system 0:17.29elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+66505minor)pagefaults 0swaps
autoreconf: configure.ac: tracing
16.01user 0.62system 0:16.64elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+74647minor)pagefaults 0swaps
autoreconf: running: /tmp/libtool/libltdl/config/libtoolize --copy --force
libtoolize: Bootstrap detected, no files installed.
0.99user 0.07system 0:01.06elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+8881minor)pagefaults 0swaps
autoreconf: running: /usr/bin/time autoconf --force
16.76user 0.59system 0:17.36elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+69373minor)pagefaults 0swaps
autoreconf: running: autoheader --force
autoreconf: running: /usr/bin/time automake --add-missing --copy --force-missing
16.09user 0.69system 0:16.79elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+85580minor)pagefaults 0swaps
16.66user 0.76system 0:17.42elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+97087minor)pagefaults 0swaps
autoreconf: Leaving directory `.'


A while ago, when debugging the m4_cdr issue, I noticed that the
ltsugar stuff that builds the tag lists takes up most of that time.
Nope, no measurements done there, sorry.  But looking at the code it
reveals that it must be at least quadratic in the number of tags.

Cheers,
Ralf