Re: speed up large library linking

[Ralf Wildenhues]

Hi Peter, others,

* Ralf Wildenhues wrote on Wed, May 11, 2005 at 04:59:33PM CEST:
> * Peter O'Gorman wrote on Wed, May 11, 2005 at 04:33:52PM CEST:
> > Ralf Wildenhues wrote:
> 
> > | My patches break one assumption held in libtool so far:  that --dry-run
> > | will cause no file changes.
> > 
> > I understand your reasons for breaking this, but it freaks me out a little
> > bit.
> 
> Why?

How may I convince you?

> > Have you reported the quadratic behavior to the bash maintainers?
> > We need to file bugs with them about this too.
> 
> No, but I will do so.  It's not just bash, however, but virtually any
> shell I could find.

Done: http://lists.gnu.org/archive/html/bug-bash/2005-05/msg00611.html

> > Note that some of your FIXMEs in the patches below are in my opinion
> > not needed, if libtool got the argument list and -objectlist was not
> > used then the kernel limit was not exceeded and $ECHO is fine.
> 
> Oh yes, you are right.  I started adding these mechanically to all
> dependent variables instead of only the ones where the object list gets
> expanded.  Will post an updated patch.

Here's a list of updated patches.  Changes to previous proposed patches:

- bugfix: in rename, we need to conserve the leading space in `$oldobjs'.
- reword doubled status output about reloadable output.
- output rename status output with func_echo for `libtool: link:' prefix.
- Remove all irrelevant FIXMEs as pointed out by Peter.
- New m4 macro OUTPUT_LONG_LIST:
It may be used to output a list which may be longer than the command
line limit.  It is fast for the good cases (either builtin $ECHO and/or
command line shorter than limit), and gets pretty slow for the worst
case.  I chose a macro as a cheap `inline' rather than a shell function
for this because the shell function call uses a significant amount of
time (in the fast case, obviously :).  If the resulting about 1% size
increase of `libtool' bugs anyone, we can change this.  This macro is
used to fix all the relevant FIXMEs added in the previous proposed
patches.

Right now, OUTPUT_LONG_LIST is defined right in ltmain.m4sh, which is
probably against Libtool macro definition policies.  Where should it go?
If outside of ltmain.m4sh, it would probably need to be namespace-clean..

Outstanding issues:
- As of now, each user of OUTPUT_LONG_LIST creates a separate scratch
  file within our scratch directory, for testing purposes.
- What to do for MinGW which does not have all necessary tools?  Keep
  the old renaming loop as a workaround in that case?
- A good testsuite test will now really have to exceed the system
  command line length.
- No error checking for write failures (full disk, etc.).  I actually
  don't know whether it can be done portably in shell code without much
  hassle.  Does anybody think this is worthwhile/necessary to pursue?
- I don't know how to implement OUTPUT_LONG_LIST without a temp file.
  This is a question to think of for an eventual branch-2-0 backport of
  the fixme patch, in case you disapprove of temp files for it.

The ranlib change will be in another patch and mail.

OK to apply this to HEAD?

Regards,
Ralf

speedup-features3.diff
Description: Text document

speedup-fixme3.diff
Description: Text document

speedup-parseargs3.diff
Description: Text document

speedup-reload3.diff
Description: Text document

speedup-rename3.diff
Description: Text document

speedup-piecewise-oldlibs3.diff
Description: Text document