Re: execvp: /bin/sh: Argument list too long

[Ralf Wildenhues]

Hello Pippijn,

* Pippijn van Steenhoven wrote on Mon, Nov 08, 2010 at 10:11:20PM CET:
> On Mon, Nov 08, 2010 at 09:56:52PM +0100, Ralf Wildenhues wrote:
> > [1] On some systems this is not true; e.g., GNU make evades the limit on
> > MSYS for this particular point, IIRC using a response file.
> 
> Can automake do something similar? The dist files are always the same
> (well, they should be), so they can be written to an extra file, which is
> then piped into xargs or something.

In principle: yes, when the list of files is completely known at
automake run time.  This would mean, no @substituted@ bits in that macro
anywhere, no conditional bits, and you don't get to override macros at
make run time like
  make DISTFILES='foo bar baz'

These new limitations would not seem very harsh for most packages;
especially for the distdir target, most conditionals are ineffective,
and users are not likely to want to override them at make run time,
but disallowing subtitutions would be a real limitation.

Even just reordering the list of files might introduce bugs with some
(admittedly fragile) setups, with packages that rely on dependency order
(thus of course disallowing parallel execution) or that contain e.g.,
both FILE and ../sub/FILE in the list somewhere.  (We do have measures
in place to cope with FILE and $(srcdir)/FILE so the latter point should
be really rare.)

> > > What version of GNU make are you using?  There was an issue in GNU make
> > > 3.81 related to setting stack limits, that I believe is resolved in GNU
> > > make 3.82.  I can't recall the details.
> > 
> > I don't think it's that.  It's that GNU make also requires the
> > per-argument limit to be large, and IIRC that is still 128K on Linux
> > (yes, speaking about the kernel only).  I wrote about this before:
> > http://thread.gmane.org/gmane.comp.gnu.make.bugs/4219
> > but my proposed hack back then was (understandably) not too popular,
> > and I haven't gotten back to anything better yet.  ;-)
> 
> In other words, I can't make dist even with GNU make 3.82? What do you
> suggest, then?

Well, you could try my patch from
http://article.gmane.org/gmane.comp.gnu.make.bugs/4219
(append /raw to the URL to get it unmangled).

I still think letting GNU make avoid the issue on Linux systems would be
the easiest way out here.

As already written, alternative workarounds include reorganizing your
code a bit to have a bit more recursive makefile setup, so that each
makefiles' list would remain below the limit.

Cheers,
Ralf