Re: Setting RLIMIT_STACK

[Alberto Bertogli]

On Sun, Jul 18, 2010 at 12:54:56PM -0400, Paul Smith wrote:
> On Sat, 2010-07-17 at 02:22 -0300, Alberto Bertogli wrote:
> > Chasing a bug, I noticed that make sets RLIMIT_STACK to RLIM_INFINITY.
> > 
> > In Linux (since 2.6.25), that actually triggers a very subtle behaviour
> > change in the way the kernel manages the memory layout. While most
> > applications are oblivious to it, some are not and can cause issues that
> > are not so easy to see.
> > 
> > The layouts are described at http://lwn.net/Articles/90311/, and while
> > Linux usually uses the new one (described there as "top-down mmap()
> > method"), when RLIMIT_STACK is set to RLIM_INFINITY it falls back to the
> > old one (the one described first).
> 
> Hi Alberto.  I was looking into this more.  Although GNU make does set
> RLIMIT_STACK to infinity while it, itself, is running, it resets the
> stack size back to the original value it had when it was started before
> it runs any commands.

Thanks for looking into it!


> That is, in job.c after we fork() to start the command we call
> setrlimit() again to reset the stack back to what it was, before we
> exec() the command we're about to run.
> 
> So, I'm not sure I fully understand the implications here.
> 
> Are you saying that this resetting of the stack limit back to what it
> was does not reset the memory layout back to using the new method in
> make's children, and this causes different behavior in make's children?
> Or that the change somehow subtly changes the behavior of _make_ in a
> less-than-optimal way?

Layout changes affect make's children, but not make itself.

The layout is decided by the kernel early, when the new process is being
created in execve(), and cannot be changed while running. So changing it
after fork() but prior to execve(), as you said, should be ok.

However, I'm not seeing this behaviour in make 3.81 (Ubuntu's, not my
own).


Peeking around the source browser I notice that the change that introduced
the setrlimit() you describe is commit 1.191, dated 2008-11-30, which is
post-3.81.

Seeing that, I tried the latest alpha release (3.81.90), and it works as
expected!

So I can confirm that the issue I was seeing has already been fixed in the
new release, and as far as I can see there are no further changes needed
to address it.

Thanks a lot,
                Alberto