Re: Dynamic modules: MODULE_HANDLE_SIGNALS etc.

[Eli Zaretskii]

> Cc: address@hidden
> From: Daniel Colascione <address@hidden>
> Date: Thu, 24 Dec 2015 09:04:49 -0800
> 
> You'd prefer Emacs to lock up or corrupt data instead?

Instead of crashing and corrupting data?  What's the difference?

Of course, if it would do that all the time, or even most of the time,
we'd consider the solution a bad one, and remove it or look for ways
of improving it.  But we are not there; in most cases the recovery
doesn't hang and doesn't corrupt any data.

> Neither you nor Paul have addressed any of the alternatives to this
> longjmp-from-anywhere behavior. You have not addressed the point that
> Emacs can crash fatally in numerous ways having nothing to do with stack
> overflow. You have not addressed the point that we already have robust
> stack overflow protection at the Lisp level, and so don't need
> additional workarounds at the C level. You have not even provided any
> evidence that C-level stack overflow is a problem worth solving.

I think we did address those, you just didn't like the responses, so
you don't accept them as responses.

> All I see is a insistence that we keep the longjmp hack stay because
> "Emacs must not crash", even though it demonstrably does crash in
> numerous exciting ways, and won't stop any time soon, because real
> programs always have bugs, and experience shows that failing quickly
> (trying to preserve data) is better than trying to limp along, because
> that just makes the situation worse.

Stack overflow recovery is an attempt to solve some of these crashes.
Having it means that users will lose their work in a smaller number of
use cases.  So it's an improvement, even if a small one.  I fail to
see in it any cause for such excitement.

> I know the rebuttal to that last point is that the perfect shouldn't be
> the enemy of the good: believe me, I've debugged enough crashes and
> hangs caused by well-intentioned crash recovery code to know that
> invoking undefined behavior to recover from a crash is far below "good"
> on the scale of things you can do to improve program reliability.

I believe you.  Now please believe me and Paul who have slightly
different experience and have come to slightly different conclusions.

> 1) Using some mechanism (alloca will work, although OS-specific options
> exist), make sure you have X MB of address space dedicated to the main
> thread on startup. At this point, we cannot lose data, and failing to
> obtain this address space is both unlikely and as harmful as failing to
> obtain space for Emacs BSS.
> 
> 2) Now we know the addresses of the top and bottom of the stack.
> 
> 3) On each time Lisp calls into C, each time a module calls into the
> Emacs core, and on each QUIT, subtract the current stack pointer from
> the top of the stack. The result is a lower bound on the amount of stack
> space available. This computation is very cheap: it's one load from
> global storage or TLS and a subtract instruction.
> 
> 4) If the amount of stack space available is less than some threshold,
> say Y, signal a stack exhaustion error.
> 
> 5) Require that C code (modules included) do not use more than Y MB of
> stack space between QUITs or calls to the module API
> 
> 6) Set Y to a reasonable figure like 4MB. Third-party libraries must
> already be able to run in bounded stack space because they're usually
> designed to run off the main thread, and on both Windows and POSIX
> systems, non-main thread stacks are sized on thread startup and cannot grow.
> 
> I have no idea why we would prefer the SIGSEGV trap approach to
> the scheme I just outlined.

Your scheme has disadvantages as well.  Selecting a good value for Y
is a hard problem.  Choose too much, and you will risk aborting valid
programs; choose too little, and you will overflow the stack.  Making
sure C doesn't use more than Y is also hard, especially for GC.  It
sounds like just making the stack larger is a better and easier
solution.

Threads make this even more complicated.  At least on Windows, by
default each thread gets the same amount of memory reserved for its
stack as recorded by the linker in the program's header, i.e. 8MB in
our case.  So several threads can easily eat up a large portion of the
program's address space, and then the actual amount of stack is much
smaller than you might think.

So on balance, I don't see how your proposal is better.