bug#40088: 27.0.90; Symbol’s value as variable is void: debugger-outer-match-data

[Noam Postavsky]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't think I understand what problem are you trying to solve, let
> alone why it's important to solve it in Emacs 27.  At least the
> original issue doesn't sound important to me, but maybe I'm missing
> something.

> I also don't think I understand what is your analysis of the issue;
> perhaps the commit log message could include more detailed
> explanation.

Yes, sorry, my analysis previously wasn't really more than 'raising a
signal while messing with the pdl stack seems bad'.  I've debugged a bit
more to see what's happening.

The underlying cause is in backtrace_eval_unrewind, which is used to
temporarily reverse let-bindings (it's called with a positive argument
to reverse bindings, and then a negative argument to re-apply them) by
backtrace--locals and backtrace-eval.  For the SPECPDL_LET_DEFAULT and
SPECPDL_LET_LOCAL cases (which occur for let-bindings on buffer-local
variables), the code calls Fdefault_value and Fbuffer_local_value on the
symbol.

For symbols which are unbound at top-level, the first (with positive
argument) call to backtrace_eval_unrewind will set the symbol's value to
unbound (putting the current value in the specpdl's "old value" slot).
On the second (with negative argument) call, backtrace_eval_unrewind
attempts to retrieve the symbol's value with Fdefault_value or
Fbuffer_local_value, but that raises a void-variable signal.  This
interrupts the restoration of the let-bindings, so any other variables
more recent on the stack will now have the wrong value.

ielm happens to have some buffer-local variables which are unbound at
top-level, which is why it can trigger this problem.  But any situation
with void variables with buffer-local let-bindings can also trigger it.

Now, all of the above is true also for Emacs 26 and earlier.  The main
difference in Emacs 27 is that debug.el calls backtrace--locals up
front, during setup.  In Emacs 26, it's only called when calling
backtrace-toggle-locals.  This means there would be hardly other
variables on the stack to get wrong value.  Also, errors in v26 ielm
evaluation don't raise the debugger (because ielm-eval-input used
conditiion-case rather than condition-case-unless-debug).

So the effects of this bug are newly significant in Emacs 27.  The fact
that it strikes when the debugger opens, means that it can multiply the
difficulty of analysing other bugs.  And the fix looks quite small and
safe enough to qualify for emacs-27.