bug#46326: 27.1.50; Excessive memory allocations with minibuffer-with-setup-hook

[Stefan Monnier]

jakanakaevangeli@chiru.no [2021-04-23 22:34:06] wrote:
> Daniel Mendler <mail@daniel-mendler.de> writes:
>> For me it seems to fix the issue. @jakanakaevangeli, can you confirm?
> Yup, confirmed.

Thanks, pushed to `master`.  Maybe we should consider it for `emacs-27`
since it's a regression in Emacs-27.

Eli, WDYT?

> The crucial part this patch does to fix this issue is
> removing depth info to avoid leaks, which is also directly related to
> bug46414.

Indeed, thanks.

> For me it seems to fix the issue. @jakanakaevangeli, can you confirm?
> I would still prefer to see a "proper fix". But given the backward
> compatibility requirements such a fix may not exist.
> Perhaps one could introduce some deprecation behavior. If a hook is removed
> and the object is not found via eq but found via equal, then print
> a warning? And then change the add-hook/remove-hook functions to eq only in
> some later version.

I suggest you `M-x report-emacs-bug` and make such a proposal.
We already have 3 places where we bump into this (`set-transient-map`,
`eval-after-load`, and `minibuffer-with-setup-hook`).

> Furthermore as a stop-gap measure one may still apply my patched
> symbol-indirection `minibuffer-with-setup-hook`, and revert it once the
> proper fix has been applied.

Yes, that was indeed the intention.  It's been pushed as well.

> (Using the symbol indirection seems to have other debuggability
> advantages. Closures are not particularly nice to debug in elisp, I hope we
> will also see some improvements regarding that.

We already have had some improvements (if byte-compiled, they should
print with a hyperlink that shows the disassembled bytecode, which is
a lot more palatable than the raw bytecode we used to present).

> It is at least on my Elisp wishlist to have better introspection for
> closures, location info etc.)

There's indeed still a fair bit of progress to be made.

> Note that `set-transient-map` already uses the symbol indirection. It may
> make sense to link to this bug from there such that one can adjust this
> function also at some later point depending on the resolution of this
> issue. The comment in `set-transient-map` reads like a bug to me
> "Don't use letrec, because equal (in add/remove-hook) would get trapped in
> a cycle." :)

Yes, it's (or was) clearly a bug in `equal`.  IIRC our current `equal`
is better in this respect, but I'd be surprised if we can't make it get
stuck in a loop in some cases.


        Stefan