bug#56682: Fix the long lines font locking related slowdowns

[Stefan Monnier]

>> IIUC they use `fontification-functions` to wrap the "select the current
>> major mode" operation around the call to `jit-lock-function` so that
>> each submode can use its own set of jit-lock clients.
>> It's harder to hook into `jit-lock-functions` to do that because you
>> need to detect when `jit-lock-functions` is modified.
>
> Do they detect when fontification-functions are modified? Sounds like just
> as much of a rare occurrence.

Could be.

>> AFAICT mmm-mode does not try to do that at all.
>
> It does the wrapping through custom font-lock-fontify-region-function and
> syntax-propertize-function. The major mode specific values are saved after
> the major mode is called. Any subsequent modifications are just ignored.

I'm not sure why they decided to wrap `fontification-functions`, but in
theory it lets them support things like a submode that's using
`nhexl-mode`, another using `nlinum-mode`, and yet another using
`glasses-mode`.

In any case, these multi mode packages always need to "look back" in
order to find which mode to use at a given position, so they inevitably
need some kind of widening to work correctly.

>>> They'll get all the corresponding performance improvements (just-in-time
>>> fontification) as a bonus.
>> Not sure what you mean by that.  `fontification-functions` is already
>> called "just-in-time" (and those packages don't re-implement jit-lock on
>> their own, they just wrap themselves around jit-lock).
> Okay then.
> Sounds like somewhere in there there's a duplication of design still.

I think the duplication is in the weird semantics of
`fontification-functions`: this supposedly holds a list of functions,
but it's difficult to make it work right with more than one function.
I think it should be replaced with a single `fontification-function` instead.

> But also: if they go on to call jit-lock anyway, then they would go through
> whatever application of narrowing that we put in there. Be that in
> font-lock-fontify-region, or in jit-lock-function.

Indeed.  But before doing so, they may(?) scan the whole buffer to find out
which submode to use at the current location.

I think it's OK to "break" those packages with some forced narrowing
until they get improved to do something sane in large files or
long lines.  But of course it should be possible to unbreak them without
recompiling Emacs.


        Stefan