bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 14.08.2022 08:23, Eli Zaretskii wrote:
> > Date: Sat, 13 Aug 2022 22:08:25 +0300
> > Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> > From: Dmitry Gutov <dgutov@yandex.ru>
> >
> > > Fontifying only the beginning of the file doesn't help when the file
> > > is first shown at another point, like when Emacs or emacsclient is
> > > invoked with the +NN[:nn] argument, or the user uses saveplace or
> > > similar package.  That's admittedly rarer than starting at the
> > > beginning, but it's a valid use case, and I wouldn't like us to
> > > dismiss it.
> >
> > The beginning of the file is the part of it which we can fontify quickly
> > enough while still doing it correctly.
>
> I know, but we are not doing only what is easy to do, do we?  We do
> (or should do) what the users expect.  In this case, if we want to
> fontify some relatively small portion of the document, it should be
> the portion around where the file is first displayed.

There's no point in doing that. Either we narrow to some area around 
point (might even using a larger radius like 1 MB), or we we only 
fontify up to some position. The former easily creates bad fontification.

The alternative, of course, is to pay the price of syntax-ppss on larger 
spans and wait the corresponding amount of time the first time the user 
scrolls to EOB. That's what the current default on the branch does.

> > Because the main alternative on offer is to use narrowing and thus risk
> > getting invalid syntax information.
>
> That alternative is a direct and somewhat simplistic way of
> restricting misbehaving fontification, so as to prevent them from
> making Emacs unresponsive.  Smarter alternatives -- and you seem to be
> arguing for such -- should be smarter, not less so.  They should be
> like what you installed for JSON on master.

js-json-mode is not an "alternative", it's just doing what font-lock is 
supposed to do (I fixed the outstanding problems with the rules).

But as Gregory shows, when you get to _really_ large files (like 1 GB 
JSON file in his example), pressing M-> will still make you wait (I have 
to wait around 20 seconds).

It's still a great improvement from what we had in Emacs 28, though, so 
I'm fine with this default too.

> > So the "don't fontify past X" strategy is simply based on the idea
> > that no fontification is probably better than unreliable and
> > obviously incorrect one.
>
> I disagree with that idea, but if someone agrees with you, they can
> simply turn off font-lock.  As was already mentioned many times in
> this endless discussion.

If someone agrees with me, they will simply be able to customize 
font-lock-large-files to choose this strategy.

I do not really insist on it being the default. But being able to choose 
this approach (in the absence of better upcoming alternatives) is a good 
thing.

> > Now, we could develop more complex approaches from there. But this can
> > be a starting point, and the user option allows people to choose the
> > strategy they're most comfortable with.
>
> Sorry, I don't consider this (fontifying the beginning of a file) a
> good starting point for making any progress towards smarter, faster
> fontifications.  The new json-mode is such a starting point, but we
> should do something similar for other major modes as well.

I'm still waiting for people to come forward with other major modes 
which have the same kind of problems. Preferably ones that are likely to 
be used with large files.

But as mentioned, that does not preclude us from having to choose what 
to do with _really_ large files.