bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 13.08.2022 20:43, Gregory Heytings wrote:
> > > I already did, and told you that these multi-second delays were still 
> > > present in larger JSON files (and therefore also in smaller files 
> > > with slower CPUs).
> >
> > Delays or "a delay"? Just the first time you scroll to EOB? Or did you 
> > do the whole dance with editing near BOB, then goto EOB, and repeat?
>
> Delays.  When scrolling to EOB and when using isearch, for example.
>
> > > > Then the goal failed because we don't disable bidi hpa? It has a 
> > > > much more noticeable effect on editing that font-lock.
> > >
> > > That's not correct, no.  The BPA algorithm makes C-n and C-p slower 
> > > at the beginning of large files only in a very specific case: when 
> > > the beginning of the buffer contains opening brackets whose matching 
> > > closing bracket is at the end of the buffer.
> >
> > To be fair, that's pretty much all JSON files.
>
> Again, the aim of this effort has _nothing_ to do with JSON files.

What kind of files are your main target?

> > The branch:
> >
> > 1) Allows everyone interested to evaluate the performance of 
> > unrestricted font-lock even in large files (single-line or not) and 
> > see how big on a problem the delays caused by syntax-ppss actually are 
> > in their experience. It's an important question.
> >
> > 2) Figure out the file sizes where syntax-ppss's performance really 
> > does become a problem. That can give us data for better defaults later.
>
> Buffers with long lines are rare enough.  It wouldn't make sense (and 
> would be a nightmare in terms of long term maintenance) to add 25 
> defcustoms to allow anyone to fine-tune what Emacs does in such buffers.

I have regularly encountered redisplay slowdown caused by long lines in 
my work. One doesn't need to have a 18 MB files to see them.

Having a 5000-10000 character line is enough to see redisplay starting 
to stutter. And, of course, there's nothing font-lock related in those 
stutters.

> > 3) Play around with two easy solutions that we discussed previously: 
> > narrowing during font-lock (one of the values for 
> > font-lock-large-files pretty much matches the current behavior on 
> > master), or fontifying only the first X characters (e.g. 1000000) of 
> > the buffer, and skipping on the rest.
> >
> > 4) It should be plain to see that implementing additional approaches 
> > should be easy enough. For instance, a hybrid like "fontify the first 
> > 1 MB correctly, and the rest - on best-effort basis". Although the 
> > value '(narrow . 1000000) should provide behavior a very similar 
> > behavior already. Maybe ever a better one: the boundaries are stable. 
> > Maybe sure to try it together with (setq bidi-inhibit-hpa t): the 
> > result looks very fast to me.
>
> These 3) and 4) assume that the major (and minor) modes are 
> well-behaving, and will obey the restrictions.  Which isn't true, and 
> was the reason why locked narrowing was introduced.

What are your examples of misbehaving major/minor modes?