bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 03.08.2022 14:56, Eli Zaretskii wrote:
> > Date: Wed, 3 Aug 2022 03:26:02 +0300
> > From: Dmitry Gutov<dgutov@yandex.ru>
> > Cc:56682@debbugs.gnu.org,monnier@iro.umontreal.ca
> >
> > On 03.08.2022 03:00, Dmitry Gutov wrote:
> > > On 02.08.2022 19:19, Gregory Heytings wrote:
> > > > > > Try to open the dictionary.json with Emacs on master a month ago.
> > > > > I believe it can also be done with the current master, just after
> > > > > setting long-line-threshold to the nil value.  Right?
> > > > Indeed.  With master from one month ago it's even more crystal-clear
> > > > that you see the statu quo ante.
> > > If I set long-line-threshold to nil, does that also disable the
> > > redisplay optimizations related to long lines?
> > >
> > > Ones that caused scrolling delays even after the buffer has been fully
> > > fontified.
> > I mean those that*fixed*  the said scrolling delays, of course.
> I don't think I understand which changes you had in mind (could you
> give a less vague pointer?), but I think the answer is NO: that
> variable disables only the recent changes related to long lines.

I'm not sure if there are separate/specific changes to speak of (sorry, 
I'm flying blind), but previously I remarked in this discussion that on 
master pushing C-p can still result in sluggish response (at the end of 
a long line), even after the current window has been fontified (meaning 
font-lock has finished its work), and Gregory remarked that I should try 
the branch feature/long-lines-and-font-locking where this was fixed.

I did try the branch and indeed did not experience that sluggishness 
anymore. But that probably is a separate issue from slow font-lock.

So to try to debug any remaining speed issues with font-lock, it would 
be great to eliminate other sources of slow display first. In particular 
ones coming from a low-level subsystem I cannot as easily 
benchmark/debug/etc as Lisp code.

The setting

(setq long-line-threshold nil syntax-wholeline-max most-positive-fixnum)

indeed makes all the speed issues come back, including the 
aforementioned sluggishness.

You can try it yourself with downloadify.js I attached to the previous 
email. It's not a huge file either: the line is 90077 characters long.