bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii]

> Date: Tue, 2 Aug 2022 04:05:57 +0300
> Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> >> The alternative being that font-lock would call syntax-ppss right away
> >> with no restriction, but then only apply highlighting to limited parts
> >> of the buffer.
> > 
> > AFAIU, this seems to assume that highlighting is much faster than
> > syntax-ppss.  Is that a given?  If not, I don't think I understand how
> > this could help.
> 
> I don't have the concrete numbers at hand, but from experience I'd say:
> 
> - syntax-ppss over the whole buffer is fast-ish. But it takes O(N) time 
> of course, and the bigger the buffer is, the longer it'll take. Not much 
> we can do about it.
> - font-lock has to do more work, so over the whole buffer it will take 
> an order of a magnitude more time than syntax-ppss.
> 
> Further:
> 
> - syntax-ppss is also important for correctness: for commands to 
> understand whether the point is inside a string, comments, etc. So it's 
> better to avoid applying narrowing when calling it. Unless you're in a 
> multiple-major-modes situation.
> - font-lock calls syntax-ppss.

I believe I was talking about syntax-ppss being called from font-lock,
indeed.  Before Gregory's changes, if you visit a large file with very
long lines, and interrupt Emacs while it is non-responsive, you will
in many/most cases find yourself in syntax-propertize or its
subroutines, and you will see that it is almost always called to
traverse the entire long line.

> So ideally font-lock is either called with undo-able narrowing, or is 
> simply passed a range of positions, and shouldn't fontify too far from them.

Many major-modes do widen the buffer, though.

> The latter seems to be the case already (if you open xdisp.c and press 
> M->, only top and bottom of the buffer are fontified)

It is not enough to look for faces in order to realize how much of the
buffer was scanned.

> with the caveat that font-lock always tries to backtrack to BOL when
> fontifying the current hunk. Which makes sense, of course, but could
> be tweaked for long lines to avoid re-fontifying the whole buffer
> again and again.

"Tweaked" how?

> IOW, IIUC the fix for font-lock performance could be better implemented 
> inside font-lock itself, as long as all the info about whether the 
> current line is "long" is available to Lisp.

No one will object to making font-lock faster.  But the experts who
can do that are few and far in-between, and seem to have other itches
to scratch, since these issues are known for a long time, and several
times were even discussed at length.