bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 05.08.2022 22:14, Eli Zaretskii wrote:
> > Date: Fri, 5 Aug 2022 22:01:24 +0300
> > Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> > From: Dmitry Gutov <dgutov@yandex.ru>
> >
> > > First, we need to establish that indeed parse-partial-sexp is called
> > > in that manner in the relevant major modes (not just one of them), or
> > > by font-lock itself regardless of the mode.
> >
> > It is called by font-lock itself, which ends up calling syntax-ppss,
> > which does its job with parse-partial-sexp.
>
> In all modes, regardless of what the mode wants to highlight?

Yes. As part of highlighting strings, for instance. It needs to know 
which intervals of characters are strings. And you can't really know 
that without scanning the buffer from the beginning.

> > > Second, we need to establish that indeed this takes a large portion of
> > > the time in the slow operations.  Not just one particular operation,
> > > but most or all of them.
> >
> > To establish that, I have described the experiment in the grandparent
> > email (with scenarios 1,2a;1,2b;1,2a,2b), and performed it myself as well.
> >
> > But I'm talking about the slowdown observed when doing 'M->'. Not about
> > any operations one might try to perform. Having said that, after the
> > initial 'M->' most of navigation operations look snappy to me. So that's
> > the slowdown I decided to investigate.
>
> We need to look at more than just M->.  C-n/C-p, C-v/M-v, C-l are also
> important, as are the time it takes from typing M-x or M-: until you
> see the prompt in the minibuffer, and the time to update the display
> after inserting or deleting a single character.

I'm not seeing any particular sluggishness in these operations when 
visiting dictionary.json.

> > All major modes we can currently use for JSON (the built-in js-mode and
> > the two json-mode's in ELPA) inherit the value of
> > syntax-propertize-function from js-mode. But there's no need for it:
> > JSON doesn't have division, or regexps, or preprocessor directives, or
> > embedded JSX structures.
> >
> > Setting syntax-propertize-function to nil speeds up parse-partial-sexp
> > significantly.
>
> Thanks, so I guess we may have a solution for JSON files, if disabling
> syntax-propertize-function doesn't have any downsides.  What about
> other modes that we see in files with long lines, like XML?

Someone will need to test it with some typical large file. xml-mode 
(alias to nxml-mode) does have a syntax-propertize-function, but it's 
probably faster than js-syntax-propertize.

> And how scalable is the solution you propose, i.e. how it behaves in
> JSON files with a much longer lines?

parse-partial-sexp is O(length of text span)

Meaning, it scales linearly. You'll see a 10x delay in a JSON file that 
is 10x as large.