bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 05.08.2022 21:14, Eli Zaretskii wrote:
> > Date: Fri, 5 Aug 2022 21:02:35 +0300
> > Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> > From: Dmitry Gutov <dgutov@yandex.ru>
> >
> > > How is (parse-partial-sexp 1 (point-max)) related to the issue at
> > > hand?
> >
> > Or perhaps I should answer this way:
> >
> > We move to near EOB.
> > fontification-functions are called.
> >
> > jit-lock
> > calls
> > (font-lock-fontify-region point-near-buffer-end (point-max))
> > which calls
> > font-lock-fontify-syntactically-region
> > which calls both
> >     (syntax-propertize (point-max))
> >     and
> >     (syntax-ppss point-near-buffer-end) -> and it calls parse-partial-sexp
> >
> > syntax-propertize will also likely call syntax-ppss itself, probably
> > through the major mode's syntax-propertize-function. But if
> > syntax-propertize-function is nil, parse-partial-sexp gets called
> > anyway, over the whole buffer, which makes it the main workload in
> > fontifying near EOB.
> >
> > Now, if syntax-propertize-function is non-nil, parse-partial-sexp will
> > also call it, and it adds its overhead (sometimes a multiple of p-p-s),
> > which also scales linearly with the length of the buffer.
> >
> > So if one can demonstrate that (parse-partial-sexp (point-min)
> > (point-max)) takes about the same time as it takes to fontify the last
> > screen-ful of a buffer, then that says that everything else that
> > jit-lock does to fontify, is negligible, time-wise.
>
> So you have demonstrated that, if visiting a file and moving inside it
> calls parse-partial-sexp to scan the entire buffer, then this could be
> some, perhaps a large, part of the slowdown.

Yes.

> First, we need to establish that indeed parse-partial-sexp is called
> in that manner in the relevant major modes (not just one of them), or
> by font-lock itself regardless of the mode.

It is called by font-lock itself, which ends up calling syntax-ppss, 
which does its job with parse-partial-sexp. I have outlined the chain of 
calls in the previous message, you can verify it by looking at the sources.

> Second, we need to establish that indeed this takes a large portion of
> the time in the slow operations.  Not just one particular operation,
> but most or all of them.

To establish that, I have described the experiment in the grandparent 
email (with scenarios 1,2a;1,2b;1,2a,2b), and performed it myself as well.

But I'm talking about the slowdown observed when doing 'M->'. Not about 
any operations one might try to perform. Having said that, after the 
initial 'M->' most of navigation operations look snappy to me. So that's 
the slowdown I decided to investigate.

> And after that, we may have some food for thought.

Here's some more:

All major modes we can currently use for JSON (the built-in js-mode and 
the two json-mode's in ELPA) inherit the value of 
syntax-propertize-function from js-mode. But there's no need for it: 
JSON doesn't have division, or regexps, or preprocessor directives, or 
embedded JSX structures.

Setting syntax-propertize-function to nil speeds up parse-partial-sexp 
significantly. Here's a patch you can try to evaluate the effect on 
dictionary.json of that change combined with the previous tweak I 
suggested. Now it takes about 5x faster to fontify the last screenful, 
on my machine. Meaning, 'M->' feels almost (but not quite) instant. And 
the fontification is still correct.

A "proper" change would involve creating a new major mode, probably, 
rather than regexp-matching against buffer-file-name. But I'm not sure 
what name to pick: 'json-mode' would step on the toes of two existing 
packages now. 'js-json-mode', maybe? Or we bring in json-mode from GNU 
ELPA (with a similar change).

Anyway, try this please:

diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index eb2a1e4fcc..ae8e980125 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3418,7 +3418,8 @@ js-mode
               (list js--font-lock-keywords nil nil nil nil
                     '(font-lock-syntactic-face-function
                       . js-font-lock-syntactic-face-function)))
-  (setq-local syntax-propertize-function #'js-syntax-propertize)
+  (unless (and buffer-file-name (string-match-p "\\.json\\'" 
buffer-file-name))
+    (setq-local syntax-propertize-function #'js-syntax-propertize))
   (add-hook 'syntax-propertize-extend-region-functions
             #'syntax-propertize-multiline 'append 'local)
   (add-hook 'syntax-propertize-extend-region-functions
diff --git a/src/xdisp.c b/src/xdisp.c
index 099efed2db..fcb2be8768 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -4391,20 +4391,6 @@ handle_fontified_prop (struct it *it)

       eassert (it->end_charpos == ZV);

-      if (current_buffer->long_line_optimizations_p)
-       {
-         ptrdiff_t begv = it->narrowed_begv;
-         ptrdiff_t zv = it->narrowed_zv;
-         ptrdiff_t charpos = IT_CHARPOS (*it);
-         if (charpos < begv || charpos > zv)
-           {
-             begv = get_narrowed_begv (it->w, charpos);
-             zv = get_narrowed_zv (it->w, charpos);
-           }
-         narrow_to_region_internal (make_fixnum (begv), make_fixnum (zv), 
true);
-         specbind (Qrestrictions_locked, Qt);
-       }
-
       /* Don't allow Lisp that runs from 'fontification-functions'
         clear our face and image caches behind our back.  */
       it->f->inhibit_clear_image_cache = true;