bug#74281: 30.0.91; font-lock mode hangs on scrolling large Scheme file

[Stefan Monnier]

> Stefan, what tools do we have to investigate slowness related to
> parse-partial-sexp?  Or maybe you have suggestions for how to speed up
> font-lock in this case?

Hmm... `parse-partial-sexp` is normally expected to be fast, unless it
has to scan a lot of text.

> Here's the profile I get while moving with C-p through the above file:

A stab in the dark, but maybe the relevant call is the one in:

          (state (if (or syntax-ppss-table
                         (not font-lock--syntax-table-affects-ppss))
                     (syntax-ppss start)
                   ;; If `syntax-ppss' doesn't have its own syntax-table and
                   ;; we have installed our own syntax-table which
                   ;; differs from the standard one in ways which affects PPSS,
                   ;; then we can't use `syntax-ppss' since that would pollute
                   ;; and be polluted by its cache.
                   (parse-partial-sexp (point-min) start)))

so the origin of the slowdown would be the (?#. "w 14") in the setting
below in `scheme.el`:

  (setq font-lock-defaults
        '((scheme-font-lock-keywords
           scheme-font-lock-keywords-1 scheme-font-lock-keywords-2)
          nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
          beginning-of-defun
          (font-lock-mark-block-function . mark-defun)))

in which case, setting a `syntax-ppss-table` should fix the problem, tho
we could also fix it by being more careful: AFAICT the purpose of this
(?#. "w 14") is only to change the syntax of `#` from "prefix" to "word"
without changing the comment-related flags, so it shouldn't cause
`font-lock--syntax-table-affects-ppss` to be set.
So, we could solve it by improving the code that sets
`font-lock--syntax-table-affects-ppss`, as in the patch below.


        Stefan


diff --git a/lisp/font-lock.el b/lisp/font-lock.el
index 203131bfd5a..f6299920c0a 100644
--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -1955,14 +1955,15 @@ font-lock-set-defaults
            (dolist (char (if (numberp (car selem))
                              (list (car selem))
                            (mapcar #'identity (car selem))))
-             (unless (memq (car (aref font-lock-syntax-table char))
-                           '(1 2 3))    ;"." "w" "_"
-               (setq font-lock--syntax-table-affects-ppss t))
-             (modify-syntax-entry char syntax font-lock-syntax-table)
-             (unless (memq (car (aref font-lock-syntax-table char))
-                           '(1 2 3))    ;"." "w" "_"
-               (setq font-lock--syntax-table-affects-ppss t))
-             ))))
+             (let ((old-syntax (aref font-lock-syntax-table char)))
+               (modify-syntax-entry char syntax font-lock-syntax-table)
+               (let ((new-syntax (aref font-lock-syntax-table char)))
+                 (unless (and (equal (cdr old-syntax) (cdr new-syntax))
+                              (memq (logand (car old-syntax) 255) '(1 2 3 6))
+                              (memq (logand (car new-syntax) 255) '(1 2 3 6))
+                              (equal (ash (car old-syntax) -8)
+                                     (ash (car new-syntax) -8)))
+                   (setq font-lock--syntax-table-affects-ppss t))))))))
       ;; (nth 4 defaults) used to hold 
`font-lock-beginning-of-syntax-function',
       ;; but that was removed in 25.1, so if it's a cons cell, we assume that
       ;; it's part of the variable alist.