Re: Giving "text quotes" syntax in font-lock-syntax-table only

[Stefan Monnier]

>> We think that the problem comes from running `syntax-ppss'.
>
> Yes, the problem is that `syntax-ppss` is affected by the current
> syntax-table yet it doesn't pay attention to syntax-table changes when
> it comes to enforcing that its cache is not stale.
>
> The arguably better way to solve the problem is to not change the
> syntax-table used inside `syntax-ppss` and `syntax-propertize`.  This is
> especially true for `syntax-propertize` if you use `syntax-ppss` in your
> `syntax-propertize-function`.
>
> The best way to do that is to set `syntax-ppss-table` to the
> syntax-table that should be used.

While I think the new approach using `font-lock-keywords` instead of
`font-lock-syntax-table` is a better solution, I think we should try and
avoid this breakage.

Could you test the patch below to see if it makes the old code
work correctly?


        Stefan


diff --git a/lisp/font-lock.el b/lisp/font-lock.el
index e0955b74abc..f4ac00d14bd 100644
--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -575,6 +575,7 @@ font-lock-syntax-table
   "Non-nil means use this syntax table for fontifying.
 If this is nil, the major mode's syntax table is used.
 This is normally set via `font-lock-defaults'.")
+(defvar-local font-lock--syntax-table-affects-ppss nil)
 
 (defvar font-lock-mark-block-function nil
   "Non-nil means use this function to mark a block of text.
@@ -1610,7 +1611,15 @@ font-lock-fontify-syntactically-region
               (regexp-quote
                (replace-regexp-in-string "^ *" "" comment-end))))
           ;; Find the `start' state.
-          (state (syntax-ppss start))
+          (state (if (or syntax-ppss-table
+                         (not font-lock--syntax-table-affects-ppss))
+                     (syntax-ppss start)
+                   ;; If `syntax-ppss' doesn't have its own syntax-table and
+                   ;; we have installed out own syntax-table which
+                   ;; differs from the standard one in ways which affects PPSS,
+                   ;; then we can't use `syntax-ppss' since that would pollute
+                   ;; and be polluted by its cache.
+                   (parse-partial-sexp (point-min) start)))
           face beg)
       (if loudly (message "Fontifying %s... (syntactically...)" (buffer-name)))
       ;;
@@ -1907,6 +1916,7 @@ font-lock-set-defaults
       ;; Case fold during regexp fontification?
       (setq-local font-lock-keywords-case-fold-search (nth 2 defaults))
       ;; Syntax table for regexp and syntactic fontification?
+      (kill-local-variable 'font-lock--syntax-table-affects-ppss)
       (if (null (nth 3 defaults))
           (setq-local font-lock-syntax-table nil)
        (setq-local font-lock-syntax-table (copy-syntax-table (syntax-table)))
@@ -1916,7 +1926,14 @@ font-lock-set-defaults
            (dolist (char (if (numberp (car selem))
                              (list (car selem))
                            (mapcar #'identity (car selem))))
-             (modify-syntax-entry char syntax font-lock-syntax-table)))))
+             (unless (memq (car (aref font-lock-syntax-table char))
+                           '(1 2 3))    ;"." "w" "_"
+               (setq font-lock--syntax-table-affects-ppss t))
+             (modify-syntax-entry char syntax font-lock-syntax-table)
+             (unless (memq (car (aref font-lock-syntax-table char))
+                           '(1 2 3))    ;"." "w" "_"
+               (setq font-lock--syntax-table-affects-ppss t))
+             ))))
       ;; (nth 4 defaults) used to hold 
`font-lock-beginning-of-syntax-function',
       ;; but that was removed in 25.1, so if it's a cons cell, we assume that
       ;; it's part of the variable alist.