|
| From: | Yuan Fu |
| Subject: | bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably |
| Date: | Mon, 11 Dec 2023 23:50:53 -0800 |
| User-agent: | Mozilla Thunderbird |
On 12/11/23 7:53 AM, Dmitry Gutov wrote:
On 11/12/2023 06:16, Yuan Fu wrote:The way we achieves this is thought parser notifiers. When tree-sitter parser reparses, it notifies us of which part of the buffer was affected by the reparse. For font-lock, we have this font-lock notifier that marks the affected buffer region as "not fontified", so redisplay will refontify those areas.(defun treesit--font-lock-notifier (ranges parser) "Ensures updated parts of the parse-tree are refontified. RANGES is a list of (BEG . END) ranges, PARSER is the tree-sitter parser notifying of the change." (with-current-buffer (treesit-parser-buffer parser) (dolist (range ranges) (when treesit--font-lock-verbose (message "Notifier received range: %s-%s" (car range) (cdr range))) (with-silent-modifications (put-text-property (car range) (cdr range) 'fontified nil)))))This notifier function will be called during redisplay [1]. I suspect that because of this timing, redisplay doesn't refontify the marked region immediately. So I added a timer, I think that ensures we mark the affected region in the next command loop?(defun treesit--font-lock-notifier (ranges parser) "Ensures updated parts of the parse-tree are refontified. RANGES is a list of (BEG . END) ranges, PARSER is the tree-sitter parser notifying of the change." (with-current-buffer (treesit-parser-buffer parser) (dolist (range ranges) (when treesit--font-lock-verbose (message "Notifier received range: %s-%s" (car range) (cdr range))) (run-with-timer 0 nil (lambda () (with-silent-modifications (put-text-property (car range) (cdr range) 'fontified nil)))))))This seems to work. Eli, do you see any problem using run-with-timer this way? What's the correct way to mark some region unfontified?[1] The chain of events if roughly: user types the last "/" -> redisplay -> fontify that character -> access parser -> parser reparses -> calls notifier.Note that it's not just font-lock. syntax-propertize has the same problem (I've described it in https://debbugs.gnu.org/67262#23).And a timer wouldn't help because syntax-ppss needs to have up-to-date information whenever it's called, not later.
I feel that font-lock and syntax-ppss have different problems and requires different solutions. For font-lock, we need to mark affected range unfontified; we just need to make sure we do that after jit-lock-fontify-now. For syntax-ppss, we need to force a reparse before entering syntax-ppss so that syntax text properties are up-to-date when syntax-ppss do its work.
c-ts-mode--emacs-set-ranges is registered as a range rule, so many tree-sitter function calls it before doing anything to make sure range is up-to-date. treesit-font-lock-fontify-region calls treesit-update-ranges at the beginning of its body, and treesit-update-ranges calls c-ts-mode--emacs-set-ranges.Here's a draft solution based on *-extend-region-functions, attached.Alas, while it works fine in python-ts-mode (for both syntax and font-lock), making it behave better than python-mode, in c-ts-mode it doesn't quite have the same effect: when you backspace over the closing "/", the highlighting is properly updated only after you make the next edit (any edit), or select another window. I'm not sure, though, if it's due to my own problems with Emacs's failure to redisplay (reported elsewhere), so more testing is welcome.But that might also be related to the use of c-ts-mode--emacs-set-ranges: printing a backtrace calls inside treesit--font-lock-notifier shows that the last notification comes also during font-lock but after treesit--font-lock-extend-region, inside c-ts-mode--emacs-set-ranges. I don't quite understand this design where the ranges are applied inside the font-lock code.
Yuan
| [Prev in Thread] | Current Thread | [Next in Thread] |