bug-gnu-emacs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#67262: python-ts-mode cannot identify triple-quoted-strings


From: Yuan Fu
Subject: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Date: Tue, 12 Dec 2023 00:32:09 -0800
User-agent: Mozilla Thunderbird



On 11/26/23 4:05 PM, Dmitry Gutov wrote:
On 27/11/2023 01:43, Yuan Fu wrote:

On 11/26/23 6:58 AM, Dmitry Gutov wrote:
On 26/11/2023 04:04, Dmitry Gutov wrote:
As for what to do about this one -- probably something involving syntax-propertize-extend-region-functions, adding an entry which would initialize the parser, but not call syntax-ppss-flush-cache directly (or at least not just that). It would signal the earlier position to extend to through some dynamic variable. This is getting tricky enough to move from the individual major modes into treesit.el proper, I think.

Alternatively, we'd trigger updates eagerly from within treesit_record_change -- that would make it slower, invalidating the comment above it. Not sure by how much, though.

It seems to me that what we need is to force a re-parse at the beginning of syntax-propertize or in syntax-ppss-flush-cache; the re-parse would cause the notifier to run, which runs python--treesit-parser-after-change.

syntax-ppss-flush-cache is called by edits (and by the re-parse). It seems like it will be odd to have execution the other way around and/or add some hook into it which would call the re-parse and extend the region to be invalidated.

syntax-propertize could have another hook added, yes. Or an advice.

But it seems better to reuse some of the existing hooks, such as syntax-propertize-extend-region-functions. It treesit.c provided a way to fetch the newly-invalidated region, the treesit-major-mode-setup could add a new function to syntax-propertize-extend-region-functions which would invoke that feature. But even now it can instantiate the parse, which would call treesit-force-reparse internally, and then collect the info from the callbacks.
syntax-propertize-extend-region-functions looks perfect. We just need to force a reparse in it and the notifier will do the rest.

And yet another way - is to extend the region to be propertized from inside the major mode's syntax-propertize-function, invalidating some earlier entries too. The main problem with that, I think, is that every ts mode will have to repeat that trick. And that authors would have to know to do that. How to make that easier and more obvious, is a question.

Finally, if I'm right that bug#66732 has a similar cause, then a shared solution that can be reused by syntax and font-lock (or preferably just fix both in the same place) would be ideal.

I'm not quite sure about how do we cause this re-parse. The straightforward approach would be calling treesit-force-reparse[1] in syntax-propertize/syntax-ppss-flush-cache. But ideally I'd like to keep tree-sitter transparent for syntax.el. Maybe we can add a hook in syntax-propertize/syntax-ppss-flush-cache.

[1] This function doesn't exist yet, but it's easy to define in lisp.

treesit-parser-root-node calls it anyway and does little else, so we could get by with just using it.

Yep.

Yuan





reply via email to

[Prev in Thread] Current Thread [Next in Thread]