Re: Tree sitter support for C-like languages

[Yuan Fu]

> On Nov 14, 2022, at 5:20 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Mon, 14 Nov 2022 00:23:20 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>> Theodor Thornhill <theo@thornhill.no>,
>> emacs-devel <emacs-devel@gnu.org>
>> 
>>>> Then when you insert the closing bracket, the parse tree is complete
>>>> 
>>>> int
>>>> foo (void)
>>>> {
>>>> int bar = 0;
>>>> }
>>>> 
>>>> Int is still in warning face because jit-lock doesn’t know it needs to be
>>>> refontified.
>>> 
>>> Doesn't tree-sitter tell us that the node for `int` has changed?
>> 
>> Yes and no, but mostly no. Tree-sitter can tell if a node “has changes”. But 
>> you need to keep the node updated as the buffer changes, which we currently 
>> don’t do.
> 
> Sorry, I don't understand: if the node's text did not change, and some
> other node (which did change) caused the first node to become
> "not-in-error", then why do we need to update the first node?  

Not specific to this node. I was saying that for any node to keep up with 
changes made to the buffer text, they need to be updated with “insertion in X, 
deletion from X to Y”. This is required by tree-sitter’s API. For this 
particular node, not updating the node might be ok, depending on hoe 
tree-sitter implements things. But of course we shouldn’t rely on that.

> And if
> the text of the node with the error did change, then we do update the
> node, don't we?

Well we update the parse tree and re-parse, but we currently don’t update the 
nodes created from the old tree. Keeping all nodes updated requires us to track 
all live nodes and update them whenever the buffer is edited.

>  So what is the problem here, exactly?  Or maybe I
> misunderstand what you mean by "update the node"?



> 
>> Even if we add this feature, I don’t know if “has changes” includes 
>> “previously inside an ERROR node but not anymore”. IIUC “has changes” means 
>> “corresponding text edited”. I need to add this feature and experiment with 
>> it to figure out what does “has changes” mean exactly.
> 
> Please do.  We must solve this problem.
> 
> Btw, do other IDEs that use tree-sitter have the same problem?  I
> doubt that, and if I'm right, we cannot afford having this problem in
> Emacs.

I wouldn’t call this a problem. The “error” in tree-sitter is not like complete 
parse failure. Let’s not highlight syntax errors for now, and see how it looks. 
In the meantime I’ll add the feature to track certain nodes for changes. Then 
if we decide this is an important feature to have, we can look at how to 
implement it.

I don’t think Atom highlight parse errors, neovim disables it by default.

> 
>> Keeping some nodes updated (ie, “watch” those nodes) isn’t too hard to 
>> implement, but it wouldn’t be a trivial change. I don’t know if we want to 
>> introduce non-trivial changes now.
> 
> If there are less invasive changes which could solve this, I agree.
> But if this is the only way, we have no choice, I think.  Again, it
> would be good to find out how other IDEs solve this.

Neovim used to highlight errors, but then disabled it by default[1]. I don’t 
know how does neovim fontify text, I will ask them if they have this problem 
and how did they solve it.

> 
> And don't worry too much about non-trivial changes, we have ample time
> before the release of Emacs 29 to find and fix any fallout.

Cool! Will do.

[1] 
https://github.com/nvim-treesitter/nvim-treesitter/commit/1a42056e092bc34ba081cb924bf0b3e3cd8cdc01

Yuan