bug#62086: 29.0.60; ruby-ts-mode regressions

[Yuan Fu]

> On Apr 12, 2023, at 2:56 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 13/04/2023 00:50, Yuan Fu wrote:
>>> On Apr 12, 2023, at 1:13 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>>> 
>>> On 12/04/2023 18:31, Dmitry Gutov wrote:
>>>> On 12/04/2023 10:05, Yuan Fu wrote:
>>>>> Actually, would it make sense to define sexp as “anything but some very 
>>>>> small punctuation and delimiters”?
>>>> Pretty much. If I understood you correctly.
>>>> E.g. in ruby-ts-mode identifiers and numbers are also sexps.
>>> 
>>> Allow me to update that.
>>> 
>>> From the previous threads, for ruby-ts-mode at least, we seem to have 
>>> concluded that it's best to treat those nodes as sexps which have visible 
>>> boundaries that are visible and don't overlay exactly the boundaries of the 
>>> contained nodes.
>>> 
>>> For example, we now exclude statement nodes and binary expression nodes 
>>> because both make forward/backward-sexp less obvious and predictable: you 
>>> move point to the beginning of 'a + b', press C-M-f, and if the jump 
>>> happens over the whole expression, this is just as likely to mismatch the 
>>> user's intention (which might have wanted to only jump over 'a'). So these 
>>> are the node we rule out.
>> User might as well want to move over the whole expression, since they can 
>> use forward-word if they want to move over smaller elements. But I guess 
>> that’s just personal preferences.
> 
> forward-word works for minor elements, but the sub-expression can be, for 
> example, a parenthesized expression (with "real" parens).
> 
> It's definitely something that can be discussed, but the above guideline 
> seems to me like something that puts the user more in control. Because as 
> handy jumping over statements can be, it's usually not what one is trying to 
> do.
> 
>>> The easiest choice would be to go back to treating only 
>>> braces/brackets/parens are sexp delimiters, but in Ruby, at least, we have 
>>> lots of constructs that are delimited with keywords (such as 'if', 'def', 
>>> 'end'), so that doesn't work. Maybe it'll work better in C/C++, where you 
>>> mostly need to be able to differentiate between different types of angle 
>>> brackets.
>> To clarify, my point is to define sexp by exclusion rather than inclusion, 
>> ie, defining a set of nodes that are not sexp, rather than defining a set of 
>> nodes that are sexp. I mentioned delimiters because they are excluded from 
>> sexp, not because they delimit sexp.
> 
> Yes, that can work. Only when the excluded type names a one-char long, 
> though, because Elisp has no lookahead. In ruby-ts-mode there are longer 
> excluded types.

Actually, I’m working on extending the “pattern” treesit-search-forward and 
friends can accept. Right now it has to be a regexp or a pred function. I plan 
to extend it to regexp | function | (regexp . function) | (or <pattern>…) | 
(not <pattern>…) | (verbatim string)

I’m not yet sure about the performance implication of the recursive patterns 
(or and not). And I’m not sure if verbatim is necessary, but I guess having it 
wouldn’t hurt.

Yuan