bug#67061: [PATCH] Improve syntax highlighting for python-ts-mode

[Dmitry Gutov]

On 12/12/2023 10:24, Yuan Fu wrote:
> > > I think "for var in range(3)" should be part of the "definition" 
> > feature
> >   because a variable is defined there. Alongside parameters.
> > I added it to definitions.
>
> Again, if we think of fontification levels, the definition feature is 
> about fontifying the function names of definitions, and it's at a low 
> level (level 1). Non-essential fontification like "var" shouldn't be 
> activated at that level. So I suggest to put it in the variable feature, 
> along with many other non-essential fontifications. (Variable feature is 
> placed at level 4.)

I disagree: 'var' in this example is not much different from a function 
parameter. It's a definite place where a variable's name introduced in 
the current scope.

Python doesn't have special keywords for variable declarations (unlike 
'let' in JavaScript or typed declaration in C), so the first time a 
variable is introduced serves as its declaration. For assignments, we 
can't easily determine which is the first time for a given scope, but 
examples like 'for var in ...' or 'except ZeroDivisionError as e:' or 
'[... for var in ...]' are all unambiguously variable definitions.

So I think that:

a) All variable definitions (functions parameters or not) should use 
font-lock-variable-name-face -- to make it easier to find where a given 
symbol is introduced.
b) No font-lock-variable-name-face highlights should be put into the 
'variable' feature, which is disabled by default. All of the examples 
above should either go into 'definition', or if somebody does like that 
approach, into some new 'variable-declaration' feature (enabled by 
default). And maybe some into 'assignment', which is on feature level 3.
c) The 'variable' feature should, at this point, only contain the 
relatively useless highlights, since we don't track variable lifetimes 
yet. That's why it's disabled by default.

The current situation across ts modes is that js-ts-mode has variable 
declarations in the 'definition' feature (and not by my hand, FWIW); 
ruby-ts-mode has a separate 'parameter-definition' feature that 
encompasses both parameters and other variables; in c-ts-mode 
highlighting for 'int i = 4' is split between 'definition' and 
'assignment' (the latter seemingly redundant); typescript-ts-mode and 
rust-ts-mode also follow the principle, more or less.

> > My thoughts about parameters. I started to extend rules for them since 
> > they are very limited now.
> > But I'm not sure what face to use for them.
> > I would like to not use the same face as for assignments, because I'd 
> > want to highlight them differently.
> > It seems that there is no appropriate face in font-lock.el, so I ended 
> > up creating my own face in my config.
> > Does it make sense to add new face for parameters in font-lock.el?
> > Or it is too small feature for its own face?
> > I also apply this face for keyword argument in function calls.
> To be honest, I don't have any good ideas. Perhaps we can add a 
> parameter face that inherits from variable name face by default, Dmitry, 
> WDYT?

As per above, parameters don't seem too different from any other 
variable declarations from my POV. They are similarly useful, so I'd 
highlight them the same way.

Do we want to have a common face which would inherit from 
font-lock-variable-name-face and would be used solely for 
function/methods parameters and nothing else? I don't object, but I 
don't quite see the point either.