Re: tree-sitter: conceptional problem solvable at Emacs' level?

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Emacs-devel@gnu.org
> Date: Thu, 09 Feb 2023 16:17:27 +0800
> 
> Holger Schurig <holgerschurig@gmail.com> writes:
> 
> > Hi, I run branch emacs-29 since some time with great success. And now I
> > wanted to test out tree-sitter and c++-test-mode. Unfortunately, I
> > stumbled into some conceptional problems and wonder if this is actually
> > solvable by Emacs, or if some would need a completely new grammar.
> >
> > The issue is: tree-sitter doesn't work well with C macros.
> >
> > I program a lot in C++/Qt. So let's look at this (valid) C++ program:
> >
> > -----------------------------------------------------------------------------
> > #include <QObject>
> >
> > class Test : public QObject
> > {
> >         Q_OBJECT
> > public:
> >         Test() : QObject() {};
> > public slots:
> >         void someSlot() {};
> > };
> > -----------------------------------------------------------------------------
> >
> > If have the libraries installed (e.g. qtbase5-dev on Debian), you can
> > compile this perfectly.
> >
> > However, tree-sitter produces a garbage syntax tree:
> >
> > - contain some bitfield node (which isn't really there)
> > - contains an error node (despite the code being compilable)
> >
> > And as a result, BOTH the indentation and the font-locking is wrong.
> >
> >
> > Would I need to create a tree-sitter grammar in JavaScript that
> > understands this macro-enhanced C++?   That would be quite difficult.
> > Or will there be a method to add some kind of tiny-preprocessor to
> > c++-ts-mode, so that it can substitute "Q_OBJECT", "signals" and "slots"
> > with nothing before handing things over to tree-sitter?
> >
> >
> > In comparison, I could teach the old cc-mode about this macro-enriched
> > C++ just with
> >
> >   (c-add-style "qt-gnu"
> >                '("gnu" (c-access-key .
> >                        
> > "\\<\\(signals\\|public\\|protected\\|private\\|public
> >                slots\\|protected slots\\|private slots\\):")))
> >
> >
> > I guess that a lot of C and C++ programs use macros. And if there is no
> > simple way to aid tree-sitter in understanding this, then I fear
> > tree-sitter enhanced modes will often be unusable on them.
> 
> My suggestion is simply to stay with CC Mode.

Suggestions for what to do for now aside, I would still want us to try
to figure out the possibilities for better handling of C/C++ macros in
tree-sitter supported modes.  I don't want to give up yet, because the
kludges similar to c-add-style used by CC mode might be possible with
tree-sitter modes as well.  Or maybe some other solution could work,
including the idea of letting tree-sitter see preprocessed source code
(although this is probably harder to implement, and must be done on
the C level).

We just started using these modes in Emacs, so it is small wonder that
issues like this are popping up, and will probably keep popping up for
some time to come.  I see no reason whatsoever to give up on
tree-sitter just because these minor problems in marginal cases are
brought up; we should instead solve them one by one.  Being minor
problems, they in no way invalidate the basic decision to try using
tree-sitter in Emacs, not from where I stand.