Re: Emacs contributions, C and Lisp

[Helmut Eller]

On Mon, Jan 12 2015, David Engster wrote:

> Helmut Eller writes:
>> On Sun, Jan 11 2015, David Engster wrote:
>>
>>> AFAIK, this is not possible with a mere GCC
>>> plugin.
>>
>> So what was your plan (or code, if any) before you abandoned this
>> project?

Thanks for the detailed answer!

> The first step would have been to replace our existing C++ parser with
> the AST that is produced by GCC. The plugin would output the same LISP
> structures that Semantic uses.

I'm a bit confused because at one side you seem to say that certain
things are not possible with plugins but at the other side you seem to
think that plugins can dump enough information to make these things
possible.

> My work so far was mainly to investigate
> how C++ types are actually stored in the AST. Especially the template
> stuff is pretty weird, and documentation is sparse. Fortunately, the
> headers are pretty well commented, but it still involves a *lot* of
> trial and error.

I can imagine that templates are complicated.  I tried to implement a
find-definition command as a GCC plugin.  My first approach was to
search the smallest subtree that contains a particular source location.
That didn't work out because GCC doesn't record "source ranges" so it's
difficult to know if a tree covers a particular location.  Another
problem is that identifiers are resolved early eg. "x + y" produces a
PLUS_EXPR (with the source location pointing to the + sign) but the
arguments are pointers to the VAR_DECLs of x and y and the source
location of those VAR_DECLs is typically a few lines earlier.

In a second attempt I made Emacs insert a custom #pragma at the place
where we want to search for a definition; similar to the gccsense
approach.  Plugins can register pragmas and that way have access to the
lexer.  That kinda works but the problem is that pragmas are only
allowed in certain places (eg. at the end of a statement) and Emacs has
to guess where those places are.

> The actual "semantic" part of parsing C++ would still be handled by
> Emacs' Semantic package. For instance, it would calculate
> completions. So obviously, those completions wouldn't match those from
> libclang w.r.t. to accuracy, but they would be *much* better than they
> are now, especially because the preprocessor is already handled, which
> is currently one of Semantic's main problems. Also, type inference would
> already be done by GCC, so you would see the resulting type from 'auto'
> and such.

Is the idea is to let GCC output some "global" information like type
declarations to enable better "local" parsing of function bodies in
Emacs?  Or do you want to do pretty much all parsing in GCC?

> One main problem would be how to parse the file you're currently working
> on, since it usually unparseable. I don't have a good answer for
> that. We would have our internal parser as fallback, but I would think
> that we could also try to internally make the file parseable before
> sending it to GCC. It would probably be very messy and involve a lot of
> closing braces.

Completion seems to be difficult problem. find-definition looks simpler
and perhaps better as a first step.

> So in a nutshell, for people familiar with clang: my approach is more
> like using libtooling/libast than libclang.

I know libclang only a bit and libtooling/libast not at all.  What I
failed to implement is like libclang's clang_getCursor.

[...]
> My plan was also to make this plugin usable for other tools. That means,
> it should not only output LISP structures, but alternatively also JSON
> and possibly XML. For instance, an external tool could build a symbol
> database for providing references. This could also serve as a starting
> point for doing refactoring. For more complicated tasks, the plugin
> could provide an AST matcher which you can query with certain
> expressions.

In general I think the heavy lifting should be done in GCC+plugin and
Emacs should only do the "easy" stuff like displaying the result.  But
for performance and other reasons it might be necessary to do at least
some parsing in Emacs too.

Helmut