Re: IDE

[Eric Ludlam]

On 10/11/2015 12:38 AM, Dmitry Gutov wrote:
> Hi Eric,
>
> On 10/10/2015 07:48 PM, Eric Ludlam wrote:
>
> > I had always intended CEDET to be a baseline for IDE like features.
> > Looking just at tagging files, those familiar with it's internals
> > recognize that it can use external tools as weak as ctags or GLOBAL, and
> > can use a more powerful external tool for parsing as well, such as using
> > JAVAC for decompiling .jar files into tags.
>
> It sounds good if all I have is a parser/tagger.
>
> But if I already have an external tool that can give me a set of 
> completions at a given position in a given buffer, and also provides a 
> go-to-definition feature, what's the advantage of going through 
> Semantic? SRecode support?

CEDET's tools are set up in layers.  As an language support author you 
can start with a series of overrides for writing a lexer or parser 
yourself.  If you have a tool that parses already (such as exuberent 
CTags), you can instead just start at the high level tagging parser and 
skip all the lower level stuff.

If you have an existing completion engine for cases where you happen to 
have an interpreter with completion, or something else, you can just 
override the completion engine directly.  The srecode tool does this, 
and there is an experimental clang version in CEDET's repository as well.

Having overrides at all 3 levels is the best, since different tools ask 
for different features, but it is not necessary as there is backup 
implementation for the higher level features.

> Not to mention that we have tools like Jedi that only (mostly) do 
> these two things, but don't provide dumps of the syntax tree.
>
> > As a backup, it also has
> > in-buffer parsing for when you've only half-written your code, or when
> > no external tool is available.
>
> That assumes we also have a Semantic grammar for the said language. 
> And then, we'll need to duplicate whatever logic is used by the 
> external tool to disambiguate between same-named methods in different 
> classes.
>
> If that's even possible.

Of course.  For strongly typed languages the problems are deterministic 
so it's possible.  For dynamic languages, they usually provide something 
you can ask.  It's just a matter of integration.

I enjoy working on puzzles like this, but for me it is a matter of free 
time.  Kids, job, etc get in the way.

> > The same philosophy is throughout CEDET. Unfortunately, those who are
> > interested in tooling who brush casually past CEDET only see that people
> > complain that it doesn't always complete right, or that the C++ part is
> > hard to setup
>
> The example we've been given in one of the previous discussions of C++ 
> support is that Semantic can't tell the difference between different 
> methods with the same name? If Z#foo returns type A, and T#foo returns 
> type B, it's been shown that Semantic doesn't know which is the type 
> 'x.foo'.
>
> Given this, how would we expect it to properly support languages with 
> type inference like Scala or Go? Not to mention dynamic languages 
> which normally have no type annotations on methods, and often use more 
> complicated algorithms for accurate code completion.

Semantic has the information to differentiate between two methods of 
different types.  There are still a few short-cuts in the completion 
engine  where it could do a little more to disambiguate.  There is a 
function to '-select-best-tag' in the analyzer, but at the end, it just 
grabs the first element from the list of possibilities.  If you have a 
particular use case in mind, it would good to have a simple example that 
shows what it is and maybe it will be easy to update.

> > If we look at the two starting key features listed for IDEs,
> > "completion" and "refactoring", the basics are all there.  CEDET has a
> > completion engine, but it is only as good as the input data.
>
> It's limited by the input data, sure, but it also has to know what to 
> do with it.
>
> If only implementing accurate code completion was as easy as writing a 
> language parser.

This is a matter of time and desire.  For the code I've been writing 
lately (arduino hobby stuff), it has been more than sufficient. David 
was interested in handling templates and was able to add support for 
that.  If someone wants it to handle something new, the data is there to 
do it.

> > CEDET also has 'srecode' which is about
> > generating code.  This bit is trickier, as the intention is you could
> > write one highlevel tool that might extract tag data from your file,
> > permute the language-independent tags, and then write them back out with
> > srecode.  There are tests showing this working, but they are simple and
> > proof of concept and not stuff someone would depend on day-to-day.
>
> SRecode should be in a better position, but I imagine it would also 
> fail often enough in dynamic languages.

I'm not quite sure what the intention is here.  Srecode is just template 
expansion, with mechanisms to enable multiple layers of templates, and 
support for "application" templates and user override templates for 
customization.

What's missing is someone taking the time to start building the logic to 
use semantic to pull buffers apart, and srecode to re-write the code.

> And in any case, one has to teach Semantic about scoping rules for 
> each given language, for SRecode to only rename, say, a variable 'foo' 
> but not function 'foo', or only the variable 'bar', but not the 
> variable 'bar' up the scope that's being shadowed.

There is a fair amount of logic for supporting language scopes. There is 
almost nothing about running filters for "all the calls to symbol FOO 
that has some particular feature".  The symref tool has a comment in it 
that says "do the filtering here", but no one has gotten around to it.

My assumption is that it would be easier to start there than start from 
scratch, unless you just hijack some external tool.

Eric