Re: Re : Re : Re : [Chicken-users] Scheme (LISP) editing

[Peter Keller]

On Sat, Mar 03, 2007 at 12:22:34AM +0100, minh thu wrote:
> Do you have an opinion regarding havind bare s-exp or sxml
> representation of the code ?

Honestly, the only thing xml is good for is for interchange of data
since so many people agree on what it looks like--but I'm bitter. :)

If you designed your own s-exp structures for code representation, or you
used sxml, you still have the arduous task of deciding what the actual
information representation structure will look like. You can't escape
that real part of the work and it is just as difficult in either form.

The nice thing about either method is that it will always be convertable
to the other representation form. So, I'd say pick a "native" model
representation form (probably raw s-exp) which best captures the problem
domain of the idea and write everything to use that.  You can always
read/write sxml representations of that model to trade between other
people. I recommend this because the native model view of the problem
domain, if well designed and efficient, means a helluva lot less "data
structure conversion code" to manipulate it.

The tricky part is that since this is an editor, it has to be line, and
also, s-exp oriented. Sometimes those might conflict in interesting ways,
especially in terms of incomplete forms or just non-scheme code.

As for ideas for features the editor (which I'm imagining can be a scheme
IDE) should have, I can spew them forever:

1. Code is never thought about as being in "files". Instead, the user
creates "views" of codes.  Initial (and permanently available) views are
human constructed collections of related code, e.g., all codes relating
to some hash table implementation.  This is a little different than say
eclipse because eclipse only shows one class at a time, and you can't
really group classes into logical blocks (unless they are part of a
package, which might not be the case in your mind), and it is different
than VC++ because that is file oriented, or suffers from the same fine
granularity problem as eclipse with the class view (and friends).

Here is a very subtle psychological observation of how humans write code.
We programmers create a 2 dimensional plane which create a "slice"
through the high dimensional execution space of conceptually linear
algorithms in a (usual) top down fashion on said plane. Function invocations
can be thought of as moving in a direction perpendicular to the plane
of viewing to another plane which contains said implementation.

This is a product of both our reading/language skills and because we
copied those skills when it came time to create computers. In hindsight,
we had no other reason to pick anything different, but to continue it
now is a damn dirty shame....

The problem is that while we can write the code that way, debugging
_unanticipated execution flow problems_ (something that happens every
day when writing code) or looking at it from other meaningful slices is
impossibly difficult and done with extreme pain because we are stuck with
the initial (and only) view that we had originally created. An editor like
eclipse tried to solve this by making a 2d view the smallest conceptual
unit possible as defined in the language (a class). In my mind, this
is too damn fine grained, since if a class references a bunch of other
classes (and they always do) you end up having a stack of windows and
digging through them to follow a code path of execution. By the time you
finish clicking the mouse, not only are you lost, but you probably can't
get back to where you were. Traditional vim/emacs goes the other extreme,
you are left with the exact 2D slices you used to create the software
and come hell or high water you will never be able to get a complete
picture of an execution path or other aspect-like query through many
(possibly dozens) of 2D slices. I *HATE* that.

2. With views as a first class entity, now you can start to perform some
extremely powerful source queries, like: give me a view of everyone who
uses a particular function in one "editor pane" (think of a traditional
window of text in an editor), perform a callgraph analysis of this
function and give me an editor pane of all the functions it ever ends
up calling, give me an editor pane of all leaf functions, show me all
functions where I perform recursion, show me all functions which match
a regex, show me all functions which have in them a specified s-exp,
show me all functions where this global variable is accessed, etc,
etc, etc.  It becomes easy to imagine a map/filter like source code
query system: (filter all functions that use this global variable,
then map this tree-rewriter across the enclosing form for that global
variable reference).

3. Mark two points, A and B, in the codebase. Then, the editor finds all
possible paths through the codebase (assuming perfect condition branches)
from A to B. It presents this information to you as a graph with possibly
many paths going from A to B. If you click on a path, you are shows the
view of code that moves you from A to B with the code paths you'd have to
take highlighted in some manner. If those code paths are not reachable
(what that means in practice needs thought), then the graph will be
the empty set of edges. Theoretically, you can mark a trail through the
source and can ask for a path which hits all of the code marks you made.

The place you'd use this is if you were looking at a logfile and saw a
bunch of debugging output detailing which functions had been called in
the order they were called. I could simply tell my editor that ordered
list and then maybe be presented with 6 or 7 callgraph views of the
codebase which *could* accomplish that. I'd click on one, and get only
the source code which could perform that path.

If debugging is enabled, this idea could show me the view of all functions
to this stack frame with binding annotations galore. Think about how
awesome that would be.

4. Instead of basic search and replace, and better than simple "refactor
this variable or function to have a different name", allow people to
specify a regex-like s-exp tree representation, then a function to
pass that s-exp into that you write in the editor, and then perform the
replacement of the s-exp with the rewritten code, performing necessary
refactoring afterwards. It is the next logical step in the evolution
of refactoring.

I don't know how many times I've seen copied code where if I change one
piece of it to some other type of code, now I have to change all of them
(but keeping the local context like debugging info mesages and stuff)
in many different files (and due to some architectural decisions in the
code base, that code couldn't be lifted to a different layer). Let the
tree rewriter do that instead.

5. I haven't even gotten to anything which would use the true 3d capabilities
of what you are writing. :) I'll leave those for later....

At this point screw even thinking about how you'll implement it, just
story board it, draw pictures of what you want it to look like doing
various things. Get a bunch of cool stuff drawn/written down, THEN
start thinking about how to implement it and what you should implement
first. Your project could be so interesting that if you just start coding
a few neat things right away, you won't see the proper abstractions to
do everything you'd like in a timely manner and then if you just keep
pounding through it, you'll end up with a tremendous amount of code which
mostly converts one model form to another before even doing anything with
those models. Then you'll get bored and stop working on it since keeping
all of that pseudo-architectural stuff in your head will have become
too complex to do in your freetime. :) Features beget abstractions,
abstractions beget plans, plans beget milestones, milestones beget
finished products.

-pete