Re: Update 1 on Bytecode Offset tracking

[Stefan Monnier]

> Sure, I'll do some more exhaustive testing. So far though, the results
> aren't great, the biggest issue being memory usage. The
> `source-map-read` can GC over 5 times more often than `read`.

Sounds fine for a prototype.

> I think it's more sensible to accept that a real implementation will
> have to be in C and this reader will just remain a prototype.

Indeed.

> Aha, I had never even considered hygienic macros in Elisp (nor had I
> recognized how trivial it is to track their source-code).  That would be
> an amazing development for Emacs Lisp, but is certainly a huge
> undertaking, not something I could fit into the GSoC timeline.

No, I'm just discussing what the longer-run might look like.

> I know that it has been done in Common Lisp (by Pascal Costanza), but
> I believe that implementation serves the sole purpose of capture
> avoidance and doesn't abstract syntax. For Emacs I assume this would
> have to be done in C, but I do wonder if an Elisp implementation would
> be possible.

I haven't thought very much about it, but I can't see any reason why it
would need to be done in C, no (tho I wouldn't be surprised if it could
benefit from a bit of help from the C side, of course).

> Doing the similar thing in Elisp -- relegating source location tracking
> to code using only a specialized kind of macro, hygienic or otherwise --
> would of course be a major loss, since it would take years for that new
> paradigm to become commonplace.

Indeed, we'll need some fallback heuristic for all the existing
`defmacro`s.

Part of the issue is "tracking source location" but another important
part is to take the annotated source code and "de-annotate" it
(recursively) to pass it to the macro, since the macro expects
a raw sexp.

That's why we've been thinking about annotated representations
which are "transparent" (i.e. can be used as if they weren't annotated).
Either using "fat cons-cells" or using "fat symbols" or storing the
annotations in an eq-hash-table.

Another way to attack the problem is to rely on the Edebug spec: you can
refrain from de-annotating all the parts marked as `form` or `body` (as
long as the annotations themselves look sufficiently like normal code,
at least).


        Stefan