Re: [Bug-zile] Nested buffers

[Gary V. Vaughan]

Hi Reuben,

On Feb 14, 2014, at 12:27 PM, Reuben Thomas <address@hidden> wrote:

> On 5 February 2014 22:30, Gary V. Vaughan <address@hidden> wrote:
> Hi Reuben,
> 
> On Feb 4, 2014, at 3:23 AM, Reuben Thomas <address@hidden> wrote:
> > I think it's also important to note this is a good fit with the structure 
> > of Lisp: while nested buffers can of course be represented naturally in Lua 
> > too (or even JSON), Lisp offers a more natural way to have them intepret 
> > themselves, because the relevant code (e.g. command) can be directly 
> > embedded in the data structure, without requiring a separate interpreter. 
> > Maybe this difference seems small, in that the "interpreter" in Lua could 
> > be trivial, but the nice thing about embedded code is that it's 
> > context-free, and can even take into account updates to the "interpreter".
> 
> All good ideas.  It's all just a simple matter of programming at this point ;)
> 
> But seriously, these are all directions that I think are worthwhile and fun.  
> I'm feeling more encouraged than ever to finish up sexpr and put a full-blown 
> (and hopefully reasonably fast) Lisp engine into Zmacs, which is perhaps the 
> limiting factor at this point?  In the mean time, pinning down a high-level 
> design for what a nested buffer actually entails, and how to implement it so 
> that the interactions are intuitive seems like a worthy next step to me.
> 
> Care to elaborate on your vision some more?
> 
> Sorry I took so long to get back to this.

No worries, I've been busy too :)

> The more I think about concrete coding, where you directly manipulate files, 
> store data structures directly on disk e.g. a list as a directory of files, 
> or an XML document as a nested directory structure, all the better intimately 
> to intermingle different languages and tools, the more I think sexps are an 
> ideal "abstract" representation, because they map easily to both concrete 
> on-disk and in-memory formats, and also are suitable for programming with 
> directly. The closest I've seen to trying to instantiate this vision is 
> http://lush.sourceforge.net/ though there the emphasis is also on seamless 
> integration with C (but I think that's an excellent thing too).

Right, that all makes perfect sense.  But, by "elaboration" I was angling for 
some specific steps to move forwards towards your vision.

For example, once I'm back to the point that zz is virtually merged into the 
combined tree at master HEAD, I want to rewrite the syntax highlighter over 
LPeg, but maintaining the TextMate inspired nested syntaxes.  I wonder whether 
this is close enough to what you envision that we might be part way to an 
implementation of a buffer that knows enough about the context of what it holds 
that you get some of the nested buffer features you want without the actual 
nesting?

Perhaps if you describe a specific user story for interacting with a completed 
nested buffer implementation, we can figure out some of the early steps needed 
to move in that direction?

> I'm unconvinced it's worth inventing another Lisp engine in Lua. Don't we 
> have enough?

We do?  I found some other toy implementations, but nothing that would make an 
efficient ELisp-a-like worth plugging into zmacs.  And many that were really 
toys with no ability to write useful extension code.

Anyway, I'm not sure I'm really inventing anything at this stage... once I'd 
simplified the previous zmacs Lisp implementation into idiomatic Lua, it became 
obvious that it cut a lot of corners (no eval/apply, no argument binding, no 
macros etc etc) that were easy enough to implement.  Once I'd put that stuff 
in, and added a REPL it seemed worth pulling out into its own project, since 
it's useful in its own right without zmacs wrapped around it.

The more interesting thing to me is that at this stage I now have 70% or so of 
a lisp->Lua compiler, which would vastly speed up lisp execution in zmacs by 
allowing it to save either the Lua code or even the Lua byte-codes for future 
execution, and of course being able to write real zmacs extensions in lisp, 
maybe even some of the zmacs implementation itself...  And I'm almost certain 
that opens the door to a lot more optimizations down the road too.

If you know of a project that is similar and already closer to those goals than 
the sexpr project I factored out of zmacs, I'm not too proud to abandon and/or 
merge what I have so far in favor of the better project.

Moving off at a bit of a tangent: Now that I've added enough functionality to 
zlisp to support robust testing, next on my TODO is migrating the old tests 
into full blown Specl specs, which was not possible with the old 
implementation, so we are definitely moving forward strongly IMHO.

And then, the next interesting item for me is to continue refining the 
interfaces between the Zile library and the editor implementations that I can 
finish merging the zz branch changesets into the new combined multi-editor tree 
in master HEAD, with a robust set of zmacs tests in place to ensure I keep 
everything working all the while.

Cheers,
-- 
Gary V. Vaughan (gary AT vaughan DOT pe)

signature.asc
Description: Message signed with OpenPGP using GPGMail