Re: Anything better for delayed lexical evaluation than (lambda () ...)?

[David Kastrup]

Mark H Weaver <address@hidden> writes:

> David Kastrup <address@hidden> writes:
>> Basically I need to evaluate dynamic code in a given lexical environment
>> rather than at top and/or module level.
>>
>> For a language that is supposed to be a building block for extension
>> languages, not really a concept that is all that unusual I would think.
>
> Guile 2 is an excellent base for building extension languages, but not
> in the way that you'd like to do it.  Unfortunately, I see no way to
> support `procedure-environment' on arbitrary procedures without
> abandoning our optimizing compiler and going back to a simple
> evaluator.

Sure.  In an optimizing compiler, I would expect "procedure-environment"
to only contain actually used parts of the environment, and that would
make (procedure-environment (lambda () '())) fabulously useless.  And of
course I am not interested in the environment of that procedure, but
rather in that where procedure-environment is called.

> I suspect it would be possible to implement a special form that
> captures its lexical environment in such a way that arbitrary code
> could later be evaluated within that lexical environment.  The
> presence of this special form would impose onerous constraints on the
> optimizer within the top-level form containing it.  In fact, I can't
> think of an optimization that would still be possible, because the
> compiler would have to assume the worst: that some other thread could,
> at any time, mutate any lexical variable or call any lexical procedure
> visible from the special form.  It gets even worse when you consider
> first-class continuations.

We are calling the Lilypond parser in that "top-level form".  The
optimizer is not much of a worry.

> I believe that this is the wrong approach, though it may be worth
> considering for the sake of allowing Lilypond to continue using its
> existing implementation strategy.

Uh, we are not talking about "implementation strategy" but language
features.

> In general, the _right_ way to build a custom extension language using
> Guile 2 is to write a compiler that converts your language into one of
> the other languages that Guile 2 supports.

Lilypond is not Scheme.  It has syntax ambiguities that are resolved by
lexical tie-ins and thus depend on the context.  You can't easily
compile it in advance.

And you are _totally_ putting the cart before the horse here.  Lilypond
is not supposed to be an extension language for Guile, but Guile is
supposed to be an extension language for Lilypond.  The acronym Guile
stands for "GNU's Ubiquitous Intelligent Language for Extension".  You
are losing sight of what Guile is supposed to be.

As an extension language, it does not make sense that it dictates the
lexical and the program structure of the system it is supposed to be
extending.

> If there's something about Lilypond's language that you believe would
> make compilation impractical, let's talk about it.

Its syntax and semantics.

<URL:http://git.savannah.gnu.org/cgit/lilypond.git/tree/lily/parser.yy>

If I call a function with an optional argument of type integer? before
an argument of type ly:music? and I encounter #x, then the value of x
decides whether this argument will be used as the optional argument or
as the following argument.  The rest of the parsing has to follow.

> Maybe the Guile experts on this list can find a clever solution, or
> else maybe we can enhance Guile to support Lilypond's language in a
> straightforward manner.
>
> I would be glad to help with this.  In the long run, it might be less
> work for us Guile hackers to implement a nice compiler for Lilypond than
> to implement and forever maintain the "capture-lexical-environment"
> special form, and it would almost certainly have better results.

You are working from the premise that Guile should govern the
architecture of the system it is supposed to be extending.  That
Lilypond is one of the few serious systems actually using Guile as an
extension language does not make it a good idea to turn it into a system
that Guile uses as its extension.  The hard way.  With lots of work.
You can't expect that kind of investment to be done for every
application that considers using Guile.

-- 
David Kastrup