Re: Anything better for delayed lexical evaluation than (lambda () ...)?

[Mark H Weaver]

Hi Andy,

Andy Wingo <address@hidden> writes:
> Am I missing something?  It has been a long thread :)

In case you haven't carefully read my earlier thread with David, I
wanted to briefly explain the difficulties as I understand them, from a
Schemer's perspective.  If I have misunderstood something, hopefully
David will correct me.

Most importantly, the design of the Lilypond language fundamentally
requires that parsing and execution are done simultaneously.  It is not
even possible to detect the boundaries between two statements without
runtime information.

To repeat an example that David provided earlier, consider a Lilypond
function that accepts one optional integer argument followed by a
required music argument.  When the parser/evaluator sees a call to this
function, it must determine the dynamic type of the first argument in
order to know where the function call ends and where the following code
begins.

The prime difficulty this causes for us is that when Scheme code
contains Lilypond code which contains Scheme code, e.g.:

>> (let ((xxx 2))
>>   #{ #(set! xxx (1+ xxx)) #})

In the general case, Lilypond needs to _execute_ the outer Scheme code
before the parser/evaluator is able to even _see_ the inner Scheme code,
because it needs to parse/evaluate the Lily code in between the two, and
we've already established that parsing cannot be not be done without
runtime information.

Therefore, the problem goes beyond simply Scheme within Scheme where the
inner Scheme is time-shifted.  That much could easily be handled by
closures like this:

(let ((xxx 2))
  (lambda () (set! xxx (1+ xxx))))

The problem is that we need to execute the (let ((xxx 2)) ...) part
before we have any idea what code is present in the "...".

Furthermore, it is not enough to simply provide an alist of lexical
variables at the point of the "...".  We need to provide a complete
environment sufficient to execute arbitrary Scheme code at that point,
which could include things like (set! xxx ...) or
(set! (procedure-with-setter xxx ...) ...) or whatever.

My suggestion is to provide something like a
(capture-lexical-environment) special form that would be put in place of
the "...", along with a "local-eval" that makes use of it.  Any
top-level form containing (capture-lexical-environment) would be
interpreted, not compiled.

Then, Lilypond could evaluate:

  (let ((xxx 2))
    (capture-lexical-environment))

which would return a lexical environment object, then proceed to
parse/evaluate the intervening Lilypond code, and then if needed could
call "local-eval" to evaluate any Scheme code within.  In the general
case this inner Scheme code could also contain a
(capture-lexical-environment), and so on.

How difficult would it be to implement this?

     Best,
      Mark