guile-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: macroexpand-1


From: Catonano
Subject: Re: macroexpand-1
Date: Thu, 31 May 2018 10:21:19 +0200

Mark,

thank you very much for explaining at lenght, I appreciate that !

2018-05-30 3:07 GMT+02:00 Mark H Weaver <address@hidden>:

> Hi,
>
> Catonano <address@hidden> writes:
>
> > 2018-05-29 17:01 GMT+02:00 Mark H Weaver <address@hidden>:
> >  > what's the problem with macroexpand-1 and syntax-case ?
> >
> >  In Guile 1.x, 'macroexpand-1' performed a single macro expansion step at
> >  the top-level form of an expression, using its old non-hygienic macro
> >  expander.  There are several problems with trying to provide such an
> >  interface in a Hygienic macro expander, and especially in the
> >  'syntax-case' expander with its support for 'datum->syntax'.  For one
> >  thing, our modern macro expander doesn't even work with the plain
> >  S-expressions which 'macroexpand-1' accepted and produced.  It works
> >  with "syntax objects", which effectively annotate every identifier with
> >  extra information needed to determine which binding it references, and
> >  also extra information needed to implement 'datum->syntax'.  This in
> >  turn requires detailed knowledge of the lexical environment in which
> >  expansion is taking place, whereas 'macroexpand-1' provides no way for
> >  the user to provide this information.
> >
> >         Mark
> >
> > I have been reading this document about the scheme higienic macros
> > https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf
> >
> > I stopped reading it when I read that the implementation relies on a
> > previously bootstrapped version of another macro expansion
> > implementation.
>
> That's not an inherent limitation of the 'syntax-case' design.  It's
> merely an unfortunate attribute of the psyntax _implementation_ of
> 'syntax-case', apparently because they didn't care enough about
> bootstrapping issues to write psyntax without the benefit of macros.
>
> 'syntax-case' could certainly be implemented without using a
> pre-existing macro expander.
>
> > But Racket has some facilities to step and debug macros, as you can
> > see here https://docs.racket-lang.org/macro-debugger/index.html
> >
> > Aren' t Racket macros higienyc ?
>
> Yes, of course, and we could certainly implement similar macro stepping
> facilities in Guile.  But that's not what you asked about in your
> previous message.  You asked about 'macroexpand-1', and my answer was
> specifically about that.  I don't see any procedure similar to
> 'macroexpand-1' in the document you referenced above.
>

My bad

I assumed that macroexpand-1 was the building block for Racket macro
stepping and inspecting tools

I' m interested in macro stepping and inspecting facilities, not in
macroexpand-1 per se


> > In this question I've been promptly suggested a quick solution to
> > perform a single macro expansion step
> >
> > https://stackoverflow.com/questions/50073207/macro-
> expansion-in-guile-scheme/50515880#50515880
>
> For posterity, here's the quick solution suggested in the link above:
>
>   (define-syntax (expand1 stx)
>     (syntax-case stx ()
>       [(_expand1 form)
>        (syntax-case #'form ()
>          [(id . more)
>           (identifier? #'id)
>           (let ([transformer (syntax-local-value #'id)])
>             (with-syntax ([expansion (transformer #'form)])
>               #''expansion))]
>          [_
>           #''form])]))
>
> This is just a toy, and not very useful in practice.
> Here's the equivalent formulation for Guile:
>
>   (use-modules (system syntax)
>                (srfi srfi-11))
>
>   (define (syntax-local-value id)
>     (let-values (((type value) (syntax-local-binding id)))
>       value))
>
>   (define-syntax expand1
>     (lambda (stx)
>       (syntax-case stx ()
>         [(_expand1 form)
>          (syntax-case #'form ()
>            [(id . more)
>             (identifier? #'id)
>             (let ([transformer (syntax-local-value #'id)])
>               (with-syntax ([expansion (transformer #'form)])
>                 #''expansion))]
>            [_
>             #''form])])))
>
> (I usually prefer to avoid using square brackets in this way, but for
> sake of comparison, I used them in the definition of 'expand1' above.)
>
> Anyway, it works the same way as in Racket for this simple example:
>
>   scheme@(guile-user)> (expand1 (or 1 2 3))
>   $2 = (let ((t 1)) (if t t (or 2 3)))
>
>
This is surprising to me

When I saw that example made in Racket for the first time I instantly
identified "syntax-local-value" as problematic

Will Guile have anything equivalent ? I asked myself

Now you show me the "(system syntax)" namespace (or module)

I didn't  suspect it existed

Does the manual mention it anywhere ? I didn' t see it

Or maybe does it belong to any scheme standard ?

Do any more (system ....) namespaces exist ?

How would I know ?


> So, what's the problem?  The first problem is that when quoting the
> resulting expansion, the binding information associated with identifiers
> in the syntax objects are lost, so hygiene is lost.  For example:
>
>   scheme@(guile-user)> (expand1 (or 1 2 t))
>   $3 = (let ((t 1)) (if t t (or 2 t)))
>
> Moving on, let's use this to try to investigate how 'define-record-type'
> works from SRFI-9 in Guile:
>
>   scheme@(guile-user)> ,use (srfi srfi-9)
>   scheme@(guile-user)> (expand1 (define-record-type <box>
>                                   (box value)
>                                   box?
>                                   (value unbox set-box!)))
>


>   $4 = (%define-record-type #f (define-record-type <box> (box value) box?
> (value unbox set-box!)) <box> (box value) box? (value unbox set-box!))
>


>   scheme@(guile-user)> (expand1 (%define-record-type #f
> (define-record-type <box> (box value) box? (value unbox set-box!)) <box>
> (box value) box? (value unbox set-box!)))
>   While compiling expression:
>   Wrong type to apply: (%define-record-type guile-user)
>   scheme@(guile-user)>
>
> So what went wrong here?  The problem is that '%define-record-type' is a
> private macro, used internally within (srfi srfi-9), and therefore not
> bound in the (guile-user) module where I'm working.  If we had been
> working with syntax objects, each identifier within the expression would
> have been annotated with the specific binding that it refers to, but as
> I noted above, that information has been stripped.
>
> The awkward error message is because this toy implementation doesn't
> check if the identifier is a macro or not.
>
> One way we could try to improve this is to write 'expandN', which
> performs N macro expansion steps, keeping them as syntax objects during
> the intermediate steps:
>
>   (use-modules (system syntax)
>                (srfi srfi-11))
>
>   (define (syntax-local-type id)
>     (let-values (((type value) (syntax-local-binding id)))
>       type))
>
>   (define (syntax-local-value id)
>     (let-values (((type value) (syntax-local-binding id)))
>       value))
>
>   (define-syntax expandN
>     (lambda (stx)
>       (syntax-case stx ()
>         ((_expandN n form)
>          (let ((n (syntax->datum #'n)))
>            (and (number? n) (integer? n)))
>          (let ((n (syntax->datum #'n)))
>            (if (positive? n)
>                (syntax-case #'form ()
>                  ((id . _)
>                   (and (identifier? #'id)
>                        (eq? 'macro (syntax-local-type #'id)))
>                   (let ((transformer (syntax-local-value #'id)))
>                     (with-syntax ((expansion (transformer #'form))
>                                   (n-1 (datum->syntax #'id (- n 1))))
>                       #'(expandN n-1 expansion))))
>                  (_
>                   #''form))
>                #''form))))))
>
> Unfortunately, this is not quite right, because it fails to add "marks"
> to the identifiers introduced by the macro transformers, and thus is not
> fully hygienic, and variable capture may occur.  However, it is better
> than what we had before, and good enough to step further into
> 'define-record-type':
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,pp (expandN 0 (define-record-type <box>
>                                       (box value)
>                                       box?
>                                       (value unbox set-box!)))
> $2 = (define-record-type
>   <box>
>   (box value)
>   box?
>   (value unbox set-box!))
> scheme@(guile-user)> ,pp (expandN 1 (define-record-type <box>
>                                       (box value)
>                                       box?
>                                       (value unbox set-box!)))
> $3 = (%define-record-type
>   #f
>   (define-record-type
>     <box>
>     (box value)
>     box?
>     (value unbox set-box!))
>   <box>
>   (box value)
>   box?
>   (value unbox set-box!))
> scheme@(guile-user)> ,pp (expandN 2 (define-record-type <box>
>                                       (box value)
>                                       box?
>                                       (value unbox set-box!)))
> $4 = (begin
>   (define-inlinable
>     (box value)
>     (let ((s (allocate-struct <box> 1)))
>       (struct-set! s 0 value)
>       s))
>   (define <box>
>     (let ((rtd (make-struct/no-tail
>                  record-type-vtable
>                  'pw
>                  default-record-printer
>                  '<box>
>                  '(value))))
>       (set-struct-vtable-name! rtd '<box>)
>       (struct-set! rtd (+ 2 vtable-offset-user) box)
>       rtd))
>   (define-inlinable
>     (box? obj)
>     (and (struct? obj)
>          (eq? (struct-vtable obj) <box>)))
>   (define-tagged-inlinable
>     ((%%type <box>)
>      (%%index 0)
>      (%%copier %%<box>-set-fields))
>     (unbox s)
>     (if (eq? (struct-vtable s) <box>)
>       (struct-ref s 0)
>       (throw-bad-struct s 'unbox)))
>   (define-syntax-rule
>     (%%<box>-set-fields check? s (getter expr) ...)
>     (%%set-fields
>       <box>
>       (unbox)
>       check?
>       s
>       (getter expr)
>       ...))
>   (define-inlinable
>     (set-box! s val)
>     (if (eq? (struct-vtable s) <box>)
>       (struct-set! s 0 val)
>       (throw-bad-struct s 'set-box!))))
> scheme@(guile-user)>
> --8<---------------cut here---------------end--------------->8---
>
> Unfortunately this is as far as we can go with 'expandN', because it
> only expands macros at the top-level of the expression.  In this case,
> the top-level expression is a 'begin' form, which is a core form.  At
> this point, a real macro expander descends into the core form and
> expands subexpressions, but in order to do this properly, it needs to
> understand the meanings of the core forms that it's descending into.
>
> For example, when descending into a 'let' form, it needs to take note of
> the variables that are bound by the 'let'.  For example:
>
>   scheme@(guile-user)> (expandN 0 (or 1 2 3))
>   $2 = (or 1 2 3)
>   scheme@(guile-user)> (expandN 1 (or 1 2 3))
>   $3 = (let ((t 1)) (if t t (or 2 3)))
>   scheme@(guile-user)> (expandN 2 (or 1 2 3))
>   $4 = (let ((t 1)) (if t t (or 2 3)))
>
> The last two outputs are the same because I made 'expandN' just smart
> enough to notice that 'let' is not a macro, in which case it stops
> gracefully without triggering an exception.
>
> Hopefully this illustrates why the old 'macroexpand-1' procedure from
> Guile 1.x, which works on plain S-expressions without extra binding
> information, and which only expands macros at the top level of the
> expression, cannot be usefully implemented on a modern hygienic macro
> expander.
>
> However, what certainly *could* be done is some kind of interactive tool
> to incrementally step macro expansions, while keeping track of the
> syntax objects behind the scenes.  To be useful in realistic cases, it
> would need to understand most if not all of the core forms in Guile.
> Those core forms are the ones defined using 'global-extend' in
> psyntax.scm.
>
>      Regards,
>        Mark
>

I had read about those bindings and marks in the paper but I had no clear
idea about what they were

Now I have a way better idea

Also, now I understand what would be required in order to implement some
macro stepping and inspecting facilities similar to those available in
Racket

I don' t know if I will actually try to implement anything, but I will
certainly use these notions to my advantage in working with Guile/Guix

So thanks


reply via email to

[Prev in Thread] Current Thread [Next in Thread]