Re: simple scheme function #{ $note #}

[David Kastrup]

Aaron Hill <address@hidden> writes:

> On 2018-05-11 23:01, David Kastrup wrote:
>>> The reason why #{ $p #} does not work as the body of a music function
>>> is that it will only evaluate to a pitch not a note, and that pitch by
>>> itself is not enough to create a music expression.
>>
>> Wrong again.  There is no such thing as a "pitch by itself" in Scheme.
>> A pitch is not a deficient music expression, rather it is not a music
>> expression at all.  It's like saying a character by itself is not
>> enough
>> to create a string, as if two characters by itself would create a
>> string.  Strings are not the same as characters of any count.
>>
>> Strings are a separate data structure from characters.  You can place
>> characters in other data structures.  You can certainly also form
>> one-character strings.
>>
>> (make-music 'NoteEvent 'pitch #{ c' #})
>>
>> is a music expression a music function can return (it's deficient for a
>> number of unchanged uses in that it is lacking a duration but you can
>> add durations afterwards if you want to with different music
>> functions).
>>
>> Two pitches are not enough to create a music expression.  Ten pitches
>> aren't.  If you want to create a music expression, you need to create a
>> music expression, like using make-music.  Or use some #{ ... #}
>> construct recognized as producing music.
>>
>>> #{ $p 4 #} works because a pitch followed by a duration clearly
>>> defines a note, which is enough to form a music expression.
>>
>> #{ $p #} #{ 4 #} is a pitch followed by a duration and does not form a
>> music expression.  The music expression is created by specific forms of
>> the #{ ... #} syntax.
>
> Okay, then none of this makes any logical sense.

You are trying to deduce stuff that has been deliberately designed.
It's not bound by the rules of logic but rather by that of convenience
as well as coherence of design.  For better or worse, part of that means
that a hand-waving sense of intuition often hits right because things
are merely appearing deceptively simple.

> It is my understanding that #{ #} is an escaping mechanism.

Deception #1.  It isn't.  It invokes a different terminal rule than
LilyPond's standard parser and, while sharing a number of non-terminal
rules, is actually a parser of its own, with its own rules and
behavior.  Much of it coincides with the kind of expression you can
write to the right of an assignment.

> Its purpose is to facilitate mixing LilyPond syntax within a Scheme
> construct.  At the end of the day, everything that is input to the
> LilyPond program becomes one long Scheme expression.

Deception #2.  LilyPond processes isolated expressions using various
hooks and resulting in various behavior.  You can actually process
things and then throw the Scheme expression away, allowing LilyPond to
run indefinitely on the same input file without any "long Scheme
expression" getting accumulated.

> LilyPond syntax is essentially nothing more than a convenient
> abbreviation for equivalent Scheme.

See above.

> To that end, there is a parser involved in processing LilyPond syntax
> to generate the desired Scheme expressions.

Not everything in LilyPond is actually representable as a Scheme
expression.  For example, Midi is converted into pure C++ expressions
not even managed by the Scheme garbage collector.  While I lean towards
calling this a historical mistake that will at some point of time get
rectified, it illustrates that your notion of what LilyPond does is at
best handwavingly useful.  It has certainly been much of my work focus
to tie things together in manners where LilyPond does not look like
bleeding illogical internals all over the place.  Your misconceptions
show that I've been somewhat successful with that work.

But it's more useful for deducing how LilyPond should be working giving
a benign human-understandable design and history rather than deducing
how it actually does work.

> It sounded like the underlying issue with the original poster's error
> was that of a limitation during parsing.  To simply include "c'" by
> itself within #{ #} causes the parser to only generate enough Scheme
> for the specified NOTENAME_PITCH token which would be an ly:pitch.
> This looked to me to be similar to when you only include "c'" by
> itself in a LilyPond file and try to compile it.

No, that is just a safeguard to avoid unbraced notes getting compiled
into wagonloads of isolated scores.  The scope of that safeguard has
been waxing and waning a lot with various refactorings of the parser.
It's not strictly necessary at all but likely to avoid more damage than
it causes.

> You minimally need to enclose it in braces to give the parser some
> context.

Nope, it's just a deliberate syntax error.  It would be easy to just let
go of it but that's more likely to cause rather than remedy trouble.

> As such, that is why I offered a solution of #{ { $p } #}, since that
> successfully gets around the error.
>
> Of course, I tried something just now that I should have tried before.
> That is, what happens with a file containing only "c' d'"?  Oh, well,
> it fails to compile as well.  Crud.  So my conclusion that the parser
> in general is able to figure out what is meant by a sequence of bare
> pitches is wrong.  The parser really does need more context to make
> sense of things.

No, it doesn't.  It was just a design choice not to admit unadorned note
events as top level expressions.  You can use something like

\displayMusic c'

and, if I remember correctly, it will compile.  That's one of the things
that worked intermittently in history.

> However, that does not immediately explain why #{ c' d' #} works.  It
> would seem that it should fail for similar reasons.

Utterly different reasons.  #{ ... #} has its own rules in the grammar.

> There is clearly magic in #{ #} that is non-obvious here.  But why is
> its behavior inconsistent?  Surely if #{ c' d' #} is "smart" enough to
> interpret two bare pitches as a sequence of notes with assumed
> durations, why can it not do the same for a single pitch.

It can, but it would be comparatively useless if it did.

> Or, said another way, HOW does one actually make #{ $p #} work for a
> music function?

What duration is that supposed to have?

> Certainly, #{ { $p } #} and #{ <> $p #} both appear to work for
> producing music with the pitch as a note, and you provided a way to
> verbosely use make-music for the task.  Is there no other way to force
> #{ #} to interpret its contents as producing music?

Of what duration?  It seems more useful to let it produce pitches.

> At this point, I must confess that I tell a lie.  To say this makes no
> logical sense was hyperbolic.  Is this not, as I have tried and failed
> to relay, simply an issue of mismatched types?  #{ $p #} results in a
> pitch not music.  That is why it cannot be the sole body of a music
> function.

Correct.

> Music functions must return music, and #{ #} will never evaluate to
> music if provided a bare pitch as input.  You are left with an
> ly:pitch that needs extra work to become music.

Correct.  Because otherwise you'd be out of a way to produce a pitch
using LilyPond syntax.

> If so, that actually does make sense.  It would be madness if #{ #}
> always tried to form a music expression with whatever you gave it,
> since that would give you no way of easily expressing the myriad of
> other syntactic element types within Scheme.

You'd need different syntax then.  But the syntax is complex enough as
it is.  If one can make do with one escape into LilyPond syntax rather
than dozens, that's considerably nicer.

Note that in 2.14 or so (2.12?), #{ #} exclusively produced music, never
anything else.  There are actually rather few ambiguous cases now
resolved to non-music that could strictly speaking be music, so it's a
reasonable tradeoff.

> Your explicit make-music example for instance must rely on #{ c' #}
> evaluating only to a pitch for the purposes of defining the NoteEvent.
>
> Now, #{ c' d' #} could be interpreted in one of two ways.  Is this a
> list of two pitches, or is this a music expression formed by two notes
> without explicit durations?

Correct.  But since LilyPond does not input "lists of pitches"
elsewhere, the rules interpret it as sequential music.  Also because of
compatibility reasons: in 2.12 or so, #{...#} always produced sequential
music.

> The latter seems to be the more common use case, so understandably
> that is what happens.  And right there is the magic.  The parser
> normally would not accept two bare pitches as music; but within the #{
> #} construct it does consider that as sufficient to produce music.

Or inside of { }.  Or inside of << >>.

> This would run counter to your assertion that "[two] pitches are not
> enough to create a music expression."  Outside of the #{ #} construct,
> yes, that appears to be the case.  A LilyPond file with just "a b c"
> will not work, and I was wrong if I implied that in my post.  However,
> this discussion is about the #{ #} construct.  And within the #{ #}
> construct, the parser, it would seem, follows different rules allowing
> #{ a b c #} to produce music but not #{ a #}.

Correct.  To wit:

embedded_lilypond:
        /* empty */
        {
                // FIXME: @$ does not contain a useful source location
                // for empty rules, and the only token in the whole
                // production, EMBEDDED_LILY, is synthetic and also
                // contains no source location.
                $$ = MAKE_SYNTAX (void_music, @$);
        }
        | identifier_init_nonumber
        | embedded_lilypond_number
        | post_event
        {
                if (!unsmob<Music> ($1))
                        $$ = MY_MAKE_MUSIC ("PostEvents", @$)->unprotect ();
        }
        | multiplied_duration post_events %prec ':'
        {
                if (scm_is_pair ($2)) {
                        Music *n = MY_MAKE_MUSIC ("NoteEvent", @$);

                        parser->default_duration_ = *unsmob<Duration> ($1);
                        n->set_property ("duration", $1);
                        n->set_property ("articulations",
                                         scm_reverse_x ($2, SCM_EOL));
                        $$ = n->unprotect ();
                }
        }
        | music_embedded music_embedded music_list {
                SCM tail = SCM_EOL;
                if (unsmob<Music> ($1))
                        tail = scm_cons ($1, tail);
                if (unsmob<Music> ($2))
                        tail = scm_cons ($2, tail);
                $$ = reverse_music_list (parser, @$,
                                         scm_append_x (scm_list_2 ($3, tail)),
                                         true, true);
                if (scm_is_pair ($$)) // unpackaged list
                        if (scm_is_null (scm_cdr ($$)))
                                $$ = scm_car ($$); // single expression
                        else
                                $$ = MAKE_SYNTAX (sequential_music, @$, $$);
                else if (scm_is_null ($$))
                        $$ = MAKE_SYNTAX (void_music, @$);
                // else already packaged post-event
        }
        | error {
                parser->error_level_ = 1;
                $$ = SCM_UNSPECIFIED;
        }
        | INVALID embedded_lilypond {
                parser->error_level_ = 1;
                $$ = $2;
        }
        ;

> Something like #{ a #} #{ 2 #} is an entirely different scenario
> altogether.  <insert "Airplane" reference here>  As you have
> introduced a second #{ #} construct, it should be no surprise the
> parser will treat each construct individually.  As neither ly:pitch
> nor ly:duration is valid to return from a music function,
> define-music-function is unhappy.
>
> And this would disagree with your comment that "There is no such thing
> as a 'pitch by itself' in Scheme.".  There very much is such a thing
> as a pitch in Scheme.

Sure.  But not "by itself": a pitch is a pitch and can _never_ be
anything accompanied by anything.  "by itself" does not make sense, like
5 in Scheme is not an integer "by itself" but an integer, period.  It's
never anything else.

> The type ly:pitch alone is evidence of that, otherwise the premise of
> the original poster's music function is flawed. Perhaps what you
> intended to stress is that a music expression within Scheme cannot
> consist of a pitch (or duration) by itself.

No, I intended to stress that "a pitch by itself" is not a sensible
Scheme type since it's either a pitch or it isn't.  It cannot be an
accompanied pitch.

> Let's pretend for a moment that #{ a #} and #{ 2 #} could evaluate to
> music.

Either could: a as a single note event, 2 as an unpitched note of
duration 2 (2.19 introduced rhythms making use of that).  It's just that
different meanings have been chosen instead since they were needed more
often.

> They would each end up defining a NoteEvent.  The ly:pitch and
> ly:duration values would be properties of their respective events and
> therefore would not be standing alone within the expression.
>
> But this all reinforces the crux of the confusion: that #{ #}
> sometimes evaluates to music and sometimes does not.  While this
> behavior can be helpful, surely it must be the entire reason why the
> original poster's function does not work as intended.

Sure.  By design.  #{...#} does not do what it does out of technical
necessity but out of best utility.  Sometimes that may interfere with
some simple uses of it, like it does here.

-- 
David Kastrup