Re: Bison C++ mid-rule value lost with variants

[Frank Heckenbach]

Akim Demaille wrote:

> > IMHO, "active types" aren't really the problem (as std::variant or
> > an equivalent implementation can handle them), but indeed it's too
> > low-level and error-prone, though that would apply to all skeletons
> > (but of course, dropping it from C is impossible because of
> > backward-compatibility).
> 
> I agree.  There's just one place I don't know too well how to
> address, that for $-1, $-2, etc.  But again that's because as
> of today Bison is too limited to recover the type.

I wasn't even aware of $-1, just read about it in the manual. How is
this reconcilable with proper LR parsing? Is it really needed for
some grammars or could it be dropped too (at least in C++)?

> You seem to have a bottom-up approach: if I use this as a
> semantic value, what can I do with it.  I look at this the
> other way round.  Everything I know about LR shows that this
> is not needed, so I'll avoid it, _unless_, I'm given a counter
> example.

Not really bottom-up. I saw that Bison supported $<>, but buggy, and
tried to figure out if I can fix the bugs which std::variant allowed
me to do. Apparently you didn't see it as supported, but buggy, but
rather as unsupported.

> I'm teaching (well, I used to teach now) LR parsing, and I don't
> want my students (well, former) to have the impression I'm fooling
> them by using something different.

I thought the user actions were rather separate from parsing theory,
but I'll take your word for it. And we agree about dropping $<>
anyway. :)

> >>> When I did the coding, std::variant actually simplified things for
> >>> me (e.g., I could avoid adding move support in Bison's variant
> >>> implementation), so if I were you, I'd probably use it even if I
> >>> dopped $<>, but if you want to avoid its small runtime overhead,
> >>> that seems possible.
> >> 
> >> I don't think we can afford to simply drop all C++s pre 17.
> > 
> > As I wrote before, there are std::variant implementations for C++14
> > (which I'm actually using mostly so far) and I think also C++11
> > (I haven't tested this).
> 
> Yes, I know.  But then, I expect license nightmares to ship
> them, and also by pre 17, I also mean 98/03.

Both mparks's and Boost's implementation are released under the
Boost Software License, described on gnu.org as "[...] a lax,
permissive non-copyleft free software license, compatible with the
GNU GPL", so I see no big problems here, for both free and non-free
software developers.

As for pre-11, personally I'm not very interested (I shied away from
C++ for a long time, and IMHO it's only become useable with C++11),
but I see your point of view. So you'll need to make (internal)
moving dependent on the compiler version then. Maybe you can define
a template that does std::move for C++11 and NOP for older
compilers, to avoid sprinkling the code with ifdefs.

> > But I've come to rely on some of the features I implemented there.
> > AFAICS, you haven't commented on them yet, so I don't know how you
> > think about them. If you really object to them (or equivalent
> > features), I might prefer to keep using my skeletons.
> 
> I didn't answer because I have already too many threads open,
> and I prefer to stay kinda focus on the issues I plan to address
> in the short term, keeping the rest for later.  I wish I could
> do more :/  And I'm sorry to keep you waiting.

No problem. As I said, for now I won't do much about it anyway. But
please understand that I'd like to have clarity about those issues
before I'll do substantial work.

> > - pre-action for empty rules with a user action: $$ is initialized
> >  to the default value of the correct type. With completely static
> >  variants, this might even be natural (or may need to call the
> >  default constructor in a switch), and should be officially
> >  documented.
> > 
> >  (In contrast, for non-empty rules with a user action, I
> >  pre-initialize $$ to an invalid variant, so if the user action
> >  forgets to set $$, a bad_variant_access will happen on access
> >  which may catch some errors in user actions. This won't be
> >  possible with static variants, but I can live without that.)
> 
> Never thought about this before.
> 
> Do you happen to have test cases?

No isolated ones (my parsers contain a number of them). Here's a
slightly simplified extract to build a formal parameter list:

  using TFormalParameters = vector <pair <TType, string>>;

  %type <TFormalParameters> fpar_list

  fpar_list:               type_name identifier {  $$      .emplace_back ($1, 
$2); }
           | fpar_list ',' type_name identifier { ($$ = $1).emplace_back ($3, 
$4); };

As a side note, this also relies on automatic moving for all $n in
the example -- let's assume TType is a move-only type (it isn't
actually, but I have other rules with move-only types and don't want
to have too many different examples here), and also for performance
(string, vector).

The first action assumes that $$ is default-initialized. Now I
realize I could also write it like this without that assumption:

    { $$ = { { $1, $2 } }; }

Though I think it looks a bit uglier and loses the similarity to the
second action, but maybe these are not important enough reasons to
justify that feature.

So maybe the pre-initialization isn't actually so important.

But I do need the default action with no user action, basically for
all rules of the form "a: b | ..." where a and b have semantic
values.

Also I think it should be clearly documented in the end what $$ can
and cannot be relied upon to contain in any case.

> > - Not sure if there will be issues WRT "%printer" and "%destructor".
> >  Since they don't work with dynamic variants, I've forbidden them
> >  in my skeletons.
> 
> I don't see why they wouldn't work.  You can trust Bison
> to provide you with the expected type, so you could
> std::get from your variant. Yet I guess a visitor is nicer anyway :)

I meant they wouldn't work with full dynamic type support (i.e. $<>)
as I implemented it (because then Bison's notion of type may not be
accurate, and the correct printer/destructor can only be chosen at
runtime, e.g. visit). If we drop $<>, it would work with
std::variant, too.

> >  I don't (want to have to) ever use "%destructor" (C++ destructors
> >  should do the work in all cases), and for printing I use a single
> >  overloaded function (yy_print_value). This works with dynamic
> >  variants using std::visit, but I think it should also work with
> >  "%printer <*>", right?
> 
> I'm not sure I understand what you mean here.

I currently use the visitor with yy_print_value. If we settle on
"%printer" (i.e. keep it as is in Bison), after dropping $<>, I'll
only have to add something like

  %printer { yy_print_value (yyoutput, $$); } <*>

in my grammars, and it will use my existing (templated and
overloaded) yy_print_value functions for all semantic types, right?

For an idea how they look like, see the "! YY_PRINT_VALUE_SIMPLE"
branch in my calc-c++17-parser.yy example.

Regards,
Frank