Re: symbol_type::token () removed?

[Frank Heckenbach]

Akim Demaille wrote:

> > and I think my use case is legitimate.
> > 
> > [...]
> > So I have things like:
> > 
> >     Lexer lex("12345");
> >     symbol_type t = lex.nextToken();
> >     assert(t.token() == token::INTLIT);
> >     assert(t.value.as<int>() == 12345);
> 
> which gives me the impression that the real use case would be something like
> 
> assert(t == token::make_INTLIT(12345))

That was my first thought as well when I read the original message ...

> WDYT?  Hum.  That would require operator==, which would generate a lot
> of code.  And it would require the value types to support ==, which so
> far we do not require.

... and that was my second thought. ;)

It might be possible to SFINAE check if all value types support
"==", as Wolfgang suggests. But what if only some do, and one only
cares about those? E.g., suppose all of your (Wolfgang) types
currently do and you rewrite your tests like above. But later you
need to add another type somewhere in the parser that has no obvious
"==" operator. You'd have to add a fake one (never to be called,
possibly just calling abort() -- declaring without defining won't
do), just to keep your test using int working. This might turn
unrelated compile-time problems into runtime aborts, also not nice.

With some more SFINAE, one could check each type individually and
always return false if it has no "==" operator. But that also seems
error-prone.

So while I originally thought it was a good idea to make the value
type comparable, the more I think about it, the more doubts I have.

On the other hand, I wouldn't mind keeping it similar to
std::variant generally. std::variant does declare "==", but makes it
UB if "==" does not yield a valid expression returning a type
convertible to bool, for any type
(https://en.cppreference.com/w/cpp/utility/variant/operator_cmp).
In practice, it seems to give a compile-time error if any type does
not have "==", but it might be runtime UB if it exists and returns
non-bool.

However, std::variant is a template, so if the "==" operator is not
used, it won't get instantiated, so there's no problem, whereas
Bison's value type is not a template, so implementing a "==" that
requires all semantic types to have one would cause compilation of
the parser to fail even if "==" isn't actually used, so as Wolfgang
wrote:

> It could be another %define option, and then static_assert the presence
> of operator== for the value types.

It pretty much would have to be another option then to avoid
breaking other parsers (or turning it into a template, just for this
purpose, and maybe similar ones in the future). Whereas
static_assert doesn't seem necessary -- compilation would just fail
anyway if some type doesn't have "==".

Another question is if "==" should include equality of locations (if
enabled) or ignore them. Either choice could be confusing to someone
who assumes the other (or doesn't consider locations), so at least
it must be documented clearly.

Regards,
Frank