Re: RIP: c++: merge symbol_type and token

[Frank Heckenbach]

Akim Demaille wrote:

> As I mentioned earlier, I would be very happy to merge parser::token
> and parser::symbol_type together.  It works well, and I was very
> happy about that.
> 
> But it's not so simple.  The problem is that then parser::token looks
> like this:
>
> [...]
>
> and then... I can have name clashes if the user defines the name
> "token", but also all the names inherited from basic_symbol.
> 
> That's annoying.
> 
> So I don't think I can follow this track.

I see. So struct token only exists to put the yytokentype values in
their own namespace to avoid those clashes. An enum class would do
the same trick (and provide for stricter type-checking), but of
course requires C++11. But perhaps then it's not too bad now to
leave it as it, with the option to easily convert it to an enum
class later.

> >> - In the signature of make_symbol, I've had to use int for the
> >>  token type, instead of the enum token_type, and then
> >>  to convert the int into token_type.  I don't like that, but I
> >>  probably don't have much of a choice.  (A template would be
> >>  overkill IMHO).
> > 
> > Why, is it because of char tokens (like '+' in your example)?
> 
> Yes, exactly.  I don't like that we accept ASCII, but we have
> to.

I see your point, but as a user of Bison I like not having to
declare tokens for syntactic fluff like ',' and '.'.

Would it help if Bison (internally) generates enum values for the
chars used, such as "TOK_46 = '.'" (with some non-conflicting
prefix), so token_type would cover all valid values and could be
used as the parameter type? (This might also be required for the
stricter type-checking of an enum class.)

Regards,
Frank