Re: Dynamic token kinds

[Akim Demaille]

Hi Frank!

> Le 25 nov. 2018 à 18:17, Frank Heckenbach <address@hidden> a écrit :
> 
> [...] Then my yylex function does this (slightly simplified from my actual
> code; GetToken, TokenText and Loc are functions of my RE-based
> lexer):
> 
>  switch (GetToken ())
>  {
>    // ...
>    case lt_Identifier:
>      if (auto i = Find (PredefinedStrings, TokenText ()))
>        return TParser::symbol_type (TParser::token_type (i->second), Loc ());
>      return TParser::make_identifier (TokenText (), Loc ());
>  }
> 
> The only interesting thing here (as far as Bison is concerned) is
> that I determine the token kind dynamically, so I can't use the
> make_FOO functions, but call the TParser::symbol_type constructor
> manually. (In my experimental changes, I had added a make_token
> function for this purpose, but I hadn't mentioned it in my
> announcements, so obviously it's not present in Bison currently;
> I'm not sure if it was very useful anyway, since it was just a
> wrapper around this constructor).
> 
> So I need to make sure that this is officially supported, i.e. this
> constructor and enum yytokentype will remain in the public interface
> in the generated header, or if other alternatives are recommended
> instead.

yytokentype is already part of the documentation, there's no plan
to change it, I think you are safe.

Wrt to the symbol constructor, you are right to be worried: I don't
consider it (so far?) to be part of the public API.  I do understand
something like it is needed, but I don't like that it looks safe
to use.

Would you be ok with parser::unsafe_make_symbol, or something like
this?