Featuritis: Token actions

[Hans Aberg]

It might be useful with actions associated directly with tokens, that can
be used to let the lexer communicate with the parser. When the parser
encounters such a token, its action is executed immediately, whereafter the
parse looks for a new token. This way one can insert actions based on
tokens pretty much anywhere in the grammar. For example:
  %token debug_off { /* set debug variable off. */ }
  %token debug_on { /* set debug variable on. */ }
  %token lexer_error { yyerror(...); YYERROR; }

The implementation should be fairly straightforward; the parser output file
might contain:

  // Backup.
yybackup:

  // Try to make a decision without lookahead:
  n_ = pact_[state_];
  if (n_ == pact_ninf_)
    goto yydefault;

 // Read a lookahead token.
  if (lookahead_ == empty_) {
yynewtoken:
    YYCDEBUG << "Reading a token: ";
    lex_();
  }

  // Convert token to internal form.
  ...

  switch (lookahead_) {
    case debug_off:
      { /* set debug variable off. */ }
      goto yynewtoken;
    case debug_on:
      { /* set debug variable on. */ }
      goto yynewtoken;
    case lexer_error:
      { yyerror(...); YYERROR; }
      goto yynewtoken;
    default:
      ;
  }

I think this might be used to solve some of the things requested by others.
As for myself, I wanted to implement the parse debug on/off feature, and
therefore started to think along these lines.

  Hans Aberg