Re: Featuritis: Token actions

[Frank Heckenbach]

Hans Aberg wrote:

> It might be useful with actions associated directly with tokens, that can
> be used to let the lexer communicate with the parser. When the parser
> encounters such a token, its action is executed immediately, whereafter the
> parse looks for a new token. This way one can insert actions based on
> tokens pretty much anywhere in the grammar. For example:
>   %token debug_off { /* set debug variable off. */ }
>   %token debug_on { /* set debug variable on. */ }
>   %token lexer_error { yyerror(...); YYERROR; }
> 
> The implementation should be fairly straightforward; the parser output file
> might contain:
> 
>   // Backup.
> yybackup:
> 
>   // Try to make a decision without lookahead:
>   n_ = pact_[state_];
>   if (n_ == pact_ninf_)
>     goto yydefault;
> 
>  // Read a lookahead token.
>   if (lookahead_ == empty_) {
> yynewtoken:
>     YYCDEBUG << "Reading a token: ";
>     lex_();
>   }
> 
>   // Convert token to internal form.
>   ...
> 
>   switch (lookahead_) {
>     case debug_off:
>       { /* set debug variable off. */ }
>       goto yynewtoken;
>     case debug_on:
>       { /* set debug variable on. */ }
>       goto yynewtoken;
>     case lexer_error:
>       { yyerror(...); YYERROR; }
>       goto yynewtoken;
>     default:
>       ;
>   }
> 
> I think this might be used to solve some of the things requested by others.

Maybe you're referring to my request re "Finding out when a token is
consumed". However AFAIUI, you do the token action immediately after
getting the token from the lexer. So it doesn't seem much different
from doing the action in the lexer.

> As for myself, I wanted to implement the parse debug on/off feature, and
> therefore started to think along these lines.

If you want it to work at the point where the "grammatical location"
actually reaches the debug on/off token (and not already when it's
read as a look-ahead token), you basically have the same problem I
had with my directives (which I solve by abusing the location
actions as you might remember).

(E.g., with `1 + 2 debug-on + 3' in a usual arithmetic grammar, your
suggestion would turn on debugging before the first addition is
reduced.)

Frank

-- 
Frank Heckenbach, address@hidden
http://fjf.gnu.de/
GnuPG and PGP keys: http://fjf.gnu.de/plan (7977168E)