Re: Reserved Words

[Akim Demaille]

 Mike> Cheers for that - I actually managed to do it a different
 Mike> way...  Using bison 1.875 - there is a -r all option - I used
 Mike> this to create a y.output which (helpfully) descends states
 Mike> (unlike some older versions..) and prints all the possible
 Mike> tokens at each state..

Actually, it doesn't...  It reports *needed* lookaheads.  Consider the
following: 

| Grammar
| 
|     0 $accept: e $end
| 
|     1 e: a 'b'
|     2  | 'a' 'a'
| 
|     3 a: 'a'
| 
| 
| Terminals, with rules where they appear
| 
| $end (0) 0
| 'a' (97) 2 3
| 'b' (98) 1
| error (256)
| 
| 
| Nonterminals, with rules where they appear
| 
| $accept (5)
|     on left: 0
| e (6)
|     on left: 1 2, on right: 0
| a (7)
|     on left: 3, on right: 1
| 
| 
| state 0
| 
|     0 $accept: . e $end
|     1 e: . a 'b'
|     2  | . 'a' 'a'
|     3 a: . 'a'

It should have written [$end] for 1 and 2, and ['b'] for 3.

|     'a'  shift, and go to state 1
| 
|     e  go to state 2
|     a  go to state 3
| 
| 
| state 1
| 
|     2 e: 'a' . 'a'
|     3 a: 'a' .  ['b']

Missing [$end] for 2.  But it just needs to know for ['b'] to make its
mind here, so it did compute the lookahead and reports it.

|     'a'  shift, and go to state 4
| 
|     $default  reduce using rule 3 (a)
| 
| 
| state 2
| 
|     0 $accept: e . $end
| 
|     $end  shift, and go to state 5
| 
| 
| state 3
| 
|     1 e: a . 'b'

Missing [$end] etc.

|     'b'  shift, and go to state 6
| 
| 
| state 4
| 
|     2 e: 'a' 'a' .
| 
|     $default  reduce using rule 2 (e)
| 
| 
| state 5
| 
|     0 $accept: e $end .
| 
|     $default  accept
| 
| 
| state 6
| 
|     1 e: a 'b' .
| 
|     $default  reduce using rule 1 (e)
| 

 Mike> Using a %pure_parser - I pass that state into my yylex - and
 Mike> then check to see if the token is allowed in that state...  

Amazing!  This is probably very fragile, no?  I mean, does it depend
on internals?

 Mike> If it is - then I let it through, if its not - the I convert it
 Mike> to the identifier token (after checking it looks like an
 Mike> identifier a-z etc)...

 Mike> Seems to work like a charm... (Over 500,000 lines of our
 Mike> language source code correctly parsed as a test).

Congrats!