Re: Reserved Words

[Mike Aubury]

I've replaced 'a' with A, 'b' with B...

All my tokens are tokenised (no '.'), and uppercased in the grammer....
In my checks - if I get a -1 - that is always passed through unchanged ($end 
can't really be an identifier!)


>From my script - we get  :


state 0 accepting :
        A

state 1 accepting :
        A or B

state 2,4,5 accepting nothing at all - I don't think yylex would be called in 
these states...

state 3 accepting
        B

(Which I think is right!)


It may be pure chance that it works - but I'm reasonably happy that it 
does....


On Thursday 20 February 2003 4:38 pm, Akim Demaille wrote:
>  Mike> Cheers for that - I actually managed to do it a different
>  Mike> way...  Using bison 1.875 - there is a -r all option - I used
>  Mike> this to create a y.output which (helpfully) descends states
>  Mike> (unlike some older versions..) and prints all the possible
>  Mike> tokens at each state..
>
> Actually, it doesn't...  It reports *needed* lookaheads.  Consider the
>
> following:
> | Grammar
> |
> |     0 $accept: e $end
> |
> |     1 e: a 'b'
> |     2  | 'a' 'a'
> |
> |     3 a: 'a'
> |
> |
> | Terminals, with rules where they appear
> |
> | $end (0) 0
> | 'a' (97) 2 3
> | 'b' (98) 1
> | error (256)
> |
> |
> | Nonterminals, with rules where they appear
> |
> | $accept (5)
> |     on left: 0
> | e (6)
> |     on left: 1 2, on right: 0
> | a (7)
> |     on left: 3, on right: 1
> |
> |
> | state 0
> |
> |     0 $accept: . e $end
> |     1 e: . a 'b'
> |     2  | . 'a' 'a'
> |     3 a: . 'a'
>
> It should have written [$end] for 1 and 2, and ['b'] for 3.
>
> |     'a'  shift, and go to state 1
> |
> |     e  go to state 2
> |     a  go to state 3
> |
> |
> | state 1
> |
> |     2 e: 'a' . 'a'
> |     3 a: 'a' .  ['b']
>
> Missing [$end] for 2.  But it just needs to know for ['b'] to make its
> mind here, so it did compute the lookahead and reports it.
>
> |     'a'  shift, and go to state 4
> |
> |     $default  reduce using rule 3 (a)
> |
> |
> | state 2
> |
> |     0 $accept: e . $end
> |
> |     $end  shift, and go to state 5
> |
> |
> | state 3
> |
> |     1 e: a . 'b'
>
> Missing [$end] etc.
>
> |     'b'  shift, and go to state 6
> |
> |
> | state 4
> |
> |     2 e: 'a' 'a' .
> |
> |     $default  reduce using rule 2 (e)
> |
> |
> | state 5
> |
> |     0 $accept: e $end .
> |
> |     $default  accept
> |
> |
> | state 6
> |
> |     1 e: a 'b' .
> |
> |     $default  reduce using rule 1 (e)
>
>  Mike> Using a %pure_parser - I pass that state into my yylex - and
>  Mike> then check to see if the token is allowed in that state...
>
> Amazing!  This is probably very fragile, no?  I mean, does it depend
> on internals?
>
>  Mike> If it is - then I let it through, if its not - the I convert it
>  Mike> to the identifier token (after checking it looks like an
>  Mike> identifier a-z etc)...
>
>  Mike> Seems to work like a charm... (Over 500,000 lines of our
>  Mike> language source code correctly parsed as a test).
>
> Congrats!