Re: Getting bison to ignore tokens... sometimes!

[Adam Smalin]

I am a bit confused. If you see an EOS then your either done the expression
or have made it legal to have it in the middle of an expression (i do it all
the time such as IF RValue mEOS CodeBlock which allows the {} after the if
to be many lines ahead)

Lets take your example.


def myfunc {
   x = x + 1
   y = x + 2
}

I would have it something like

DEF Var mEOS FuncBody

FuncBody: '{' mEOS FuncBodyLoop '}'
FuncBodyLoop: FuncBody2 | FuncBodyLoop FuncBody2 //mine is more complex tho
FuncBody2: FuncBodyE aEOS | FuncBodyS mEOS
FuncBodyE: ... | RValue
RValue: IntLiteral | RExpr | Var
RExpr: Rvalue '+' Rvalue

essentially you will have

FuncBodyLoop(
FuncBody2(FuncBodyE(RValue(x = RExpr(x + 1) )) aEOS )
FuncBody2(FuncBodyE(RValue(y = RExpr(x + 2) )) aEOS )
)

Maybe i overlooked something but i doubt it ;P

note the mEOS after the '{'. The mEOS before the body should handle any
newlines after { but before the first statement. Each expression/statement
will match new lines at the end by use of aEOS (which means one or more
since you MUST have one at least to end the statement) or mEOS (which means
it already ended with something like '}' such as a while loop but have it
for trailing new lines)

btw have you read flex & bison? http://oreilly.com/catalog/9780596155988  It
made it much easier for me to understand how to write something.


On Wed, Jun 1, 2011 at 5:02 PM, Paulo J. Matos <address@hidden> wrote:

> Hello Adam,
>
> Thanks for the input. Well, it really depends on your language.
> In your case, what would happen if I write a StructBodyE over
> different lines? The lexer would report EOSs in the middle of the
> expression that the parser would fail to match. That's why that
> technique doesn't work for me.
>
> Cheers,
>
> Paulo Matos
>
> On Wed, Jun 1, 2011 at 7:25 PM, Adam Smalin <address@hidden>
> wrote:
> > I am a bit confused. Why cant you make \n an EOS?
> > I use this code for my EOSs
> >
> > EOS:   '\n' | ';'
> > aEOS:   EOS | aEOS EOS
> > mEOS:       | mEOS EOS
> >
> > Essentially what i do is have two parts of everything. One ends with E
> > meaning expression the other is S (statement).  Then i write this
> >
> > StructBody:
> >       StructBodyE aEOS
> >     | StructBodyS mEOS
> >
> > I also have A which i only use with if statements since else is only
> valid
> > if the previous line is an if/elseif. However you can cheat and use the
> > lexer to ignore newlines before an else statement.
> >
> >
> >
> > On Tue, May 31, 2011 at 10:37 AM, Paulo J. Matos <address@hidden>
> wrote:
> >>
> >> Hi,
> >>
> >> I am pretty sure bison cannot simply ignore tokens. However, I don't
> know
> >> what's the usual flex/bison pattern to solve this issue.
> >>
> >> I am implementing a parser for a language that terminates statements
> with
> >> a newline, however, the newline doesn't always terminate statements...
> >> only after a statement.
> >>
> >> -----
> >> x = x + 1
> >> y = x + 2
> >>
> >> -----
> >>
> >> This should parse ok, since each statement is terminated by a new line.
> >> If I tell lexer to lex '\n' as a EOS (end of statement) then something
> >> like this won't parse:
> >>
> >> -----
> >> def myfunc {
> >>    x = x + 1
> >>    y = x + 2
> >> }
> >> -----
> >>
> >>
> >> The reason for this is that it gets the tokens '{', '\n', 'x'
> (whitespace
> >> ignored) and says : "hey, I am not expecting a EOS there". Now, I could
> >> fill my parser with EOS between every two tokens, however, that's crazy
> >> and there must be a better way.
> >>
> >> Any suggestions?
> >>
> >> Cheers,
> >> --
> >> PMatos
> >>
> >>
> >> _______________________________________________
> >> address@hidden https://lists.gnu.org/mailman/listinfo/help-bison
> >
> >
>
>
>
> --
> PMatos
>