[Fwd: Re: How would you attack this?]

[Steve Murphy]

Hello--

Will Estes, working on the latest versions of flex, got the below stuff from me, some feedback after working on a simple pure parser for a small project on Asterisk.

He mentions that the bison team should be informed also... So here you go. Pure parsers and integration
with flex, and some side-issues, were my challenges. In the below, you will see that perhaps some simple things could be done to improve the integration between bison and flex.

murf




-----Forwarded Message-----

> From: W. L. Estes <address@hidden>
> To: Steve Murphy <address@hidden>
> Subject: Re: How would you attack this?
> Date: 06 Sep 2004 08:15:22 -0400
> 
> On Thursday, 15 July 2004,17:57 -0600, Steve Murphy wrote:
> 
> > OK, got things running. A few more nits for you! Well, I consider these
> > "Legitimate Complaints" about
> > flex/bison, now. Slightly more serious than "Nit"...
> 
> We always appreciate any problem reports. flex should just work as documented.
> 
> > 1. Location Integration with Bison.
> > 
> > I was a bit disappointed that  yycolumn is not kept up to date. While
> > this variable might be easy
> > for the scanner to keep up to date, it is a bit harder for the
> > programmer to do so!
> > Experimentation led me to use the following formula to get what should
> > have been in yycolumn:
> > 
> > yylloc_param->first_column = (int)(yytext -
> > YY_CURRENT_BUFFER_LVALUE->yy_ch_buf);
> 
> I'll have a look at this and see if we can (and should put it in.
> 
> > 2. Clumsy Integration with yyerror
> > 
> > Had to relocate the yyerror routine to the last section of the flex
> > scanner input file.  This isn't so much a complaint against flex as
> > bison;;; I had to do some real dancing in bison to get yyerror to have
> > the args I needed. All I wanted to do was have yyerror show the string
> > it was parsing, and where in the string the token
> > was located that caused the error. But to do this, you need access to
> > the BUFFER_STATE. And that, to me, is deep-down in the bowels of the
> > flex scanner. Not nice.
> 
> bison issues are tricky. You want to be sure to talk to both flex and
> bison maintainers if you think there's something that both of should
> be doing. On the other hand, sometimes it's ahrd to tell who should be
> doing what.
> 
> > 3. PURE == PAIN
> > 
> > Here's another complaint that applies perhaps more to bison than to
> > flex. Bison allows exactly one arg to poor yyparse. And, if you are
> > defining a PURE (reentrant) parser, it seems to me only logical that
> > your scanner is going to be PURE also. If the scanner is PURE, it has to
> > have some initialization before it is called. This must therefore be
> > done before you call yyparse-- as yyparse calls yylex automatically,
> > it's a gamble to try to do it in one of the rules or something.
> > Therefore, the scanner_t var is the elected candidate to pass down into
> > yyparse. BUT--- that's not all. a Pure parser can't form it's output (if
> > it has any) into some global var, as earlier bisons were seemingly meant
> > to do.  A pure parser has to RETURN the data structure, evaluation
> > results, whatever to the func that called the parser. Since yyparse uses
> > the int return value to return a more general status code,
> > you can't do it there. So, you create a struct with the inputs to
> > yyparse/yylex, and the results you want to fetch, and you pass a pointer
> > to that.
> 
> Pretty much as above. This is where things get frustrating. It would
> have been nice if the original designs had included things like
> thread-safety. Now we're stuck without it and we've got our own design
> goals and so forth to deal with in addition.
> 
> -- 
> Will Estes
> http://tldz.org/

-- 
murf:   Steve Murphy, 	       CTO & VP Technology
        57 Lane 17        Electronic Tools Company
        Cody, Wyoming          work: (307)754-8154
        82414-9688             home: (307)754-5675
        email:                    address@hidden