Re: Bison/flex compatibility revisited

[Paul Eggert]

Bruce Lilly <address@hidden> writes:

> for future use, how about something less likely to clash,
> e.g. yy_bison_ as a prefix?

I dunno, that sounds a bit long.  But before settling on a solution
I'd like to understand the problem better.

First, POSIX says this about yacc:

  Conforming applications shall not use names beginning in yy or YY
  since the yacc parser uses such names.
  -- <http://www.opengroup.org/onlinepubs/007904975/utilities/yacc.html>

So, strictly speaking, a portable lexer cannot make visible to a
yacc-generated parser any names beginning with yy or YY, other than
those listed under the yacc spec (namely yylex, yyerror, YYDEBUG,
yydebug, YYSTYPE, yylval, and yychar, and perhaps YYSTYPE).  Whenever
flex generates a lexer that defines other names beginning with yy or
YY, the resulting lexer does not conform to POSIX, and might not work
with some POSIX-compliant yacc implementation.

Now, for lex POSIX says this:

  Except for input(), unput(), and main(), all external and static
  names generated by lex shall begin with the prefix yy or YY.
  -- <http://www.opengroup.org/onlinepubs/007904975/utilities/lex.html>

This means a POSIX-compliant lex-generated lexer can generate any
external name starting with yy, including yyparse!  Which means that
in general one cannot combine a lex-generated lexer with a
yacc-generated parser.

So it appears that this name clash is present in POSIX; it's not just
a property of flex and bison.  If we solve the problem with flex and
bison, our solution will be an extension to POSIX, which may allow
flex and bison to work with each other, but will not necessarily solve
the problem with a non-flex lexer or a non-bison parser.

Ouch.

Anyway, there are two parts of the problem here.  The first is that a
yacc-generated parser must stick to names beginning with yy/YY, to
avoid colliding with user-defined macros.  This is true even for
private names like local variables, in C at least, because macros can
wreak havoc with local variables.  (Presumably C++ parsers use
namespaces and disallow macros, so they don't have this problem.)
Here, if there is a clash, Bison can easily rename the private name to
avoid the collision.

How about if we reserve 'yyy', 'yyY', 'YYy', and 'YYY' for this use.
In other words, flex should never use any name preceded by these
prefixes, for any purpose.  This policy would have prevented the
yylineno problem that prompted this thread.

The second part of the problem is public names.  If flex introduces a
new feature called (say) 'yygreater', and Bison introduces a different
feature with the same name, then there will be a collision.  I'm a bit
reluctant to switch to a prefix like 'yy_bison_' to address this, just
as I'm sure flex users would be reluctant to switch to 'yy_flex_' for
all the flex extensions: there is an existing user base, and the
longer names are less likely to become standard in the future.

Perhaps we can just leave this second problem for now, and try not to
step on each other's names.  (I can, however, promise that Bison won't
reserve names beginning with 'yy_flex_'.  :-)


> FYI, as of now the complete list (unless I missed some) of yy and
> YY prefixed names used by the flex-generated header is:

Unfortunately it would take a nontrivial amount of work to go through
the Bison templates and code, and decide which uses of these names
were intended, and which were not.