Re: why are locations dictated by bison?

[Hans Aberg]

At 10:41 -0500 2002/01/16, Bruce Lilly wrote:
>Bison 1.28 uses 256 for error, not 1.  And it uses -1 for end.

Actually, Bison makes various token value translations, and I do not keep
track of them in detail. For exammple all values <= 0 are translated into 0
by the code:
  /* Convert token to internal form (in yychar1) for indexing tables with */

  if (yychar <= 0)              /* This means end of input. */
    {
      yychar1 = 0;
      yychar = YYEOF;           /* Don't call YYLEX any more */

      YYDERR("Now at end of input.\n");
    }
  else
    ...
where:
  #define YYEOF 0

If 1 is not used for error, that is great (but one must first check that
the token value 256 is not translated into 1 in some other part).

>0 is reserved, but one can define a NUL token and return that
>from flex via a rule like:
>
>\0  { return NUL; }

It would be great if the initial state could be changed to
  #define YYEOF -1
and
  if (yychar < 0)               /* This means end of input. */
    {
      yychar1 = 0;
      yychar = YYEOF;           /* Don't call YYLEX any more */

      YYDERR("Now at end of input.\n");
    }
  else
    ...
thus freeing up the values 0 as well, but I am not sure one can do so: I
think the Bison manual does not tell that the parser actually accepts all
<= 0 values,  and does not mention the YYEOF macro, but instead says that
the lexer should return 0 at the end. So a change will probably break some
code.

But if it was possible, then one would have reserved the full range
0..2^N-1 for characters alone, with error at 2^N, other tokens starting at
2^N+1, and end/start states < 0 (N = 8 for ASCII, N = 20 or 24 for Unicode).

  Hans Aberg