Re: Bison Files and Token Order

[Joel E. Denny]

On Mon, 21 Jan 2008, Joel E. Denny wrote:

> On Tue, 22 Jan 2008, tim wrote:
> 
> > That looks OK. It used to work (putting token numbers on %left/%right/%
> > nonassoc), but bison has had a number of incompatible changes over past
> > years. 
> 
> Thanks for reporting this.  Yep, Open Group says Yacc requires it, and it 
> looks like Bison has been missing it since 1.50.

I committed this to fix it.

Index: ChangeLog
===================================================================
RCS file: /sources/bison/bison/ChangeLog,v
retrieving revision 1.1767
diff -p -u -r1.1767 ChangeLog
--- ChangeLog   6 Feb 2008 09:57:35 -0000       1.1767
+++ ChangeLog   17 Feb 2008 00:11:18 -0000
@@ -1,3 +1,21 @@
+2008-02-16  Joel E. Denny  <address@hidden>
+
+       Accept a token number in a %left, %right, or %nonassoc for POSIX
+       conformance.  Reported by Tim Josling at
+       <http://lists.gnu.org/archive/html/bug-bison/2008-01/msg00010.html>.
+       * NEWS (2.3a+): Mention.
+       * doc/bison.texinfo (Precedence Decl): Describe how literal strings
+       and code numbers are treated by precedence declarations.
+       * src/parse-gram.y (precedence_declaration): Use symbols.prec instead
+       of symbols.1.
+       (symbols.prec): New, just like symbols.1 but uses symbol.prec instead
+       of symbol.
+       (symbol.prec): New, just like symbol but allows INT.
+       * src/symtab.c (symbol_user_token_number_set): Remove an aver that no
+       longer holds.
+       * tests/regression.at (Token number in precedence declaration): New
+       test case.
+
 2008-02-06  Juan Manuel Guerrero  <address@hidden>
 
        DJGPP specific issues.
Index: NEWS
===================================================================
RCS file: /sources/bison/bison/NEWS,v
retrieving revision 1.184
diff -p -u -r1.184 NEWS
--- NEWS        1 Dec 2007 19:44:35 -0000       1.184
+++ NEWS        17 Feb 2008 00:11:18 -0000
@@ -154,6 +154,10 @@ Changes in version 2.3a+ (????-??-??):
   See the section `Freeing Discarded Symbols' in the Bison manual for further
   details.
 
+* %left, %right, and %nonassoc can now declare token numbers.  This is required
+  by POSIX.  However, see the end of section `Operator Precedence' in the Bison
+  manual for a caveat concerning the treatment of literal strings.
+
 Changes in version 2.3a, 2006-09-13:
 
 * Instead of %union, you can define and use your own union type
@@ -978,7 +982,7 @@ End:
 -----
 
 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
-2004, 2005, 2006, 2007 Free Software Foundation, Inc.
+2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc.
 
 This file is part of Bison, the GNU Compiler Compiler.
 
Index: doc/bison.texinfo
===================================================================
RCS file: /sources/bison/bison/doc/bison.texinfo,v
retrieving revision 1.248
diff -p -u -r1.248 bison.texinfo
--- doc/bison.texinfo   31 Jan 2008 00:53:21 -0000      1.248
+++ doc/bison.texinfo   17 Feb 2008 00:11:22 -0000
@@ -34,7 +34,8 @@ This manual is for @acronym{GNU} Bison (
 @value{UPDATED}), the @acronym{GNU} parser generator.
 
 Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
-1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free Software Foundation, 
Inc.
+1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software
+Foundation, Inc.
 
 @quotation
 Permission is granted to copy, distribute and/or modify this document
@@ -4088,7 +4089,7 @@ once.  These are called @dfn{precedence 
 @xref{Precedence, ,Operator Precedence}, for general information on
 operator precedence.
 
-The syntax of a precedence declaration is the same as that of
+The syntax of a precedence declaration is nearly the same as that of
 @code{%token}: either
 
 @example
@@ -4126,6 +4127,18 @@ When two tokens declared in different pr
 the one declared later has the higher precedence and is grouped first.
 @end itemize
 
+For backward compatibility, there is a confusing difference between the
+argument lists of @code{%token} and precedence declarations.
+Only a @code{%token} can associate a literal string with a token type name.
+A precedence declaration always interprets a literal string as a reference to a
+separate token.
+For example:
+
address@hidden
+%left  OR "<="         // Does not declare an alias.
+%left  OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
address@hidden example
+
 @node Union Decl
 @subsection The Collection of Value Types
 @cindex declaring value types
Index: src/parse-gram.y
===================================================================
RCS file: /sources/bison/bison/src/parse-gram.y,v
retrieving revision 1.123
diff -p -u -r1.123 parse-gram.y
--- src/parse-gram.y    1 Dec 2007 19:44:36 -0000       1.123
+++ src/parse-gram.y    17 Feb 2008 00:11:23 -0000
@@ -1,7 +1,7 @@
 %{/* Bison Grammar Parser                             -*- C -*-
 
-   Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007 Free Software Foundation,
-   Inc.
+   Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software
+   Foundation, Inc.
 
    This file is part of Bison, the GNU Compiler Compiler.
 
@@ -192,12 +192,12 @@ static int current_prec = 0;
 %type <integer> INT
 %printer { fprintf (stderr, "%d", $$); } INT
 
-%type <symbol> id id_colon symbol string_as_id
+%type <symbol> id id_colon symbol symbol.prec string_as_id
 %printer { fprintf (stderr, "%s", $$->tag); } id symbol string_as_id
 %printer { fprintf (stderr, "%s:", $$->tag); } id_colon
 
 %type <assoc> precedence_declarator
-%type <list>  symbols.1 generic_symlist generic_symlist_item
+%type <list>  symbols.1 symbols.prec generic_symlist generic_symlist_item
 %%
 
 input:
@@ -399,7 +399,7 @@ symbol_declaration:
 ;
 
 precedence_declaration:
-  precedence_declarator type.opt symbols.1
+  precedence_declarator type.opt symbols.prec
     {
       symbol_list *list;
       ++current_prec;
@@ -424,6 +424,19 @@ type.opt:
 | TYPE           { current_type = $1; tag_seen = true; }
 ;
 
+/* Just like symbols.1 but accept INT for the sake of POSIX.  */
+symbols.prec:
+  symbol.prec
+    { $$ = symbol_list_sym_new ($1, @1); }
+| symbols.prec symbol.prec
+    { $$ = symbol_list_prepend ($1, symbol_list_sym_new ($2, @2)); }
+;
+
+symbol.prec:
+    symbol { $$ = $1; }
+  | symbol INT { $$ = $1; symbol_user_token_number_set ($1, $2, @2); }
+  ;
+
 /* One or more symbols to be %typed. */
 symbols.1:
   symbol
Index: src/symtab.c
===================================================================
RCS file: /sources/bison/bison/src/symtab.c,v
retrieving revision 1.89
diff -p -u -r1.89 symtab.c
--- src/symtab.c        21 Sep 2007 22:53:57 -0000      1.89
+++ src/symtab.c        17 Feb 2008 00:11:23 -0000
@@ -1,7 +1,7 @@
 /* Symbol table manager for Bison.
 
-   Copyright (C) 1984, 1989, 2000, 2001, 2002, 2004, 2005, 2006, 2007 Free
-   Software Foundation, Inc.
+   Copyright (C) 1984, 1989, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008
+   Free Software Foundation, Inc.
 
    This file is part of Bison, the GNU Compiler Compiler.
 
@@ -358,8 +358,6 @@ symbol_user_token_number_set (symbol *sy
 {
   int *user_token_numberp;
 
-  aver (sym->class == token_sym);
-
   if (sym->user_token_number != USER_NUMBER_ALIAS)
     user_token_numberp = &sym->user_token_number;
   else
Index: tests/regression.at
===================================================================
RCS file: /sources/bison/bison/tests/regression.at,v
retrieving revision 1.117
diff -p -u -r1.117 regression.at
--- tests/regression.at 22 Dec 2007 18:35:03 -0000      1.117
+++ tests/regression.at 17 Feb 2008 00:11:23 -0000
@@ -1,6 +1,6 @@
 # Bison Regressions.                               -*- Autotest -*-
 
-# Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free Software
+# Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software
 # Foundation, Inc.
 
 # This program is free software: you can redistribute it and/or modify
@@ -1181,3 +1181,64 @@ state 2
 ]])
 
 AT_CLEANUP
+
+
+
+## ---------------------------------------- ##
+## Token number in precedence declaration.  ##
+## ---------------------------------------- ##
+
+AT_SETUP([[Token number in precedence declaration.]])
+
+# POSIX says token numbers can be declared in %left, %right, and %nonassoc, but
+# we lost this in Bison 1.50.
+
+AT_DATA_GRAMMAR([input.y],
+[[%{
+  #include <stdio.h>
+  void yyerror (char const *);
+  int yylex (void);
+%}
+
+%error-verbose
+%left TK1 1 TK2 2 "tok alias" 3
+
+%%
+
+start: TK1 sr_conflict "tok alias" ;
+
+sr_conflict:
+  TK2
+  | TK2 "tok alias"
+  ;
+
+%%
+
+void
+yyerror (char const *msg)
+{
+  fprintf (stderr, "%s\n", msg);
+}
+
+int
+yylex (void)
+{
+  static int const input[] = { 1, 2, 3, 0 };
+  static int const *inputp = input;
+  return *inputp++;
+}
+
+int
+main (void)
+{
+  return yyparse ();
+}
+]])
+
+AT_BISON_CHECK([[-o input.c input.y]], [[0]],,
+[[input.y:24.5-19: warning: rule useless in parser due to conflicts: 
sr_conflict: TK2 "tok alias"
+]])
+AT_COMPILE([[input]])
+AT_PARSER_CHECK([[./input]])
+
+AT_CLEANUP