[PATCH 1/2] gram: correct token numbering in precedence declarations

[Valentin Tolmer]

In a precedence declaration, when tokens are declared with a litteral char
(e.g. 'a') or with a full name (e.g. B), Bison used to behave differently :
the litteral tokens would be numbered first, and then the other ones,
leading to the following grammar :
%right A B 'c' 'd'
being numbered as such : 'c' 'd' A B.

* src/parse-gram.y (symbol.prec): Called the numbering method on token read.
* tests/conflicts.at (new): Added a test to control the behaviour.
---
 NEWS               |    8 +++++
 src/parse-gram.y   |   13 ++++++--
 tests/conflicts.at |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 105 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index a48ef71..aea72bc 100644
--- a/NEWS
+++ b/NEWS
@@ -290,6 +290,14 @@ GNU Bison NEWS
   when declaring associativity at the same time, with %left (or %right,
   %precedence, %nonassoc), B was inferior to A.
 
+** Token numbering preserved in case of mixed char and non-char declaration
+
+  When declaring tokens with a litteral character (e.g. 'a') or with a name
+  (e.g. B) in a precedence declaration, Bison used to number the litteral
+  tokens first.  For example, the following grammar :
+  %right A B 'c' 'd'
+  would lead to the tokens declared in this order : 'c' 'd' A B.
+
 ** Useless precedence and associativity
 
   Contributed by Valentin Tolmer.
diff --git a/src/parse-gram.y b/src/parse-gram.y
index 4e887e8..e4d36cc 100644
--- a/src/parse-gram.y
+++ b/src/parse-gram.y
@@ -503,8 +503,17 @@ symbols.prec:
 ;
 
 symbol.prec:
-  symbol     { $$ = $1; }
-| symbol INT { $$ = $1; symbol_user_token_number_set ($1, $2, @2); }
+  symbol
+    {
+      $$ = $1;
+      symbol_class_set ($1, token_sym, @1, false);
+    }
+| symbol INT
+    {
+      $$ = $1;
+      symbol_user_token_number_set ($1, $2, @2);
+      symbol_class_set ($1, token_sym, @1, false);
+    }
 ;
 
 /* One or more symbols to be %typed. */
diff --git a/tests/conflicts.at b/tests/conflicts.at
index 07ff178..fd3d5c3 100644
--- a/tests/conflicts.at
+++ b/tests/conflicts.at
@@ -87,6 +87,92 @@ AT_BISON_OPTION_POPDEFS
 AT_CLEANUP
 
 
+## -------------------------------------------------- ##
+## Token declaration order : mix of char and non-char ##
+## -------------------------------------------------- ##
+
+# This test checks that when several tokens are declared by the same keyword,
+# some of them defined as a character ('a'), others as simple textual reference
+# (A), they are declared correctly left to right.
+# Previously, the following test would declare the states in the order 'o' 'p'
+# M N, instead of M N 'o' 'p'.
+
+AT_SETUP([Token declaration order : mix of char and non-char])
+
+AT_DATA_GRAMMAR([[input.y]],
+[[%token 'a' 'b' C D
+%token E F 'g' 'h'
+%right 'i' 'j' K L
+%right M N 'o' 'p'
+%%
+exp: 'a'
+   | 'b'
+   | C
+   | D
+   | E
+   | F
+   | 'g'
+   | 'h'
+   | 'i'
+   | 'j'
+   | K
+   | L
+   | M
+   | N
+   | 'o'
+   | 'p'
+;
+%%
+]])
+
+AT_BISON_CHECK([[--report=all -o input.c input.y]], 0, [], [ignore])
+AT_CHECK([[cat input.output | sed -n '/^State 0$/,/^State 1$/p']], 0,
+[[State 0
+
+    0 $accept: . exp $end
+    1 exp: . 'a'
+    2    | . 'b'
+    3    | . C
+    4    | . D
+    5    | . E
+    6    | . F
+    7    | . 'g'
+    8    | . 'h'
+    9    | . 'i'
+   10    | . 'j'
+   11    | . K
+   12    | . L
+   13    | . M
+   14    | . N
+   15    | . 'o'
+   16    | . 'p'
+
+    'a'  shift, and go to state 1
+    'b'  shift, and go to state 2
+    C    shift, and go to state 3
+    D    shift, and go to state 4
+    E    shift, and go to state 5
+    F    shift, and go to state 6
+    'g'  shift, and go to state 7
+    'h'  shift, and go to state 8
+    'i'  shift, and go to state 9
+    'j'  shift, and go to state 10
+    K    shift, and go to state 11
+    L    shift, and go to state 12
+    M    shift, and go to state 13
+    N    shift, and go to state 14
+    'o'  shift, and go to state 15
+    'p'  shift, and go to state 16
+
+    exp  go to state 17
+
+
+State 1
+]])
+
+AT_CLEANUP
+
+
 ## ------------------------------- ##
 ## Useless associativity warning.  ##
 ## ------------------------------- ##
-- 
1.7.9.5