Re: better error reporting with bison and flex

[Tim Van Holder]

Guillaume Rousse wrote:
> Which is really not meaningful for user :/ I'd prefer to at least
> display current line, and avoid refering to internal grammar symbols.

Part 2 - not referring to internal grammar symbols: use string aliases
for your tokens, like so:

%token <sometype> MY_IDENTIFIER "identifier"
%token <sometype> MY_FUNC_ABS   "abs()"

The verbose error reporting will use those friendlier names, and you can
use them in your grammar too, if you like:

delete_statement : "DELETE" "identifier" "FROM" table_name "." ;

Part 1: showing the current line

There's two ways, really:
- if you always read from a file, and know the file name, you could
  open the file, skip (yylineno - 1) lines, read a line and show it.
  This is suboptimal and only possible in rare cases (usually you don't
  know what you're reading from), but doesn't require any scanner
  changes.
- start flex in a special state (say, BUFFER_LINE), in which an entire
  line is matched.  Save yytext in your parser context.  Then put the
  entire line back and switch to the INITIAL state.  All that then
  remains is to switch to the BUFFER_LINE state when you scan a newline.
  (Even cleaner would be to enable a state stack in flex, and push the
  BUFFER_LINE at the start of a file and when a newline is seen; that
  way it doesn't interfere with any other states you might be using).

Example (consider this pseudo-code; it's a heavily stripped and
C++-to-C-ified version of one of my own scanners, and I may have
stripped too much):

% // start of flex rules

  {
    if (context.line == 0) { // Fresh file - set up the initial state
      context->line = context->column = 1;
      BEGIN(INITIAL);
      yy_push_state(BUFFER_LINE);
    }
  }

<BUFFER_LINE>^[^\r\n]*/{EOL}? { // Get & save entire line
  context->current_line = strdup (yytext);
  yy_pop_state ();
  yyless (0);
  yy_set_bol ();
}

<*>{EOL} {
  context->column = 1;
  ++context->line;
  yy_push_state(BUFFER_LINE);
}

%

void
yyerror(parser_context_t* context, const char* message)
{
  ++context->parse_errors;
  if (yytext[0] == '\0')
    complain(context->line, context->column - yyleng,
             format(_("%s at empty token"), message));
  else
    complain(context->line, context->column - yyleng,
             format(_("%s at token `%s'"), message,
                   ((yytext[0] == '\n') ? _("[end-of-line]") :
                    yytext)));
  maybe_show_context_line(context);
}

Note that context is a structure that's set up as a lex & parse
parameter (to avoid global variables).
maybe_show_context_line() only prints something if current_line is set,
and adds markers based on the column number and yyleng.