Re: [Flex-help] How to debug bison/flex program? usage of yyerror()

[Peng Yu]

On Wed, Dec 30, 2009 at 1:05 PM, Marcel Laverdet <address@hidden> wrote:
>
>
>> Do you actually mean '.' matches non newline?
>
> Yes, apologies :)
>
>> I made some corrections. Now yylval_string.l becomes the following. You
> said yyerror should not be used in the flex file. I'm wondering what I
> should use to replace the line '.       { yyerror("mystery character %c\n",
> *yytext); }' in order to catch errors.
>
> Really, it's up to you. You can replace it with whatever you want. You
> could replace it with assert(false) if you want or fprintf(stderr, "oh
> no!"), I don't know what you want it's your program. It's just a matter of
> what you want to do when something unexpected happens. Bison will generate
> error messages for you when something bad happens, but Flex is not in a
> position to generate error messages. In my scanners I usually do one of two
> things. I either memoize an error message in my yyextra (or just as a
> global if you're not using "%option reentrant"), which I can then raise to
> the client of my scanner/parser. Or, I do something like "return
> UNEXPECTED_CHARACTER;" (you would have to define this token). Then, Bison
> would generate an error such as "Error: Unexpected token
> UNEXPECTED_CHARACTER on line 5".

If I return UNEXPECTED_CHARACTER in yylval_string.l, how to change the
BNF in yylval_string.y?

>yylval_string.l
%option nodefault
%{
# include "yylval_string.tab.h"
%}
%%
[0-9]+ { yylval=atoi(yytext); return NUMBER; }
[[:space:]] { /*SPACE*/ }
.       { return UNEXPECED_CHARACTER; }
%%
>yylval_string.y
%{
#  include <stdio.h>
%}
%token NUMBER UNEXPECED_CHARACTER
%%
numbers:
       NUMBER { printf("NUMBER = %d\n", $1); }
       | numbers NUMBER { printf("NUMBER = %d\n", $2); }
 ;
%%
main()
{
  yyparse();
}

yyerror(char *s)
{
  fprintf(stderr, "error: %s\n", s);
}


> On Wed, 30 Dec 2009 09:39:30 -0600, Peng Yu <address@hidden> wrote:
>> On Wed, Dec 30, 2009 at 12:54 AM, Marcel Laverdet <address@hidden>
>> wrote:
>>>
>>>
>>> 1) [:space:] is a character class expression. If you want one or more
>>> spaces you would do [[:space:]]+. What your scanner is looking for
> right
>>> now is one of either ":, s, p, a, c, or e". Does that make sense? Just
>>> wrap
>>> it in another set of []'s
>>>
>>> 2) . only matches newline, the documentation is not lying :). The
> message
>>> you're seeing is from the spaces, since your [:space:] does not match
>>> them
>>> correctly. Additionally, you're hitting the default rule, whose default
>>> action is to print the matching character. Pretend that this always
>>> exists
>>> at the bottom of your file:
>>
>> Do you actually mean '.' matches non newline?
>>
>>> <*>.|\n fprintf(strerr, "%c", *yytext);
>>>
>>> You can (and should) override this using "%option nodefault" which will
>>> make it so that your scanner fails to build if you don't handle all
>>> inputs.
>>> This is a much more reasonable behavior than just spitting out to
> strerr.
>>>
>>> 3) yyerror is a macro defined by bison, not flex. There should be no
>>> yyerror in your lex file. bison's definition of yyerror is not ("%s",
>>> ...)
>>> it's something different (which depends on your options to bison).
>>
>> I made some corrections. Now yylval_string.l becomes the following.
>> You said yyerror should not be used in the flex file. I'm wondering
>> what I should use to replace the line '.      { yyerror("mystery character
>> %c\n", *yytext); }' in order to catch errors.
>>
>> %option nodefault
>> %{
>> # include "yylval_string.tab.h"
>> %}
>> %%
>> [0-9]+ { yylval=atoi(yytext); return NUMBER; }
>> [[:space:]] { /*SPACE*/ }
>> .     { yyerror("mystery character %c\n", *yytext); }
>> %%
>>
>>
>>
>>> On Tue, 29 Dec 2009 23:02:33 -0600, Peng Yu <address@hidden>
> wrote:
>>>> I have the source files listed at the end of the message. I basically
>>>> want to parse a file with only numbers (separated by spaces) and print
>>>> the numbers out. It is an overkill to use bison/flex. But I just want
>>>> to try how to use bison/flex.
>>>>
>>>> I need to understand how to debug the program. Could somebody help me
>>>> with the following three questions?
>>>>
>>>> 1. I don't understand why the error message is printed. Shouldn't the
>>>> regexes [0-9]+ and [:space:] match all the strings in 'test.txt'.
>>>>
>>>> 2. I suspect that '.' is matched to EOF. I'm not sure if I'm correct.
>>>> But it seem that EOF can not be printed (is it why '%c' is printed
>>>> literally?).
>>>>
>>>> 3. Why yyerror() in the .l file has two arguments but it has one
>>>> argument in the .y file? Are they the same function or two different
>>>> functions?
>>>>
>>>> $ make
>>>> bison -d yylval_string.y
>>>> flex yylval_string.l
>>>> cc -o yylval_string yylval_string.tab.c lex.yy.c -lfl
>>>> $./yylval_string< test.txt
>>>> NUMBER = 133
>>>> error: mystery character %c
>>>>
>>>> NUMBER = 7
>>>> error: mystery character %c
>>>>
>>>> NUMBER = 33
>>>> error: mystery character %c
>>>>
>>>> NUMBER = 76
>>>>
>>>> NUMBER = 35
>>>>
>>>>
>>>> --------------------------------source files listed
>>>> below---------------------------
>>>>
>>>> $cat yylval_string.y %{
>>>> #  include <stdio.h>
>>>> %}
>>>>
>>>> %token NUMBER
>>>>
>>>> %%
>>>>
>>>> numbers:
>>>>        NUMBER { printf("NUMBER = %d\n", $1); }
>>>>        | numbers NUMBER { printf("NUMBER = %d\n", $2); }
>>>>  ;
>>>>
>>>> %%
>>>> main()
>>>> {
>>>>   yyparse();
>>>> }
>>>>
>>>> yyerror(char *s)
>>>> {
>>>>   fprintf(stderr, "error: %s\n", s);
>>>> }
>>>>
>>>>
>>>> $cat yylval_string.l
>>>> %{
>>>> # include "yylval_string.tab.h"
>>>> %}
>>>>
>>>> %%
>>>> [0-9]+ { yylval=atoi(yytext); return NUMBER; }
>>>> [:space:] { /*SPACE*/ }
>>>> .     { yyerror("mystery character %c\n", *yytext); }
>>>> %%
>>>>
>>>> $cat Makefile
>>>>  .PHONY: all
>>>>
>>>> all: yylval_string
>>>>
>>>> yylval_string:        yylval_string.l yylval_string.y
>>>>       bison -d yylval_string.y
>>>>       flex yylval_string.l
>>>>       cc -o $@ yylval_string.tab.c lex.yy.c -lfl
>>>>
>>>> $cat test.txt
>>>> 133 7 33 76
>>>> 35
>>>>
>>>>
>>>
> ------------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Verizon Developer Community
>>>> Take advantage of Verizon's best-in-class app development support
>>>> A streamlined, 14 day to market process makes app distribution fast
> and
>>>> easy
>>>> Join now and get one step closer to millions of Verizon customers
>>>> http://p.sf.net/sfu/verizon-dev2dev
>>>> _______________________________________________
>>>> Flex-help mailing list
>>>> address@hidden
>>>> https://lists.sourceforge.net/lists/listinfo/flex-help
>>>
>