Re: CSV Parser - Empty values

[John]

> one,two,,,three,four,"data,with comma"
>
> I wish to return an EMPTYDATA_TOK if an empty value is
> encountered (2 commas without any data in between)
>

The most elegant solution is for this scanner to return more than one
token (EMPTYDATA_TOK followed by DELIMETER_TOK).  This could easily be
done if flex had a token buffer. Then the rule's action would look
like this:

,,+    {  for(i=0; i < yyleng-1; i++){
             pushback(DELIMETER_TOK);
             pushback(EMPTYDATA_TOK);
                  }
                  pushback(DELIMETER_TOK);
                  RETURN;
           }

Where RETURN pops the token buffer and returns the token.  Subsequent
calls to yylex would pop the token buffer until it is empty.