Re: CSV Parser - Empty values

[Hans-Bernhard Broeker]

On Tue, 18 Feb 2003, Ahsan A. wrote:

> Hi,
> 
> I'm new to flex & Bison, and I'm trying to write a csv
> parser. My question is regarding the lexical scanner:
> 
> In a .csv file, if there is line:
> 
> one,two,,,three,four,"data,with comma"
> 
> I wish to return an EMPTYDATA_TOK if an empty value is
> encountered (2 commas without any data in between)

You're searching for the solution in the wrong direction. This is not
really something for the lexer to do. Whether or not a given token is
empty should be detected by the *parser* instead.

> What I wish to do is return an EMPTYTOK, when an empty
> value (between two commas ) is matched. 

Without introducing scanner states, you can't return EMPTYTOK by itself.
What you need would be a rule that catches ,, or a , at the start or end
of the line, and returns "EMPTYTOK_WITH_DELIMITERTOKEN" for it.

And even that wouldn't catch

        ,"",

which also is an empty token, in at least one possible interpretation of 
CSV files.

-- 
Hans-Bernhard Broeker (address@hidden)
Even if all the snow were burnt, ashes would remain.