Re: [bug-gawk] Parsing standard CVS data by gawk

[Andrew J. Schorr]

On Wed, Jul 08, 2015 at 12:17:17AM +0300, Jarno Suni wrote:
> For some use cases it is not necessary to have all data in memory in
> one time. This might not be optimal implementation for such a case. I
> also wrote an implementation that reads input line by line.

I think the normal gawk usage pattern is to handle one record at a time.
Reading the entire file into memory is not scalable, so cannot be held
out as a general solution to the problem. My sense is that the proper
solution is to write a CSV parser extension in C that handles all the corner
cases properly. We are seeking a volunteer to implement this.

Thanks,
Andy