Re: Reading data from large ascii-file

[marco atzeri]

On Tue, Apr 19, 2011 at 9:41 AM, Peter Norlindh  wrote:
> Hi,
>
> I am analyzing data from a 400MB ascii-file.  There are about 30 million
> data points in the file and the execution takes forever.
>
> Currently the program reads and processes the file element-by-element ( val
> = dlmread(fileName, "emptyvalue", [i, col, i, col]) ) and I suspect that
> this is a very unafficient way to do it.  Could you suggest a better
> approach?

surely not reading one element at time as you are doing with [i, col, i, col]

The RANGE parameter may be a 4-element vector containing the upper
     left and lower right corner `[R0,C0,R1,C1]' where the lowest index
     value is zero.

if the data is already in matrix form, you should read all the matrix
in one pass
with just

DATA = dlmread (FILE)



>
> Regards,
> hpon

Marco