Re: about reading large text files (> 10 GB) into an array

[marco atzeri]

On 10/4/2012 7:00 AM, address@hidden wrote:
> Hi Everyone,
> I have a very large text file (say, 10 GB) each line of which contains a
> float-point number. Can I read all the numbers in this file into an
> array in Octave? I tried textread() but it complains something about
> "out of bound" array, which I think may suggest Octave's incapability of
> handling large arrays.
> So, is there a way to read all the 10 GB numbers into an array in Octave?
>
> Thanks,
> Tom

how many rows/numbers ?
Have you tried dlmread or load ?
http://www.gnu.org/software/octave/doc/interpreter/Simple-File-I_002fO.html#Simple-File-I_002fO

10GB characters to be converted in less than 2^32 floats/doubles
should be possible with a standard octave.
More than 2^32 needs an octave built with 64bit array option.

Marco