[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: reading ascii matrix
From: |
Paul Kienzle |
Subject: |
Re: reading ascii matrix |
Date: |
Fri, 9 Feb 2001 20:47:51 +0000 |
User-agent: |
Mutt/1.2.5i |
If the data is too large for your machine, you may want to process it
in blocks. E.g.,
for i=1:p/1e5+1
blk=fscanf(myfile,"%f",2e5);
blk=reshape(blk,length(blk)/2,2);
a=blk(:,1); b=blk(:,2);
...
end
Note that a,b will probably not contain 1e5 data points each on
the final iteration.
On Fri, Feb 09, 2001 at 09:25:40AM -0500, Ben Bauer wrote:
> Info:
> Octave, version 2.1.9 (i686-pc-linux-gnu).
> (PIII 550 with 256M ram)
> Linux 2.2.16
> egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
>
>
>
> I have a plain ascii file of 2 columns and
> approx 2.8 million rows that I need to do some
> statistics on. In my octave script I have
>
> for i=1:p
> [a(i),b(i)]=fscanf(myfile,"%f %f","C");
> ...
>
> where p is the number of records in the file.
>
>
> This program has been reading in the data for
> 2 days now. Currently just less than half the
> data are read in according to a print command I
> have in the loop. There must be a better way!
> The the entire script runs in about 1.5 minutes
> for a shorter file of 20000 data pairs, with the
> majority of the time taken in reading the data.
>
>
>
> Any useful suggestions would be appreciated. Thanks.
>
>
>
> -------------------------------------------------------------
> Octave is freely available under the terms of the GNU GPL.
>
> Octave's home on the web: http://www.octave.org
> How to fund new projects: http://www.octave.org/funding.html
> Subscription information: http://www.octave.org/archive.html
> -------------------------------------------------------------
>
>
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------