[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Slowness in function 'open'
From: |
Dmitri A. Sergatskov |
Subject: |
Re: Slowness in function 'open' |
Date: |
Thu, 21 Jun 2007 16:02:43 -0500 |
On 6/21/07, Mark B. <address@hidden> wrote:
....
The size of my file is 30Mb
...
- Is there a faster way to load this type of files? Maybe less compatible
with Matlab, more native of Octave, but more efficient (this is something
other programs do: leave the unoptimized version of a function for legacy
purposes but provide a new that is more efficient)
Since you have not mentioned what is your hardware, it is hard to make a
meaningful comparison. But in any case on my CoreDuo II (2GHz / 2GB)
Fedora 7 octave 2.9.9 reading 5 rows x 512000 lines file xxx.dat (25 MB):
octave:1> tic; a=load("xxx.dat"); toc
Elapsed time is 6.645401 seconds.
octave:2> tic; a=load("xxx.dat"); toc
Elapsed time is 6.690552 seconds.
(*** Observation A -- I do not see effect of caching)
octave:3> save -binary "a.bin" a
octave:4> tic; load("a.bin"); toc
Elapsed time is 0.037651 seconds.
octave:5> ls -l a.bin
-rw-rw-r-- 1 dima dima 20480045 2007-06-21 15:51 a.bin
Alternatively:
octave:1> fh = fopen("xxx.dat")
fh = 3
octave:2> tic; b=fscanf(fh, "%f %f %f %f %f", [5, inf]); toc
Elapsed time is 4.445282 seconds.
octave:3> tic; b = b'; toc
Elapsed time is 0.050301 seconds.
(This is an optional step, if one wants b to have the same dimensions as a)
- Is this a known issue? i.e., is everybody else having the same performance
with the same type of files of comparable sizes?
It is a known issue that reading a formatted ASCII takes longer than reading
a binary file.
Hope that helps.
Sincerely,
Dmitri.
--
Re: Slowness in function 'open', Dmitri A. Sergatskov, 2007/06/21