[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Need a faster way of reading files.
From: |
Ole Jacob Hagen |
Subject: |
Need a faster way of reading files. |
Date: |
Wed, 2 Jun 2010 08:23:54 -0700 (PDT) |
Hi,
I have large data files, with 90 columns. Typical size is 600-700 MB.
Data is stored in ascii format. This is data collected for 7200 s, with
sampling interval 100 Hz.
I am using data = load('mydata.txt'), but this is very very slow, compared
to SciPy.
SciPy uses one minute, while Octave is using approx 5-10 minutes. Haven't
measured time consumption, but can come back with that one. ;-)
I've tried to use fread, but the data is not recognizable at all.
I know how many columns I have in my data file.
How can I accelerate my data loading into memory in Octave?
Best regards,
Ole J. Hagen
--
View this message in context:
http://octave.1599824.n4.nabble.com/Need-a-faster-way-of-reading-files-tp2240331p2240331.html
Sent from the Octave - General mailing list archive at Nabble.com.
- Need a faster way of reading files.,
Ole Jacob Hagen <=