[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: cannot load large matrix, new datatype/fseek
From: |
daniel heiserer |
Subject: |
Re: cannot load large matrix, new datatype/fseek |
Date: |
Fri, 18 Jun 2004 03:36:21 -0500 |
Hi,
> I have a large matrix in the following mat file:
> 8153243903 14. Jun 16:49 ftn51.mat
>
> The matrix is approximately 5,000,000 x 200 and needs around 8GB of
> space.
> Unfortunately octave cannot read the matrix. It either stops immediatley
> on my 32bit machine (of course) or on a 8GB 64bit machine it simply
> crashes.
>
> I only need a few rows of that matrix, but all columns.
>
> I have two attesmpts for reading that matrix:
>
> 1) write an m-file which reads only the elements I want.
> For that I am not sure if I can "naviagte" using fseek in a file larger
> then 2GB.
> If anyone up to know has written an m-file based .mat parser I would be
> glad for
> any help
So this idea is bad. I am even not able to open the file:
[myfile,msg] = fopen("ftn51.mat", "r", "ieee-le")
myfile = -1
msg = File too large
What has to be done to improve this? Must I change the octave related
int4's to int8s?
-- daniel
> 2) Implement a new data type OOC-matrix. This datatype holds only the
> few values
> describing the matrix in memory (imaginary flag, n-columns,n-rows).
> The rest of the matrix is a blob on disk.
> Whenever this datatype is used in a function or whatever it shall load
> the necessary elements from disk.
> As I haver never implemented a new datatype I ask for some documentation
> or examples so I can start realizing it. Also I must somehow be able to
> tell "load" that the matrix in the file shall be assigned to that
> datatype.
>
> I am glad for any help,
>
> daniel
--
Mit freundlichen Gruessen
Daniel Heiserer
--------------------------------------------------------------
Dipl.-Phys. Daniel Heiserer, BMW AG, Knorrstrasse 147, 80788 Muenchen
Abteilung EK-212
Tel.: 089-382-21187, Fax.: 089-382-42820
mailto:address@hidden
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------