[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: cannot load large matrix, new datatype/fseek

From: daniel heiserer
Subject: Re: cannot load large matrix, new datatype/fseek
Date: Fri, 18 Jun 2004 03:36:21 -0500


> I have a large matrix in the following mat file:
> 8153243903 14. Jun 16:49 ftn51.mat
> The matrix is approximately 5,000,000 x 200 and needs around 8GB of
> space.
> Unfortunately octave cannot read the matrix. It either stops immediatley
> on my 32bit machine (of course) or on a 8GB 64bit machine it simply
> crashes.
> I only need a few rows of that matrix, but all columns.
> I have two attesmpts for reading that matrix:
> 1) write an m-file which reads only the elements I want.
> For that I am not sure if I can "naviagte" using fseek in a file larger
> then 2GB.
> If anyone up to know has written an m-file based .mat parser I would be
> glad for
> any help

So this idea is bad. I am even not able to open the file:

[myfile,msg] = fopen("ftn51.mat", "r", "ieee-le")
myfile = -1
msg = File too large

What has to be done to improve this? Must I change the octave related
int4's to int8s?

-- daniel

> 2) Implement a new data type OOC-matrix. This datatype holds only the
> few values
> describing the matrix in memory (imaginary flag, n-columns,n-rows).
> The rest of the matrix is a blob on disk.
> Whenever this datatype is used in a function or whatever it shall load
> the necessary elements from disk.
> As I haver never implemented a new datatype I ask for some documentation
> or examples so I can start realizing it. Also I must somehow be able to
> tell "load" that the  matrix in the file shall be assigned to that
> datatype.
> I am glad for any help,
> daniel

Mit freundlichen Gruessen
    Daniel Heiserer

Dipl.-Phys. Daniel Heiserer, BMW AG, Knorrstrasse 147, 80788 Muenchen
Abteilung EK-212
Tel.: 089-382-21187, Fax.: 089-382-42820

Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:
How to fund new projects:
Subscription information:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]