[Octave-bug-tracker] [bug #38326] save -binary or load fails with big matrices

[John W. Eaton]

Follow-up Comment #3, bug #38326 (project octave):

Yeah, we want to fix this so that large files can be opened and read, at least
on 64-bit systems where you will be able to store all the data, provided
that:

* each data object is less than 2GB

* some data objects are larger but contain less than about 2e9 elements.

* some data object contain more than 2e9 elements and you've compiled with
--enable-64 (which is not yet fully functional, so just doing that will not
solve your problem).

These changes will require a new version of the Octave binary file format
since the current format uses 32-bit integers for writing dimensions,
regardless of the actual size of octave_idx_type.

For the Matlab file format, I don't think it is possible to have arrays larger
than 2GB unless you use the HDF5-based file format and Octave does not yet
support loading files of that type.

On 32-bit systems, it should also be possible to load portions of large files,
but you clearly won't be able to load an entire file larger than 2GB into
memory all at once.  I can't remember whether Octave's load command currently
supports loading only some variables from a file.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?38326>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/