Re: Long save and very long load times

[Dmitri A. Sergatskov]

On Fri, Jun 5, 2020 at 8:44 PM Brett Green <green.brett.r@gmail.com> wrote:

On Fri, Jun 5, 2020 at 7:43 PM Dmitri A. Sergatskov <dasergatskov@gmail.com> wrote:

You are probably saving it as formatted as ascii. That takes a lot of time. Use binary (or hdf5 or mat)

Dmitri.

--

On Fri, Jun 5, 2020 at 8:30 PM Doug Stewart <doug.dastew@gmail.com> wrote:

did you try a binary file format?

--

DAS

Thank you both! Yes, I had been doing the default save which saves it in ASCII.

On average, -hdf5 takes 14s, -mat takes 9s, and -binary takes 6s to save. Loading from -binary took only 7s.

This is another good reminder to go back and question the inconveniences I grew accustomed to as a beginner, before knowing that they could be resolved...

What OS do you use? Your times still look slowish to me:

octave:1> xxx = repmat (randn(2000), [1,1,4,10]);
octave:2> whos xxx
Variables visible from the current scope:

variables in scope: top scope

   Attr Name        Size                     Bytes  Class
   ==== ====        ====                     =====  =====
        xxx      2000x2000x4x10         1280000000  double

Total is 160000000 elements using 1280000000 bytes

octave:3> tic; save xxx1.oct xxx -binary ; toc
Elapsed time is 0.407136 seconds.
octave:4> tic; save xxx2.oct xxx -binary ; toc
Elapsed time is 0.407825 seconds.
octave:5> tic; save xxx11.hdf5 xxx -hdf5 ; toc
Elapsed time is 0.640946 seconds.
octave:6> tic; save xxx12.hdf5 xxx -hdf5 ; toc
Elapsed time is 0.64018 seconds.
octave:7> tic; save xxx.txt xxx  ; toc
Elapsed time is 70.3177 seconds.

This is with a rather old-ish SSD (Samsung 850 EVO).

Dmitri.

--