[Octave-bug-tracker] [bug #42579] textscan error out of memory or dimension too large

[Philip Nienhuis]

Update of bug #42579 (project octave):

                Category:             Interpreter => Octave Function        
                Priority:              5 - Normal => 3 - Low                
              Item Group:              Regression => Performance            
                  Status:                    None => Postponed              
        Operating System:       Microsoft Windows => Any                    

    _______________________________________________________

Follow-up Comment #5:

FYI, some comments from the one who rewrote and extended textscan.m & friends
to what it is now.

textscan has been implemented as an m-file function. Its work is actually done
behind the scenes by strread.m, another m-file function.

The way the whole setup worked and still works now isn't very efficient; lots
of intermediate results on cellstr arrays with many memory copies. Yet I'm
surprised that a 37 MB file expands into 1.5-2 GB in RAM.
All this inefficiency exists because of the many, many, many options offered
by textscan and strread. As far as I'm concerned, for many complicated and big
files one had better resort to Perl or so.

This textscan/strread situation will only get better when a binary (=compiled)
textscan has been implemented. ATM no one has the time and priority for it.
2 or 3 years ago JWE has issued a first patch that could recognize input
parameters; the rest still has to be written. Having experience with the
m-file versions, I suppose it's a lot of work that may easily take a year or
more to mature and reach/surpass current ML compatibility.

Why isn't csv2cell usable for you? I've worked on that (I'm also the
maintainer of the io package where csv2cell and cell2csv live) and as I have
plans to improve those two functions I could use some input.

Anyway I'll mark this bug as postponed, lower priority, and clean up several
other statuses (like: regression -> performance - it was never better than it
is now; OS -> Any - same issues on Linux; Interpreter -> Octave function)


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?42579>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/