[Octave-bug-tracker] [bug #51589] crash on regexp

[Rik]

Follow-up Comment #6, bug #51589 (project octave):

@Mike: If you want to research how to tune PCRE I'm fine with that. 
Otherwise, I'm fine with calling this an upstream issue in PCRE.

@Marcin: Neither Matlab nor Octave is exceptionally good at text processing. 
If you have lots and lots of data, and several different block identifiers, it
is probably better to select a different tool such as Perl to pre-process the
data.

I will say that the normal paradigm when dealing with large data is to work on
it in small chunks.  It may have been only a demonstration program, but
loading an entire file as the attached script does is always going to be a
problem.


fid=fopen("oc.txt",'r');
text = char (fread (fid, "uchar")');
fclose(fid);


A better plan of attack would be something like this


fid = fopen ("oc.txt", "r");
data = zeros (1e3, 11);  # Choose something large
cnt = 1;

while (ischar (text = fgets (fid)))
  ## Check text for your condition.
  ## strcmp, strfind, regexp, etc. might be possibilities
  ## Simple check to ignore text entries
  if (isletter (text(1)))
    continue;
  endif

  ## Otherwise, assume data just gets added to array
  data(cnt++, :) = sscanf (text, "%f").';
  if (cnt == rows (data))
    data = resize (data, [2*cnt, 11]);  # Grow array occasionally,
  endif

endwhile

fclose (fid);
## Remove unused elements
data = resize (data, [cnt, 11]);



This still isn't particularly fast, but it's not prone to segfaults.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?51589>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/