Re: Very slow reading of a 2D matirx

[David Bateman]

John W. Eaton wrote:

> On 19-Oct-2005, Alireza wrote:
>
> | I have a moderately sized 2D array (180X180), that want to read into octave.
> | The reading is extremely slow. I have tried to create the array first, but
> | the speed is still the same.
> | 
> | I have tried this on various machines and octave versions, from the rusty to
> | bleeding!
> | 
> | Any thoughts and suggestions as to why this is and what needs to be done!
>
> Please try the following patch.  It is relative to the current CVS
> (2.9.x).  To apply it to older versions, you will need to change the
> occurrences of octave_idx_type to be int instead.
>
> With the patch, I can load the example file you sent in about 1
> second, compared to about 60 seconds without the changes.
>
> Note that if you put any non-constant value in a list like this, or
> concatenate user-defined objects, the performance will be bad again.
> I'm not sure how to fix that problem, since concatenation of mixed
> types or user-defined types must use the "cat" operators, which
> operate on a pair of elements at a time and repeatedly resize as they
> are called (I think, as I don't have time to check right now -- maybe
> David Bateman can quickly say for sure).
>  
The tree is transversed once extracting the dim_vector's to get the size 
of the return value and to find any dimensioning error. The type of the 
return value is then assumed to be the same as the first element to be 
concatenated which is resized to be the correct size of the return 
value.  The concatenated elements are then inserted into this return 
value by the do_cat operator in pairs, with promotion of the return type 
handled at the same time if needed. The promotion process makes a copy 
and so this might be a cause of a slow-up, but not in this case as there 
is no promotion. The pairs of values passed to do_cat are the return 
value and the value to be inserted into it. The first call to do_cat 
starts with an empty return value and inserts the first value since I 
found it faster to do this rather than resize the first value. It seems 
that resize is slower than it should be, though this isn't the cause of 
the issue here...

The case causing the problem, although common is a difficult one to 
handle with the current scheme, as do_cat is called on the octave_value 
itself, so I can count 5 levels of function calls for each call to the 
do_cat operator, which is ok when the insert itself takes a significant 
time as for larger matrices being concatenated. However inserting one 
scalar after another 180x180 times this probably becomes 
catastrophically slow. I think the only way to treat this is the way you 
have by special casing real/complex matrices of this form, with perhaps 
the ability to mix real complex values so that [1,1i;1i,1] would also be 
special cased, since this is likely to come up similarly to the case here...

Regards
David

--
David Bateman                                address@hidden
Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph) 
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax) 
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as: 

[x] General Business Information 
[ ] Motorola Internal Use Only 
[ ] Motorola Confidential Proprietary



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------