Re: indexing improvements - advice wanted

[Jaroslav Hajek]

On Wed, Oct 29, 2008 at 11:06 AM, David Bateman
<address@hidden> wrote:
> Jaroslav Hajek wrote:
>>
>> On Mon, Oct 27, 2008 at 7:36 PM, Jaroslav Hajek <address@hidden> wrote:
>>
>>>>
>>>> Also it seems to me that a
>>>> case like
>>>>
>>>> A = speye(1e6,1e6)
>>>> A (A != 0) = randn(1e6,1);
>>>>
>>>>
>>>
>>> Currently, I've only dealt with dense array indexing. I haven't yet
>>> looked at sparse arrays, so I'm not even sure I can achieve some
>>> optimizations there. In the current patch the only minor changes to
>>> the sparse code are those caused by the changed idx_vector interface.
>>> But I definitely intend to look at sparse matrices as well, at minimum
>>> I want to clean up the interfaces as I did with Array<T>.
>>>
>
> Ok, I looked at your patch and it seems that you haven't yet taken into
> account the sparse indexing and assign at all as the methods "freeze" and
> "clear_index" of the index class are still used, and you've removed these
> from the idx_vector class. So something will have to be changes there.
>

With the current version, things will work. I've left the old
idx_vector::freeze & co. interfaces in idx_vector class for
compatibility but marked them as obsolete (currently, there is nothing
to "freeze" about idx_vector, because it's basically read-only and
everything important is computed at startup).
Also, the indexing system in Sparse<T> is left untouched, to avoid a
larger change than necessary.


> In any case it wasn't the indexing of sparse variables that I was mentioning
> here, but rather the use of a sparse variable as an index... A real sparse
> variable doesn't make much sense as an index, as the zeros in the variable
> will result in indexing errors, so conversion of this to a full array before
> using it as an index is a reasonable thing to do (to treat the case where a
> full matrix is stored as a sparse matrix). However when the sparse matrix is
> logical it in fact makes a lot of sense like in the example I gave above.
> The fact that the above will explode my memory at the moment has always
> annoyed me.
>

Hmmm, thinking about this... unless you want to make 'A(A != 0) =
something' a trick in the parser (possible but quirky), the robust
solution is to implement sparse logical matrices, so that A != 0 can
be computed as a sparse logical matrix and then converted to
idx_vector.

I guess this is a good idea, so I'm adding this to my long TODO list :)

>> The updated patch is uploaded, this time passing the test suite.
>> There were no flawed tests, after all - they were all bugs in my code.
>> So, the only changes in tests are caused by the different error
>> messages.
>> Of course, the presented speed-up numbers do still hold.
>>
>> Unless anyone objects, I'd like to apply this after 3.2.x is forked.
>>
>
> Seems fine to me..
>
> D.
>
> --
> David Bateman                                address@hidden
> Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph) Parc Les
> Algorithmes, Commune de St Aubin    +33 6 72 01 06 33 (Mob) 91193
> Gif-Sur-Yvette FRANCE                  +33 1 69 35 77 01 (Fax)
> The information contained in this communication has been classified as:
> [x] General Business Information [ ] Motorola Internal Use Only [ ] Motorola
> Confidential Proprietary
>
>



-- 
RNDr. Jaroslav Hajek
computing expert
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz