Re: benchmarks

[David Bateman]

Paul,

> ============================
> I.A   Matlab 0.48 : Octave 2.01
> 
>   tic;
>     a = abs(randn(1500, 1500)/10);
>     b = a';
>     a = reshape(b, 750, 3000);
>     b = a';
>   timing = toc;
> 
> Octave's random number generators are slower than they need
> to be.  octave-forge helps, but there are even better ones available.
> E.g., http://www.jstatsoft.org/v05/i08/
> 
> I've written to the author asking for permission to release it under
> an GPL compatible license, but didn't get a response.  I haven't
> had time to recode it in my own words.  It'll be tricky since the paper
> contains the source code.


I don't think these tests will take octave-forge into account, so speed 
increases in octave-forge won't show up in these tests. This is a good
justification to get the randon generator code from octave-forge into 
octave itself.

> I.D Matlab 0.73 - Octave 0.35 -  R 0.24
>     b = a'*a;
> 
> Octave could be faster if it skipped the transpose and used blas
> directly with the appropriate transpose code.   Note, the tester
> did not use a self-generated atlas, so these results don't mean
> much.

This is a lot of work and cruft foe a small gain. You'd have to pass
', *, and '* differently. Then you'd have to create a new function for
the transpose multiply. Is it worth it?

> I.E Matlab 0.24 - Octave 0.78
>     c = a\b';
> 
> ===================================
> 
> I don't understand how O-Matrix is consistently faster
> on the following tests.  Aren't they all using the same
> underlying libraries?  Or is O-Matrix hand-optimized
> for Intel architecture?
> 
> II.A Matlab 0.48 - Octave 0.96 - O-Matrix 0.17
>     b = fft(a);
> Not sure if he used fftw.
> 
> II.B Matlab 0.86 - Octave 2.30 - O-Matrix 0.44
>     b = eig(a);
> 
> II.C Matlab 0.27 - Octave 1.02 - O-Matrix 0.17
>    b = det(a);
> 
> II.D Matlab 0.33 - Octave 0.21 - O-Matrix 0.22
>    b = chol(a);
> 
> II.E Matlab 0.23 - Octave 0.47 - O-Matrix 0.11
>    b = inv(a);
> 

The version they tested had many functions using the underlying linpack
routines rather than lapack and atlas. This explains the speed reduction
of inv and det, etc.

> ========================================
> III.A Matlab 2.11 - Octave 2.06 - O-Matrix 0.59
>     b = (phi.^a - (-phi).^(-a)) / sqrt(5);
> 
> III.B Matlab 0.84 - Octave 0.73 - O-Matrix 0.47
>     b = ones(a, a)./((1:a)' * ones(1, a) + ones(a, 1) * (0:(a-1)));


Same solution as for I.B should help out here.


Cheers
David

-- 
David Bateman                                address@hidden
Motorola CRM                                 +33 1 69 35 48 04 (Ph) 
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax) 
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as: 

[x] General Business Information 
[ ] Motorola Internal Use Only 
[ ] Motorola Confidential Proprietary