[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #44491] svd is slow and uses too much memory
From: |
Carnë Draug |
Subject: |
[Octave-bug-tracker] [bug #44491] svd is slow and uses too much memory |
Date: |
Mon, 09 Mar 2015 14:57:43 +0000 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.4.0 |
Update of bug #44491 (project octave):
Category: Octave Forge Package => Octave Function
Release: other => 3.8.2
Summary: statistics package: princomp is slow and uses too
much memory => svd is slow and uses too much memory
_______________________________________________________
Follow-up Comment #3:
>From a previous IRC discussion, the problem is actually on svd being very very
slow
2015-03-06 12:33:19 serviscopeMinor well the super-super-super slow thing is
confirmed to be in SVD. Not checked for results ...
2015-03-06 12:34:52 serviscopeMinor I'll check for correctness now
2015-03-06 12:43:02 serviscopeMinor s=100;
2015-03-06 12:43:02 serviscopeMinor times=[];
2015-03-06 12:43:02 serviscopeMinor r=0;
2015-03-06 12:43:02 serviscopeMinor for i=1:100
2015-03-06 12:43:02 serviscopeMinor n = floor(10000^(i/s));
2015-03-06 12:43:03 serviscopeMinor data = rand(n, 64);
2015-03-06 12:43:05 serviscopeMinor tic
2015-03-06 12:43:07 serviscopeMinor a = svd(data');
2015-03-06 12:43:09 serviscopeMinor t1= toc;
2015-03-06 12:43:11 serviscopeMinor tic
2015-03-06 12:43:13 serviscopeMinor [U, S, V] = svd(data');
2015-03-06 12:43:15 serviscopeMinor t2=toc;
2015-03-06 12:43:19 serviscopeMinor r = r + sum(U(:)) + sum(V(:)) +
sum(S(:));
2015-03-06 12:43:21 serviscopeMinor errors = sum(abs(U*S*V' -
data')(:));
2015-03-06 12:43:23 serviscopeMinor times(end+1,:) = [n, errors,
t1, t2];
2015-03-06 12:43:25 serviscopeMinor i
2015-03-06 12:43:27 serviscopeMinor end
2015-03-06 12:43:29 serviscopeMinor hold off
2015-03-06 12:43:31 serviscopeMinor clf
2015-03-06 12:43:33 serviscopeMinor loglog(times(:,1), times(:,2), 'b-');
2015-03-06 12:43:35 serviscopeMinor hold on
2015-03-06 12:43:37 serviscopeMinor loglog(times(:,1), times(:,3), 'g-');
2015-03-06 12:43:39 serviscopeMinor loglog(times(:,1), times(:,4), 'r-');
2015-03-06 12:43:41 serviscopeMinor legend('errors', 'svd', '[u, v, s]=
svd');
2015-03-06 12:43:43 serviscopeMinor running that
2015-03-06 12:43:45 serviscopeMinor no obvious errors, just massive
slowness.
2015-03-06 12:47:24 serviscopeMinor so there's certainly an awful bug in
SVD.
2015-03-06 12:47:43 serviscopeMinor in terms of speed
2015-03-06 12:49:12 serviscopeMinor I'm just checking to see if all the SVDs
are valid.
2015-03-06 12:49:21 serviscopeMinor depending on the person's data, though
the
PCA is not unique.
2015-03-06 12:55:24 serviscopeMinor carandraug: OK, the final script is
this:
2015-03-06 12:55:31 serviscopeMinor s=100;
2015-03-06 12:55:31 serviscopeMinor times=[];
2015-03-06 12:55:31 serviscopeMinor r=0;
2015-03-06 12:55:31 serviscopeMinor for i=1:100
2015-03-06 12:55:31 serviscopeMinor n = floor(10000^(i/s));
2015-03-06 12:55:32 serviscopeMinor data = rand(n, 64);
2015-03-06 12:55:34 serviscopeMinor tic
2015-03-06 12:55:36 serviscopeMinor a = svd(data');
2015-03-06 12:55:38 serviscopeMinor t1= toc;
2015-03-06 12:55:40 serviscopeMinor tic
2015-03-06 12:55:42 serviscopeMinor [U, S, V] = svd(data');
2015-03-06 12:55:44 serviscopeMinor t2=toc;
2015-03-06 12:55:46 serviscopeMinor r = r + sum(U(:)) + sum(V(:)) +
sum(S(:));
2015-03-06 12:55:50 serviscopeMinor errors = sum(abs(U*S*V' -
data')(:)) +
sum(abs(U*U'-eye(size(U)))(:)) + sum(abs(V*V'-eye(size(V)))(:));
2015-03-06 12:55:52 serviscopeMinor times(end+1,:) = [n, errors,
t1, t2];
2015-03-06 12:55:54 serviscopeMinor i
2015-03-06 12:55:56 serviscopeMinor end
2015-03-06 12:55:58 serviscopeMinor hold off
2015-03-06 12:56:00 serviscopeMinor clf
2015-03-06 12:56:02 serviscopeMinor loglog(times(:,1), times(:,2), 'b-');
2015-03-06 12:56:04 serviscopeMinor hold on
2015-03-06 12:56:06 serviscopeMinor loglog(times(:,1), times(:,3), 'g-');
2015-03-06 12:56:08 serviscopeMinor loglog(times(:,1), times(:,4), 'r-');
2015-03-06 12:56:10 serviscopeMinor legend('errors', 'svd', '[u, v, s]=
svd');
2015-03-06 12:56:12 serviscopeMinor so
2015-03-06 12:56:14 serviscopeMinor apparently all the SVDs are valid
2015-03-06 12:56:16 serviscopeMinor as in they reconstruct the original
matrix
and the matrices are all orthogonal.
2015-03-06 12:56:20 serviscopeMinor I believe that's sufficient for it to be
correct.
2015-03-06 12:56:22 serviscopeMinor but very, very, VERY slow.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?44491>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/