Re: error: memory exhausted or requested size too large for range of Octave's index type --

[Martin Helm]

Please keep the list on cc.

Am Mittwoch, 16. März 2011, 07:58:42 schrieb nuncio m:
> I am using octave v3.4.0 on opensuse 11.3
> Thanks
> 
> On Wed, Mar 16, 2011 at 12:13 AM, Martin Helm <address@hidden> wrote:
> > Am Dienstag, 15. März 2011, 18:05:44 schrieb nuncio m:
> > > Hi list,
> > > 
> > >         Here is the code I am trying to do a PCA analysis
> > >         
> > >         filename='test_eof.txt';
> > >         oufile='pc.txt';
> > >         data=load(filename);
> > >         data1=reshape(data,25200,132);
> > >         data2=data1';
> > >         f=detrend(data2,0);
> > >         R=f'*f;
> > >         [C,L]=eig(R);
> > >         pc1=f*C(:,1);
> > >         pc2=f*C(:,2);
> > >         fid=fopen(oufile,"wt");
> > >         fprintf(fid,'%10.3f \n',pc1);
> > >         fclose(fid);
> > >        
> > >        The code never completes the computation, it shows an error
> > 
> > "error:
> > > memory exhausted or requested size too large for range of Octave's
> > > index type -- trying to return to prompt"
> > > 
> > >        Can anyone suggest where it goes wrong.
> > >        Thanks
> > >        Nuncio
> > 
> > Not a problem at all with 32 bit octave if you use version 3.4 since only
> > the
> > index is limited to 2e9 not the bytesize of the array. An array can have
> > up to
> > 16 GB (if you have enough RAM + Swap).
> > With version 3.2 of course this does not work for the 32 bit version.
> > 
> > Which version of octave is that what you are using and which operating
> > system?
> > It is probably easy to update.

Ok so the bad news is that the build on a 32bit system cannot handle that even 
on linux. Sounds reasonable if the pointers are only 32 bit.

Looking at your code it is the R=f'*f which blows up to 25200x25200 just to 
calculate the eigen vectors.
Isn't it better to calculate the singular values for f directly and work with 
them  instead of creating this enormous large matrix product?