Re: Octave Memory Exhausted Error Anyone?

[Ben Abbott]

On Nov 5, 2010, at 10:55 PM, William Krekeler wrote:
> 
> -----Original Message-----
> From: Ben Abbott [mailto:address@hidden 
> Sent: Friday, November 05, 2010 9:30 AM
> To: William Krekeler
> Cc: address@hidden
> Subject: Re: Octave Memory Exhausted Error Anyone?
> 
>> On Nov 5, 2010, at 10:17 PM, William Krekeler wrote:
>> 
>>> Follow up to my post Nov 3 2010 10:12:33 CDT. Since no one responded I'll 
>>> rephrase my question.
>>> 
>>> Does anyone know a reason why Octave 3.2.4 in Windows can not use more than 
>>> 1.5 GB of physical memory?
>>> 
>>> Bill Krekeler
>>> 
>>> 
>>> From: William Krekeler [mailto:address@hidden 
>>> Sent: Wednesday, November 03, 2010 10:13 AM
>>> To: address@hidden
>>> Subject: Octave Memory Exhausted Error
>>> 
>>> I am in the process of testing Octave compatibility of a code set developed 
>>> in Matlab. The code successfully completes in Matlab 7.8.0.347 (R2009a). 
>>> Despite having more than enough system memory I am getting the following 
>>> error in Octave:
>>> "error: memory exhausted or requested size too large for range of Octave's 
>>> index type -- trying to return to prompt."
>>> Any help determining the source of this error, or a work around, is greatly 
>>> appreciated.
>>> 
>>> Relevant Machine Details:
>>> Windows XP Pro x64
>>> 12 GB physical ram
>>> Octave 3.2.4, pre-compiled install version
>>> 
>>> Script Details for Original Failure:
>>>                I've isolated the code failure to one line, but the error is 
>>> not specific to this line of code as simplifying the line of code also 
>>> fails. The original line of code and the simplified test line are as 
>>> follows:
>>> 
>>>      <start code>
>>> ImageCube = uint16( floor( (ImageCube - 
>>> minIC).*(single(2^16-1)/(maxIC-minIC)) ) );
>>> % [ above generates error]
>>> ImageCube = ImageCube - minIC;
>>> % [above generates error ]
>>>      <end code>
>>> 
>>>                Variable and Memory Information is as follows:
>>> 
>>>                ImageCube = 648x480x648 single array using 806215680 bytes 
>>> (~ 0.8 GB)
>>>                minIC = 1x1 single array using 4 bytes
>>>                maxIC= 1x1 single array using 4 bytes
>>> 
>>>                At failure Octave is using 828,728 kB (~0.8 GB) of memory, 
>>> total memory used on system = 3.25 GB, total memory free is roughly 8.6 GB 
>>> for 12 GB total physical memory.
>>> 
>>> Testing:
>>>                Following the memory error above and a restart of Octave the 
>>> following command succeeds. The code below creates a variable whose size is 
>>> ~1.4 GB using the 1024 definition, at 4 bytes per element of the 376999999 
>>> element array. Note 377999999 elements fails with the same error. The test 
>>> code implies two things, first, my original variable does not violate a 
>>> maximum number of supported elements in an array, second, that my original 
>>> variable does not violate a maximum memory allowed per variable. Though the 
>>> test could be saying that Octave is being limited to roughly 1.5 GB of 
>>> memory, which if the code duplicates an array during the math operations 
>>> would mean that my original 0.8GB ImageCube variable would become a 1.6GB 
>>> memory requirement during subtraction of the full array. Failure to perform 
>>> mathematical operations on the large array is confirmed by the failure of 
>>> the subtraction operation in the following codeset.
>>> 
>>>      <start code>
>>> Clear all; ImageCube = ones(376999999, 1, 'single' );
>>> % [ above  succeeds ]
>>> ImageCube = ImageCube - 0;
>>> % [ above fails ]
>>>      <end code>
>>> 
>>> Any ideas for what could be causing this error or a work around?
>>> 
>>> 
>>> Bill Krekeler
>> 
>> Is your octave built for 64 bits?
>> 
>> I haven't used 64 bit Octave myself. However, my impression is that it 
>> remains experimental.
>> 
>> Ben
> 
> No it is not compiled for 64 bit. It is the pre-built 32 bit binary available 
> here: http://octave.sourceforge.net/. 64 bit XP has good 32 bit support so I 
> do not expect that this will be a problem. Also, I'm seeing a 1.5 GB limit, 
> well below the maximum addressable 32 bit memory space.
> 
> Bill Krekeler

For 32 bit apps, the entire program as well as all variables need to fit within 
2G. Does Windows 64 allow each 32 bit app to access 2G of memory, or does it 
work like a 32 bit OS and require that all 32 bit programs run within the same 
2G or memory?

Ben