Re: memory surprise; help a struggling newbie

[Richard Frith-Macdonald]

On 22 Apr 2004, at 07:19, Travis Griggs wrote:

> From the irc channel the other day, I was under the impression that 
> factory methods do an implicit AUTORELEASE. But I noticed that 
> operating on that assumption, my C-program-infected with objc, seemed 
> to be growing in memory usage, watching it with top at least. So I 
> threw together the attached simple program. I'm running with debian 
> gnustep-base1 here, and gobc 3.3.3, both from the testing branch.
>
> Running the ./memLoop program, watching it under top, it does indeed 
> grow, after about 2 minutes of wall clock time, linux kills it because 
> its 500+ MB. What am I missing? The only real object allocation should 
> be the duplicate strings created over and over and over again. I even 
> tried adding the redundant AUTORELEASE(key); line, but still to no 
> avail. I compiled with -o0, thinking the compiler was inlining my 
> function, thus causing the ARP to never change stack frames and never 
> release. Still, no improvement. I'm wondering if there's something 
> wrong with stringWithFormat:.

The problem is that you are never releasing the objects you are 
creating.
When the documentation says an object is autoreleased, what it means is 
that it is set up to be released when the current autorelease pool is 
destroyed ... if you never release the autorelease pool then the 
objects in it are never released.

For normal GNUstep applications, you generally get away with ignoring 
the issue since everything is even-driven, with all code executing 
within callbacks or notifications sent by a run loop, and the run loop 
creates and destroys autorelease pools enclosing each event.

If your code is not operating in this sort of situation, you need to 
create/destroy the autorelease pools yourself.
eg. at the start and end of the addEntriesTo() function.

> ------
>
> On another note, and this is a bit of an apples-2-oranges comparison, 
> but I'd like to understand it better. Even before this program runs 
> out of memory, even when it hasn't used up much yet... VisualWorks 
> Smalltalk beats it out by nearly 2:1 speed wise. I ran both programs, 
> letting their interval counter run up to 100000, the inner loop 
> creating 100 keys and adding them to the dictionary, this makes for 
> ten million object allocations. VW did it in 55 seconds (lowly 800 Mhz 
> Duron here). memLoop (the attached objc program) took 99 seconds. The 
> VW snippet looks like:
>
> | dict interval staticValue |
> dict := Dictionary new.
> interval := 0.
> staticValue := 'BigStaticString'.
>
> [1 to: 100 do: [:i | dict at: 'key' , i printString put: staticValue].
> interval := interval + 1.
> interval \\ 10000 = 0 ifTrue: [interval out]]
>                 repeat
>
> This leads me further to believe that its the stringWithFormat: that 
> is causing the real problem, because that's the real difference. The 
> Smalltalk VM's memory usage under top didn't budge during the test 
> btw.

Probably the difference is that the objc code is using unicode strings 
and the smalltalk code is not ... that on its own would likely produce 
a factor of two difference.

Then there is the fact that you are using/misusing the autorelease 
mechanism ... it's convenient, but not the most efficient way to code.  
Rather, you should use explicit retain/release where you can (and 
certainly inside loops that are executed many times).

eg. for high performance with non-unicode strings you could do -

sprintf(buf, "key%d", i);
key = [[NSString alloc] initWithCString: buf];
[theDictionary setObject: staticString forKey: key];
RELEASE(key);


> --
> Travis Griggs
> Objologist
> Key Technology
> Achille's Heel?!?! What about "Goliath's Forehead"
>
>
>
> -----------------------------------------
> This email message is intended only for the addressee(s) and contains 
> information that may be confidential to and/or copyrighted by Key 
> Technology.  If you are not the intended recipient, please notify the 
> sender by reply email and immediately delete this email.  Use, 
> disclosure or reproduction of this email by anyone other than the 
> intended recipient(s) is strictly prohibited.  Any views expressed in 
> the email are those of the individual sender unless the sender 
> expressly states them to be the views of Key Technology.  No 
> representation is made that this email or any attachments are free of 
> viruses.  Virus scanning is recommended and is the responsibility of 
> the recipient.
> #include <Foundation/Foundation.h>
> #include <stdint.h>
> #include <unistd.h>
>
> NSString* staticString = @"BigSharedEntry";
>
> void addEntriesTo(NSMutableDictionary* theDictionary)
> {
>         int32_t i;
>
>         for(i = 0; i < 100; i++)
>         {
>                 NSString* key = [NSString stringWithFormat: @"key%d", i];
>                 AUTORELEASE(key);
>                 [theDictionary setObject: staticString forKey: key];
>         }
> }
>
> int32_t main(int32_t argc, char** argv)
> {
>         CREATE_AUTORELEASE_POOL(pool);
>         NSMutableDictionary* theDictionary = [NSMutableDictionary new];
>         
>         int32_t interval = 0;
>         while(1)
>         {
>                 addEntriesTo(theDictionary);
>                 interval++;
>                 if((interval % 10000) == 0)
>                 {
>                         printf("%u\n", interval);
>                 }
>         }
>         [pool release];
> }