Re: [Swarm-Support] memory problems

[Paul E Johnson]

I agree that the drop on the EZDistribution object should give all the 
memory back, but I don't think most people have tried to use that class 
in this way and so, if there is a memory leak, we have not been alerted 
to it yet.  I found a memory leak, which I still have not solved, in a 
method that used probes to retrieve double-valued variables, and if you 
are doing that, I expect you could see the same issue.

I think that you should redesign your class so that each agent only 
creates one EZDistribution as an instance variable and then use it over 
and over.  There is no need to create and drop at every time.

The other (better!) possibility is you should find a better way to 
collect the information you want.  For example, are you just trying to 
get a tabulation of how many agents there are in each group?  If so, 
consider using a method like (off the top of my head, excuse typos)


suppose there is an instance variable

int tabulator[6]; //need 6 because your example shows values from 0 to 5
                  //not 5 binCount as you have;

and somewhere you initialize that to 0's

Then call a method that does the tabulation. This will be much faster 
than creating a big old EZDistribution.

- tabulateDistribution
{
   id index = [group begin];
   id anObject;

   for (anObject=[index next]; [index getLoc == Member]; 
anObject=[index next])
       {
         int agentValue = [anObject getTypology];
         tabulator[agentValue]++;
        }
}


Or, if you just want to calculate an average or standard deviation, you 
can do that more easily with EZGraph, I can show you how if you need.

Sorry, gotta run to a meeting.


Here's my old email about the problem:

One of the guys who ran the Artificial Stock Market for a long long time 
said it grew from 6meg of RAM to  300meg. I said you are crazy, but he 
was right.

After trying every thing I know (dmalloc, etc), I could not find a leak 
in ASM-2.2.1, but I have a strong hint that it is in Swarm itself.

Here is the source code.where you can see all this for yourself:
http://artstkmkt.sourceforge.net/updates/ASM-2.2.1.tar.gz

There are some functions that I've used to get parameters. Here they 
are, from BFParams.m:

id
makeProbe (id obj, const char *ivarName)
{
  id probe = [VarProbe createBegin: [obj getZone]];
  [probe setProbedClass: [obj getClass]];
  [probe setProbedVariable: ivarName];
  return [probe createEnd];
}

double
getDouble (id obj, const char *ivarName)
{
  id probe = makeProbe (obj, ivarName);
  double ret = [probe probeAsDouble: obj];
  [probe drop];
  return ret;
}

int
getInt (id obj, const char *ivarName)
{
  id probe = makeProbe (obj, ivarName);
  int ret = [probe probeAsInt: obj];
  [probe drop];
  return ret;
}

There are many usages of getInt() and in BFagent.m, I had 2 usages of 
the getDouble().


After experimentation, I found the memory leak is caused by the use of 
getDouble(), but not getInt().

You can check for yourself. In BFagent.m, change

 divisor = getDouble(privateParams,"lambda")*forecastvar;
To:
  divisor = 0.3*forecastvar;

Also change:
  variance = getDouble(privateParams, "initvar");
To:
  variance = 4.00;

After that, the memory usage of the ASM stays steady for thousands and 
thousands of steps.

That makes me think that the getInt() (the integer probe) is OK, but the 
probe for a double has some issue.

So, where?


There is still a memory leak if you just do like so:


double
getDouble (id obj, const char *ivarName)
{
 id probe = makeProbe (obj, ivarName);
 //  double ret = [probe probeAsDouble: obj];
  double ret = .5;
 [probe drop];
  return ret;
}

Hence, it is not probeAsDouble: that triggers the leak. I reason 
therefore the problem has to be in the creation of the probe for a 
double valued ivar. maybe in VarProbe.m, the allocated memory for the 
IVAR "probedVariable" is not getting dropped. But I don't see why it 
would not be.

Aside from gazing at the source code, where I find nothing wrong, I 
don't know how to attack this.

If you advise me, I will try.

--
Paul E. Johnson                       email: address@hidden
Marcello wrote:
> Hi,
> I have a swarm piece in Object C. It does everything using about 1% of
> my 512mg of memory. Now I added one EZDistribution to each Agent (which
> is a SwarmObject). The idea is that each agent creates a list of other
> agents, passes it through a EZDistribution to get some statistics and
> drop everything right after (see code). The problem is that when this
> piece of code is in the program, the program sucks up a large amount of
> memory, something like 5% of the memory each 100 calls to agents
> (according to "top"). 
> After some checks I know is the EZDistribution that creates the memory
> problem, but I do not know how to solve it. Shouldn't my code create the
> EZobject and dispose of it each time? 
> Thanks for any help youcan provide
> marcello
>
> id a;
> // here I create a "group" as a List. this does not create memory
> problem.
>
>
>  a=[EZDistribution createBegin: [self getZone]];
>   [a setTitle: "stats"];
>   [a setGraphics:0];
>   [a setBinCount: 5];
>   [a setLowerBound: 0];
>   [a setUpperBound: 5];
>   [a setCollection: group ];
>   [a setProbedSelector: M(getTypology) ];
>   [a createEnd];
>   [a update];
>   [a drop ];  //shouldn't this line dispose of "a"?
>   [group drop];
>
> thanks alot
> mc



--
Paul E. Johnson                       email: address@hidden
Dept. of Political Science            http://lark.cc.ku.edu/~pauljohn
1541 Lilac Lane, Rm 504
University of Kansas                  Office: (785) 864-9086
Lawrence, Kansas 66044-3177           FAX: (785) 864-5700