Re: [Swarm-Support] huge memory allocation

[Ken Cline]

--- Bill Northcott <address@hidden> wrote:

> On 21/08/2004, at 3:14 AM, Rouilly, Vincent wrote:
> > My question:
> >
> > For my perfect simulation, I would like to create a 5000x5000 Grid2d 
> > space + 3 Discrete2D planes.
> >
> > Within this space, I would like to insert up to 750000 agents .... 
> > (huge I know, and all as to be instantiated at the same time and for 
> > the same amount of time).
> >
> >  Of course I have memory allocation problems at the moment.
> >
> > How could I handle such a simulation ? Is there a way to increase the 
> > memory allocation capacity of Swarm (or my computer) ? Are they any 
> > limitations within swarm I should be aware of ? Using a cluster with 
> > Swarm is possible ?
> >
> Some more information would be helpful:
> What is you current system?    - CPU/RAM/OS etc
> How big can you go?  -  Does 500x500x3 with 75000 agents work?
> 
> That way you can see the scale of your problem.  My back of the 
> envelope feel is that there is no way you will get all that in VM on a 
> 32 bit machine.   Your talking 100 simultaneous HDTV displays with 
> every pixel calculated!
> 
> Maybe it work on a 64 bit platform, but you might need truckloads of 
> DRAM to stop the paging making everything horribly slow, and of course 
> there is the small issue of cost.  It may still be horribly slow.
> 
> Otherwise, you try to box clever and see if you can break it into 
> subproblems with looser coupling.  That is also what you would need to 
> do to run on a cluster.  Swarm has no builtin cluster support, but 
> maybe you could help add it.
> 
> Finally you just might have to lower your expectations.
> Bill Northcott

Another thought (which you've probably already considered)...

If your agents exist in only a fixed number of possible states[1]
then you could possibly minimize the memory consumption by following
the state machine design pattern.  You'd still need at least one
pointer per agent, ie the handle to its current state, and hence
you'll still likely need to lower those expectations a bit. =:-(

Are you convinced that qualitatively different dynamics will
emerge at the 10^5 scale which don't exist with fewer agents?
If not, then perhaps a semantics-based solution will work, eg
rename the Agent class as "100Agents".  Just a joke, but if your
model has some type of agent (affiliation) hierarchy then that a
natural place to begin cutting it up.  (See previous discussions
on this list related to clustering and parallelization of models.)

Ken.
________
[1] Of course, any program (that terminates) has a finite number
    of states... the tricky part is enumerating them. =8-]



=====
_________________________________________________________
 Ken Cline                             W: (443) 287-2636
 address@hidden