[Swarm-Modelling] Parameter-fitting and model comparison methods for ABM?

[Steve Railsback]

Hi-

I am trying to write up some recommendations for analysis of agent-based
models, and am not sure what to say about traditional methods for (1)
fitting model parameters to data and (2) comparing model versions by
their ability to fit data. 

In the stochastic simulation literature (e.g., Law and Kelton 1999)
there is discussion of some traditional techniques like maximum
likelihood estimation; but I've never seen such techniques (also
including Akaika's Information Criterion and Bayesian analysis) applied
to agent-based models. Instead, the few examples of parameter-fitting
I've found use a simple filtering process- simulate a billion
alternative parameter sets and identify the ones that produce acceptable
results.

Lacking a background in statistics, I can't help wondering if there are
not some fundamental obstacles to using these traditional techniques for
ABMs, due to things like: 

- The observed relation of a particular output to a particular input
potentially being extremely weird and 'noisy' even if the input has a
simple, strong effect on the agent behavior that produces the output. 

- The large number of pathways by which a system can arrive at a
particular state?

- With AIC, the analysis depends heavily on the number of parameters in
the model, which by itself could be a very interesting problem. What
really constitutes a 'parameter'?

- Does the conventional equating of degrees of freedom with # of
parameters make sense? 

Does anyone have any literature, experience, understanding??

Thanks,

Steve Railsback

Lang, Railsback & Assoc.
Arcata CA