Re: [certi-dev] Re: CERTI-Devel Digest, Vol 28, Issue 5

[Eric NOULARD]

Le Fri, 25 Apr 2008 14:02:11 +0200,
Christian Stenzel <address@hidden> a écrit :

> > Why to do scientific computation with HLA ?
> > To avoid a gateway (overhead) between various execution environments
> > (burden of mastering and deploying many tools).
> >
> > Do we have a lot of applications which integrate scientific
> > computation in a distributed event-based simulation ?
> > I am thinking to the simulation of avionics systems which could
> > require more elaborate models of the plane and environment physics.
> > Christian could add more examples here ?
> >   
> The general term "scientific computation" includes a wide field. Wiki
> says: Computational science (or scientific computing) is the field of
> study concerned with
> constructing mathematical models 
> <http://en.wikipedia.org/wiki/Mathematical_model> and numerical
> solution techniques and using
> computers to analyze and solve scientific 
> <http://en.wikipedia.org/wiki/Scientific>, social scientific 
> <http://en.wikipedia.org/wiki/Social_science> and engineering 
> <http://en.wikipedia.org/wiki/Engineering> problems.
> In practical use, it is typically the application of computer
> simulation <http://en.wikipedia.org/wiki/Computer_simulation> and
> other forms of computation <http://en.wikipedia.org/wiki/Computation>
> to problems in various scientific disciplines.
> 
> My opinion is that also many event-based simulations refer to the
> field of scientific computations.
> Event-based simulations are often a very high abstraction
> of real systems. In some parts of the engineering community
> the modeling of dynamic systems via differental equations or partial 
> differental equations is more common.

This is more common for example when the solution of your problem
is governed by physics equation 
(Electromagnetics, Fluid Mechanics, Acoustics, ...)
over a large domain (a plane, a car, the sea, the earth ...)

The size of the data coming from the discretized equations
you manipulate may be huge see example at
http://math.nist.gov/MatrixMarket/.

Those problems did (?does?) requires High Performance Computing
http://en.wikipedia.org/wiki/High_Performance_Computing
using either SHM or MP parallel computation model.


> 
> There are two common ways to combine continous and discrete models to 
> so-called hybrid models.
> One way is to detect events in a continous simulation, a good
> standard example is "the bouncing ball". 

[...]
The example is clear, agreed.

 
> Besides Amdahl's law the ratio between communication effort and 
> processing effort is very important. 

Yes that true and most of High. Perf. Comp. application I know
do care about data locality in order to avoid communication
and when possible overlapping local computation with asynchronous
communication is done.

Overlapping local computation with asynchronous communication is 
something done explicitly with Message Passing API (MPI, PVM, BSP)
even if one may thought that should mean that local computation
is linked to "lookahead" notion of HLA.

Lookahead may be viewed (locally) as "go ahead compute" you do not
depend or put dependency on others.

> All problems with a small granularity value are appropriate to
> compute in a distributed fashion (PDES: LPs have a big
> lookahead). Probably we will get a speedup. This means that such an 
> application can run on top of nearly any middleware
> (see today's grid applications). When the communication effort
> increases the efficiency of the underlying communication
> infrastructure becomes more and more important (as Eric already
> mentioned).

In fact I think that when you lookahead goes down to zero
you may turn your PDES in a time-triggered simulation for
which you may gain performance because you don't need the TM
overhead anymore. (I do exagerate on purpose).
 
> My opinion is that HLA can be used for parallel processing and in 
> particular for PDES. The communication
> efficiency depends on the used RTI and that's why on implementation
> details.

I do totally agree.
The RTI may makes the parallel processing efficient
but in order to compete with equivalent MPI implementation
we may need more information than a local lookahead
Would the federate produce periodic data?
   The RTI does not know that

Is it high volume or low volume?
   When a federate publish attribute the RTI does not even know
   the size of the attribute (HLA 1.3) since attribute are untyped.
   The RTI discovers the size on UAV
 
> On the one hand you can use message-passing systems, on the other hand
> SHM based systems. Historically mp-based approaches are
> applied on distributed memory architectures (e.g. beowulf-clusters)
> whereas SHM is used in close coupled memory architectures. Today, the
> usage of a specific programming scheme for a specific architecture is
> no more required,

Yes but we inherit two similar "standard"
The one for Message Passing is called MPI
(http://www.mpi-forum.org/, http://www.open-mpi.org/), 
The one for SHM is called OpenMP (http://www.openmp.org/), 

Note that with MPI-2 you get one-sided communication
(MPI_Put/MPI_Get) which is just "similar" to HLA 
UpdateAttributeValue/QueryAttributeValue

> VSHM based on mp-systems are thinkable:

And they do exists for a long time 
(see for example: http://citeseer.ist.psu.edu/raina92virtual.html)
either as software layer or even at hardware level
with NUMA machines
(http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access)

But even on NUMA HPC people do care about data-locality.


> 
> All mp-based approaches have in common, that the communication between
> the sender and the receiver is explicit. That means I have to specify
> the receiver of my message, e.g. PVM does it by TIDs. The
> synchronization is always implicit through e.g. blocked receive
> operations etc..

I do not agree.
Sometimes you always do asynchronous send/receive (MPI_ISend/MPI_IRecv)
and then explicitly synchronize using a barrier (MPI_Barrier).

> On the other side SHM approaches have primitives for an explicit
> synchronization (mutex, semaphor), the communication is
> implicit (changing the value of a shared variable).

Yes I agree.
But once you have SHM you may sometimes want "purely local" data
see http://en.wikipedia.org/wiki/Thread_Local_Storage

> 
> In my opinion, HLA do not really fit in one of these categories. The 
> communication scheme in HLA is implicit mainly caused through the
> declarationmanagement services.
> A sender or publisher of information does not know who will receive
> that information.

This true for MPI "point-to-point" communication 
(see Chapter 3 of http://www.mpi-forum.org/docs/mpi1-report.pdf)

But this is not with "Collective" communication
(see Chapter 4 of http://www.mpi-forum.org/docs/mpi1-report.pdf)
Using MPI_BCast you
send to a "Communicator" which may be viewed as an "HLA Region"
the sender does not have to know who belongs to the communicator.

Honestly, I do exagerate on this, the thing I am trying to tell
is that those Programming Models (HLA, SHM, MPI) do share concepts
such that I think you may well implement an HLA RTI over MPI.

Some parts may be tricky but you may do it.


> A subscriber also does not know who produced that information. In the 
> view of an HLA
> application using TM services the synchronization is
> also implicit because sync primitives are provided through the RTI.

That perfectly true for sync.

> 
> I would not say that we have a new programming paradigm for parallel 
> applications but HLA is a little bit special :).

It is not that much different and I think differences blurs when
you look all message passing/shm/HLA in the eye.



> Eric mentioned barrier synchronizations. In the field of PDES barriers
> are also used in some second generation conservative sync algortihms 
> (synchronous sync algorithms).

I was precisely talking about that which I read in Fujimoto's book.

> I suppose that these algorithms can also be applied to an HLA 
> implementation. 

Yes and I'm interesting in studying/doing this.


> At time CERTI uses the CMB-algorithm (nullmessage) for conservative 
> synchronizations. The synchronous
> sync algortihm can natively handle zero lookahead. Probably it could 
> handle small lookahead federates much more faster than CMB.

That exactly what I meant by near to "time-triggered" simulation.
 
> Hope that someone find this discussion useful. For me it is a good 
> possibility to order my own thoughts :).

I find it interesting thank you for your thought.


--
Erk