Re: [certi-dev] Re: CERTI-Devel Digest, Vol 28, Issue 5

[Eric Noulard]

2008/4/22, Pierre SIRON <address@hidden>:
>
>  Hello Eric,
>  mon cher voisin de l'autre côté du couloir,
>  I think that the discussion is more open between MPI and HLA.

Yes you are right it is, I didn't want to "close" it telling
MPI is for scientific computation and HLA is not.
I was just wanting to know more about Christian motivation about
using HLA.

>  I have directed an internship on the subject of scientific computation with
> HLA
>  (parallel resolution of a linear system, Kathrin Quince, 2006).
>  We had also the problem of a huge matrix transfer, our solution has been
>  to transmit the matrix by blocks. It was not very efficient, but, after
> that,  the following computations were correct.

Yes I can imagine that.
That's a point where HLA and MPI differs.
MPI primary goal is efficiency for data exchange and not ease of use
and dynamicity.
Even if MPI-2 did come with many API for more dynamic communication,
as far as I know we are not near close to HLA flexibility in this area.

HLA favorize ease of use (in some sense) and dynamicity
(federate may join, participate and leave dynamically the federation).

On the contrary
MPI global communication (broadcast, multicast, reduction, barrier) are builtin
and for example reduction/barrier are usually done _VERY_ efficiently
using log(n) algorithm like butterfly (or better if the network topology enables
a better algorithm)

We are not too far away from HLA _needs_ which may need the barrier
in order to implement conservative time synchronization algorithm
(see Fujimoto's Parallel and Distributed Simu. Systems pp. 68--70

>  Why to do scientific computation with HLA ?
>  To avoid a gateway (overhead) between various execution environments
>  (burden of mastering and deploying many tools).

I'm pretty sure that for huge data and "not too smart" UAV HLA service
implementation MPI will outperform HLA even with a gateway overhead.
(we need benchmark for this though)

>  Do we have a lot of applications which integrate scientific computation
>  in a distributed event-based simulation ?
>  I am thinking to the simulation of avionics systems which could require
>  more elaborate models of the plane and environment physics.

Yes I'm really sure there are [complex] scientific app which would
benefit from HLA.

>  Christian could add more examples here ?
>
>  Are the HLA services appropriate to write parallel programs ?
>  A first answer, we can write such programs with HLA.

Yes we do.
But my point is _efficiency_, current HLA services (at least HLA 1.3
and IEEE-1516 I _currently aware of_) do not offer
efficient data exchange services usually needed by scientific computation:

- periodic exchange
- efficient broadcast in a group/sub-group
- barrier
- reduction

>  Are the data management services appropriate ?
>  We can express point to point communication (a single publisher
>  and a single subscriber).
>  We can express one-to-many communication (data distribution).
>  We can express many-to-one commmunication (reduction operation
>  but without the power of an binary tree communication scheme).

You forget Many-to-Many, since usual reduction objective
is to obtain the same reduced value for every stakeholder.

Yes I totally agree, expressivness is there, we lack efficiency.
Regarding x-to-Many or Many-to-x we I think a doable
implementation optimization would be to implement
binary-tree/butterfly/.... algorithm.

The next level optimization would be to find a mean to
avoid to transmit meta-data.
Current CERTI implementation of UAV/RAV message contains
meta-data for EACH exchange:
    - at least 5 inrt32 value (type, exception, federate, federation, object)
    - then values

For high rate data transfer of relatively low volume
value this is a performance killer.
For high volume exchange this may be less important.


>  The DDM services can be used to indicate the receiver of some data.
>
>  Are the time management services appropriate ?
>  A general parallel application requires a complex synchronization
>  of tasks. These services can be useful.
>  Even a fork-join mechanism can be easily (but not intituively
>  for a Fortran programmer) written.
>
>  How can we explain the superiority of MPI for the data transfers ?
>  a) Lower overhead of the MPI layer (latency) ?

YES.

>  b) Execution of MPI applications on efficient architectures
>  (processors and networks) ?

True two. Plus the fact that in MPI each "MPI-federate" agree
about the communication scheme "by design".
In HLA the RTI is responsible for communication optimization.

>  c) A lot of data transfer optimizations ?
>  (even in the case of MPI above TCP ?)

YES I think so, 10-years and more of open MPI implementation
(mpich, LAM, Open MPI) have matured to include many optimization.

>  What are the MPI optimizations that cannot be included
>  in a RTI implementation ?

None, beside the reduction which is not expressed directly
by HLA API so that RTI would have to "guess" which may not be possible.

>  In the case of CERTI, we could study the direct connection
>  between RTIA for some objects (with a new transport attribute).

I think we should:
   1) First control and reduce precisely the size of Messages (network
and local)
   2) Implement reduction algorithm for synchronization
   3) Try aggregate UAV communications and/or implement
       a distributed RTIG in order to be able to implement efficient
       x-to-many / many-to-x algorithm without centralized RTIG overhead.
   3) Examine how to propose HLA extension for user-specified "global
communication".

-- 
Erk