RE: [Gomp-discuss] parallell vs. distributed

[Scott Robert Ladd]

Lars Segerlund wrote
>   The reason I want to make a distinction is that fx. odinmp has an
> implementation which can run ontop of mpi ! ( hence distributed ), and
> thus is not sharing memory with the other 'threads'.

I've always broken the terminology of parallel computing into categories
based on architecture:

1) "Uniprocessor" applications, which may include "threads" to enable
interface concurrency and limited background processing (printing while
typing in a word processor, for example).

2) "SMP" applications, generally hosted on individual SMP workstations. This
is the focus of OpenMP.

3) "Cluster" applications, which divide work across a tightly-coupled
network of individual processing units (which include CPUs, memory, and
often local storage). This is the realm of MPI and PVM.

4) "Distributed" applications, involving loosely-couple networks of
disparate computers working asychronously. An example is address@hidden

OpenMP was designed for SMP-like applications. Note that "SMP" is not
limited to dual processor desktop machines; an example of an "SMP
Supercomputer" is the new SGI Altix.

There is a great deal of crossover among technologies and tools; for
example, some implementations of MPI can work in an "shared memory" (SMP)
environment. However, crossing problem domains is bringing fish out of their
water, so to speak: MPI *can* do SMP, but it may not be the best choice for
the job.

Just as no one programming language can effectively express all possible
softwares, I don't think one parallel architecture can address all four
categories I've described above. Combinations are certainly possible: I've
recently had several people ask me about using MPI in concert with OpenMP on
clustered SMP systems.

I still think there is reason for GNU to consider developing a "high
performance computing initiative" that encompases a GPL-free MPI and OpenMP.

In the end, I think it best if GOMP focuses on OpenMP in its native SMP
environment, leaving cluster and distributed applications to MPI and PVM.

..Scott