[bug #38092] Optionally support distributed memory machines using MPI

[anonymous]

Follow-up Comment #7, bug #38092 (project findutils):

There are several reasons why the other way around doesn't work:

1) If the final program is not MPI aware mpirun will still have to call xargs
but the second xargs would have to take its input from command line arguments,
e.g.

xargs -t --arg-file files -P 1 -n 2 mpirun -n 1 xargs -t -P 1 -n 1 

will try to run:

mpirun -n 1 xargs -t -P 1 -n 1 file1 file2 
^Cmpirun: killing job...

but since no input is given on stdin the command hangs.
On the other hand if the final program is MPI aware xargs isn't needed at all,
but in that case this logic must be added to each program separately. Without
xargs threading logic would similarly have to be added to each program
separately.

Additionally in many cases mpirun still must be the first program that is
executed so the complete command line for above would look like:

mpirun -n 1 xargs -t --arg-file files -P 1 -n 2 mpirun -n 1 xargs -t -P 2 -n
1

I'm not even sure which arguments should be given to which program in the
above but clearly this is much more difficult to use than:

mpirun -n 1000 xargs --arg-file files -P 10 -n 1

would be, which leads to:

2) not standard: MPI standard does not specify how programs using MPI should
be started and based on some experiments recursive calls to mpirun just don't
work: http://www.mpi-forum.org/docs/mpi22-report/node195.htm

On some systems there is a way to launch separate parallel programs:
aprun -n 12 ./app1 : -n 8 ./app2 : -n 32 ./app3
but again the division of work would have to be done manually by e.g. creating
separate argument files for each xargs.


3) Other reasons probably also exist but the first two are show stoppers for
the other way around anyway.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?38092>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/