Properly scheduling multi-threaded/process jobs

[Douglas A. Augusto]

Hi,

Is there a way to inform GNU Parallel that a certain job consumes more than
one slot so that the job is properly scheduled?

Consider the case where there are many OpenMP/MPI jobs that need to be
executed. Some of them use 2 threads, others launch 3 processes each, and so
forth. It seems that there is no way of telling GNU Parallel to be aware of the
fact that each job requires a different number of slots. Therefore, if we have
a job that uses 2 threads (2 slots) and another that uses 3 processes (3
slots), GNU will probably launch both of them simultaneously on a machine with,
say, only 3 cores (3 slots), instead of launching the 2-slot job first and,
after it finishes, the 3-slot job.

More specifically, I am looking for an option "--slots" like:

   parallel -j 3 --slots {3} "echo {1} sleeping processes for {2} seconds; 
mpiexec -n {1} sleep {2}" ::: 2 3 ::: 5 10 ::: 2 3

or

   parallel -j 3 --header : --slots {SLOTS} "echo {N} sleeping processes for 
{T} seconds; mpiexec -n {N} sleep {T}" ::: N 2 3 ::: T 5 10 ::: SLOTS 2 3

such that the values from the argument labeled "3" (or "SLOTS") specify the
number of slots taken by the corresponding jobs:

   (will take 2 slots) 2 sleeping processes for 5 seconds
   (will take 3 slots) 3 sleeping processes for 5 seconds
   (will take 2 slots) 2 sleeping processes for 10 seconds
   (will take 3 slots) 3 sleeping processes for 10 seconds

Note that (1) the argument "SLOTS" should not be combined with the other
arguments; and (2) in this example, with -j 3, GNU Parallel should launch only
one job at a time.

-- 
Douglas A. Augusto