Re: parallel + blast + LSF

[Ole Tange]

On Fri, May 8, 2015 at 2:42 PM, Giuseppe Aprea <giuseppe.aprea@gmail.com> wrote:

> I read again the first part of my message about semaphores and it doesn't
> really express what I meant. Taking the two commands (or any other couple
> you think is appropriate to show the differences):
>
> sem --fg --no-notice -j 2 echo ::: 1 2 3 4
> parallel --no-notice -j 2 echo ::: 1 2 3 4

Note that ::: is ignored for sem. If you read the SYNOPSIS of the man
page, you will see that sem does not read arguments from :::, ::::,
-a, or standard input.

Instead you have to give the full command to sem:

  sem --fg -j 2 echo 1
  sem --fg -j 2 echo 2
  sem --fg -j 2 echo 3
  sem --fg -j 2 echo 4

A more realistic example is:

  sem -j 2 'sleep 1;echo 1 finished';   echo sem 1 exited
  sem -j 2 'sleep 2;echo 2 finished';   echo sem 2 exited
  sem -j 2 'sleep 3;echo 3 finished';   echo sem 3 exited
  sem -j 2 'sleep 4;echo 4 finished';   echo sem 4 exited
  sem --wait; echo sem --wait done

In toilet analogy this uses 2 toilets (-j 2). GNU sem takes '1' to a
toilet, and exits immediately. While '1' is sleeping, another GNU sem
takes '2' to a toilet, and exits immediately.

While '1' and '2' are sleeping, another GNU sem waits for a free
toilet. When '1' finishes, a toilet becomes available, and this GNU
sem stops waiting, and takes '3' to a toilet, and exits immediately.

While '2' and '3' are sleeping, another GNU sem waits for a free
toilet.  When '2' finishes, a toilet becomes available, and this GNU
sem stops waiting, and takes '4' to a toilet, and exits immediately.

Finally another GNU sem waits for all toilets to become free.

- o -

One of the situations in which this is useful is in 'crontab': If you
have a command that runs every minute, but that it will fail, if two
are running at the same time, you need a way to postpone the next run
if one instance takes longer than 1 minute.

If you cannot allow a certain command to be run twice at the same
time, then you can simply prepend 'sem --id myid' to the command, and
the second instance will not be allowed to run before the first is
finished.

- o -

If the concept of a semaphore is still hazy please read:
http://en.wikipedia.org/wiki/Semaphore_%28programming%29


/Ole