Re: parallel hanging when used with pipe

[Ole Tange]

That very much looks like a bug caused by a race condition.

Unfortunately I need it more reproducible to be able to fix it.

So I would like to ask you to try to reproduce it on a virtual machine
from: http://sourceforge.net/projects/virtualboximage/files/ using
data that you are allowed to give me.


/Ole


On 3/30/13, juncus@gmail.com <juncus@gmail.com> wrote:
> I was using parallel to do blast searches on a large number of
> biological sequences.
>
> cat sequences.fasta | parallel --block 2k --recstart '>' --pipe -j 30
> "blastn -db nt -task blastn -evalue 0.00001 \
>   -outfmt '6 qseqid sseqid pident length mismatch gapopen qstart qend
> sstart send evalue bitscore ppos' \
>   -max_target_seqs 5 -query -" > results.txt
>
> It hung after a few hours.  Only about 40% of the input file had been
> processed.  No blastn processes were running, and the parallel process
> was in sleep (S) state.
>
> I killed the parallel process and got this message:
>
> parallel: SIGTERM received. No new jobs will be started.
> parallel: Waiting for these 0 jobs to finish. Send SIGTERM again to stop
> now.
>
>
> This message seems somehow pathological.  Parallel is waiting for zero
> jobs to finish.  This must surely be connected with why it was hanging
> (it didn't get the message that jobs were done, so it didn't spawn new
> jobs.)
>
> This is not happening all the time, so difficult for me to diagnose.  Any
> clues?
>
> Thanks very much,
> Owen
>
>
> $ parallel --version
> GNU parallel 20130222
> Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free
> Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> GNU parallel comes with no warranty.
>
> Web site: http://www.gnu.org/software/parallel
>
> When using GNU Parallel for a publication please cite:
>
> O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
> ;login: The USENIX Magazine, February 2011:42-47.
>
>