Re: starvation with "--load" on multicore (was: parallel stops working for no obvious reason)

[Ole Tange]

On Tue, Apr 3, 2012 at 1:49 PM, Ole Tange <tange@binf.ku.dk> wrote:
> On Tue, Apr 3, 2012 at 2:35 AM, Ole Tange <ole@tange.dk> wrote:
>> On Mon, Apr 2, 2012 at 10:59 AM, Thomas Sattler
>> <sattler@med.uni-frankfurt.de> wrote:

>> export PARALLEL="--load 100% --verbose"
>> echo PARALLEL=$PARALLEL
>>
>> for i in $(seq 2 10); do
>>  i2=$[i*i]
>>  echo creating $i2 files:
>>  seq $i2 | parallel -j100 -DX echo {}
>>  echo
>> done
:
> I have now solved the issue. Unfortunately the current solution breaks
> a lot of other stuff, but there is no need for futher debugging.

The solution is now in git. As the solution required some "rewiring"
of process flow, it would be good if you test out this version if you
run a lot of jobs or use --pipe as either of these may cause issues.


/Ole