Re: Multiple sqlworkers

[Ole Tange]

On Mon, Apr 6, 2020 at 5:22 AM Thomas Bereknyei <tomberek@gmail.com> wrote:

> Running some experiments with the sqlmaster/sqlworker system shows
> that running multiple independent workers results in many jobs being
> done multiple times. This is noted in the manual as
>
> "If you have more than one --sqlworker jobs may be run more than once."
>
> In simple testing, this is not an occasional thing, but results in
> nearly all jobs being performed N times.

So I tested:

seq 100 | parallel --sqlmaster sqlite3:///fisk/fisk 'sleep 4;echo {}'
(
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
wait
)  | wc -l

This gives 100..103 on my machine. Thus 0..3 jobs were run twice.

So I cannot reproduce the error you see.

That is not to say that you have not found a bug, but I cannot
reproduce it on my system.


/Ole