Re: Alpha release: 20130307

[Jay Hacker]

This is awesome.  It works for me (as you said, plus or minus a few processes).  Thank you.

Would it be possible to use this for the  '--slowstart X' feature, that starts one job at a time, waits X seconds, and looks at the load before starting another?

On Wed, Mar 6, 2013 at 8:23 PM, Ole Tange <tange@gnu.org> wrote:

I have implemented an instant --load calculator. With this it should
be possible to tell GNU Parallel to spawn a process for each CPU that
is currently sitting idle.

On a 64 core machine this:

seq 10000 | parallel --load 20% burnP6 &
sleep 2
seq 10000 | parallel --load 100% burnMMX &
sleep 4
seq 10000 | parallel --load 100% burnBX &

will start 14 burnP6, 52 burnMMX, and 0 burnBX. If the first parallel
is killed, the last remaining parallels will spawn around 14 processes
it total.

It should also count processes that are waiting for local disks as
non-idle. So this:

ls *iso | parallel --load 1 -j0 -v cat

will only spawn 3 cat and not one for every iso file.

The biggest problem I have seen so far is that it is not very
accurate. So it may shoot a little over or under the target (2-4). But
it seems way better than the previous --load.

http://alpha.gnu.org/gnu/parallel/parallel-20130307.tar.bz2

Please give it a spin and provide feedback.


/Ole