Re: Parallelism a la make -j <n> / GNU parallel

[John Kearney]

Am 04.05.2012 21:13, schrieb Mike Frysinger:
> On Friday 04 May 2012 15:02:27 John Kearney wrote:
>> Am 04.05.2012 20:53, schrieb Mike Frysinger:
>>> On Friday 04 May 2012 13:46:32 Andreas Schwab wrote:
>>>> Mike Frysinger <vapier@gentoo.org> writes:
>>>>> i wish there was a way to use `wait` that didn't block until all the
>>>>> pids returned.  maybe a dedicated option, or a shopt to enable this,
>>>>> or a new command.
>>>>>
>>>>> for example, if i launched 10 jobs in the background, i usually want to
>>>>> wait for the first one to exit so i can queue up another one, not wait
>>>>> for all of them.
>>>> If you set -m you can trap on SIGCHLD while waiting.
>>> awesome, that's a good mitigation
>>>
>>> #!/bin/bash
>>> set -m
>>> cnt=0
>>> trap ': $(( --cnt ))' SIGCHLD
>>> for n in {0..20} ; do
>>>
>>>     (
>>>     
>>>             d=$(( RANDOM % 10 ))
>>>             echo $n sleeping $d
>>>             sleep $d
>>>     
>>>     ) &
>>>     
>>>     : $(( ++cnt ))
>>>     
>>>     if [[ ${cnt} -ge 10 ]] ; then
>>>     
>>>             echo going to wait
>>>             wait
>>>     
>>>     fi
>>>
>>> done
>>> trap - SIGCHLD
>>> wait
>>>
>>> it might be a little racy (wrt checking cnt >= 10 and then doing a wait),
>>> but this is good enough for some things.  it does lose visibility into
>>> which pids are live vs reaped, and their exit status, but i more often
>>> don't care about that ...
>> That won't work I don't think.
> seemed to work fine for me
>
>> I think you meant something more like this?
> no.  i want to sleep the parent indefinitely and fork a child asap (hence the 
> `wait`), not busy wait with a one second delay.  the `set -m` + SIGCHLD 
> interrupted the `wait` and allowed it to return.
> -mike
The functionality of the code doesn't need SIGCHLD, it still waits till
all the 10 processes are finished before starting the next lot.

it only interrupts the wait to decrement the counter.

to do what your talking about you would have to start the new subprocess
in the SIGCHLD trap.


try this out it might make it clearer what I mean

set -m
cnt=0
trap ': $(( --cnt )); echo SIGCHLD' SIGCHLD
for n in {0..20} ; do
    (
        d=$(( RANDOM % 10 ))
        echo $n sleeping $d
        sleep $d
        echo $n exiting $d
    ) &
    : $(( ++cnt ))
    if [[ ${cnt} -ge 10 ]] ; then
        echo going to wait
        wait
    fi
done
trap - SIGCHLD
wait