[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: input buffering
From: |
Ole Tange |
Subject: |
Re: input buffering |
Date: |
Thu, 7 Dec 2017 21:56:33 +0100 |
On Wed, Nov 15, 2017 at 2:05 PM, Ryan Berckmans
<ryan.berckmans@gmail.com> wrote:
> When using GNU parallel 20160522 with bash 3.2.57(1)-release, my use case is
> of the form
>
> slow-computation | parallel --pipe sh -c "quick-computation1 |
> quick-computation2"
>
> In this case, slow-computation outputs one job per line of stdout. Each line
> of input is unbuffered and immediately available to parallel.
>
> parallel seems to wait for input EOF before starting the first job. Is it
> possible to make parallel eagerly start jobs as they become available?
GNU Parallel _does_ start the jobs immediately. What GNU Parallel does
_not_ is to give you the output.
This is a bug, which is mentioned here:
https://www.gnu.org/software/parallel/man.html#EXAMPLE:-GNU-Parallel-as-queue-system-batch-manager
If you make a patch that fixes this bug, I will be very interested.
/Ole
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: input buffering,
Ole Tange <=