Re: Enhancement request for tee - please add the option to not quit on SIGPIPE when someother files are still opened

[Jirka Hladky]

Hi all,

I would like to have the example of process redirection similar to this one 

tee -p </dev/zero >(head -c1 | wc -c ) > >(head -c10M | wc -c)

in the tee's manual page. What do you think about it and what is the process to request it? 

Thanks a lot

Jirka

On Sun, Nov 22, 2015 at 9:34 PM, Pádraig Brady <address@hidden> wrote:

On 22/11/15 20:13, Bob Proulx wrote:
> Pádraig Brady wrote:
>> +If you want to further process the output from process substitutions,
>> +and those processes write atomically (i.e. write less than the system's
>> +PIPE_BUF size at a time), that's possible with a construct like:
>
> When I read this I read a couple of the sections and it causes me
> confusion.  Let me break down my reading of this into parts to
> communicate what I am reading.
>
>> +If you want to further process the output from process substitutions,
>> +and those processes write atomically
>
> I don't understand the use of "atomically" here.  Use of it causes me
> to think of operations that cannot be split into smaller parts and
> therefore are completely one way before the action or completely
> another way after the action.  I am probably missing something but
> that doesn't seem to be the case here.  What action is atomic?  Is
> that really what was meant to be communicated?
>
>> +If you want to further process the output from process substitutions,
>
> I have to believe the intention was something regarding "further
> process the output" but I fail to understand the intention.  Surely
> the output of the process substitutions are just piped along
> normally.  I am sure there is a subtle point here that I am missing.
>
>> ... (i.e. write less than the system's
>> +PIPE_BUF size at a time), that's possible with a construct like:
>
> When I read this "that is, in other words" clarification indicating
> smaller write blocks it makes me wonder what is the atomic operation
> in the tee output.  Is there really an atomic operation?  Or is it
> simply passing on the data blocks?  That wouldn't really be atomic if
> I were to read(0,buf,1) the data, right?
>
>> +@example
>> +tardir=your-pkg-M.N
>> +tar chof - "$tardir" \
>> +  | tee >(md5sum --tag) > >(sha256sum --tag) \
>> +  | sort | gpg --clearsign > your-pkg-M.N.tar.sig
>> +@end example
>
> I don't see how the example illustrates writes less than PIPE_BUF in
> size specifically.

Basically I was trying to say that this is not a general solution.
I.E. there are restrictions on what the process substitutions can output
so that consumers of the _combined_ output can make sense of it.
Specifically the process substitutions must write <= PIPE_BUF bytes
at a time to avoid intermingled data.  Now the writes() themselves
are async, so generally you also want to sort the combined output
before processing, which means the process substitutions would
need to be writing a sortable --tag.  This overlaps with the
Decorate Sort Undecorate pattern mentioned elsewhere in the document.

The other notes about waiting for process substitutions with
a further pipe, would be best in the bash manual or Greg Wooledge's wiki etc.

cheers,
Pádraig.