[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: SV: Wish: checksumming *sum filter
From: |
Pádraig Brady |
Subject: |
Re: SV: Wish: checksumming *sum filter |
Date: |
Fri, 10 May 2019 07:44:21 -0700 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 |
On 10/05/19 06:13, Ole Tange wrote:
> Pádraig wrote:
>
> :
>
>> The fact that *sum would need to consume/buffer all the input would mean
>> that the parallelism
>> from the rest of the pipe is lost (well I suppose the process startup
>> overhead is parallelized).
>
> Just like sort/wc and *sum if used in a pipe today:
>
> cat file | sha256sum | ...
Right.
*sum, and wc are simple as they reduce the input to a bounded size.
sort is not simple internally and must deal with
writing out temp files.
>> I.E. it's functionally equivalent to:
>>
>> Get_untrused_input > /tmp/blah
>> sha256sum -c <(echo "$chksum /tmp/blah") &&
>> ...
>> rm /tmp/blah
>
> Yes, except you can do it on a read-only file system and in a shell that does
> not support <().
In the general case write access would probably be required, like with sort
above.
Note sort uses $TMPDIR if set to identify a writable file system.
Re support for <(), one could use another temp file for that.
cheers,
Pádraig