[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: pipes that write to a file

From: Pádraig Brady
Subject: Re: pipes that write to a file
Date: Mon, 28 Jun 2021 23:32:28 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:84.0) Gecko/20100101 Thunderbird/84.0

On 27/06/2021 01:13, Bruno Haible wrote:

'envsubst' is a small program, part of GNU gettext, that is meant to be used
within pipes (it reads from stdin and writes to stdout).

Someone proposes to add an --in-place option so that it would read from a
file and write its output to the same file. [1]

However, this would not cover all use-cases. People often use 'envsubst'
and other commands, together, in a pipe, and want the output to overwrite
the input file.

So, people come to recommend the 'sponge' program from moreutils [2]:

    prog1 < FILE | prog2 | prog3 | sponge FILE

But this program has the drawback that, requiring another step in the
pipeline, the exit code of the previous command gets lost. While

    prog1 < FILE | prog2 | prog3 > $tmpfile
    mv $tmpfile FILE

allows logic that inspects the exit code prog3,

    prog1 < FILE | prog2 | prog3 | sponge FILE

does not.

What is the right solution?

- The code that uses a temporary file [3] is quite cumbersome to write,
   and is inefficient if the data size is large (since the temporary file
   will live on a different partition).
- I understand that an 'inplace' program has been rejected [4].

How about adding a  -o FILE / --output FILE  to the program prog3
('envsubst' in my case, but 'cat' and other programs would also be
candidates for it), that creates a new temporary file in the same
directory as FILE and moves it to FILE when done? This would provide
for an atomic replacement. The Gnulib module 'supersede' would
implement most of this logic; it is already used by GNU msgfmt's '-o'

In other words, move the 'sponge' logic into the last step of the pipe.

Or should the shells be adapted so that people can write

    prog1 < FILE | prog2 | prog3 >+ FILE

and that operator '>+' writes to FILE but only after the pipe has completed?



This seems like another use for an "inplace" tool.
It wasn't rejected, rather never implemented.
There were some example scripts provided,
but really it should be a separate binary
giving greater flexibility and access to system calls like
exchangedata() on osx or RENAME_EXCHANGE in renameatu() etc.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]