Re: why generate output to tmp file and mv to actual file?

[John Calcote]

Thanks everyone - great responses! Answers all my questions.

On Fri, Jun 1, 2018 at 1:00 PM, Eric Blake <address@hidden> wrote:

> On 06/01/2018 01:45 PM, John Calcote wrote:
>
>> I recently ran across some sample code in section 19.4 of the Autoconf
>> manual (modified slightly to reduce example):
>>
>> $(TESTSUITE): $(srcdir)/testsuite.at $(srcdir)/package.m4
>>          autom4te --language=autotest -I '$(srcdir)' -o address@hidden 
>> address@hidden
>>          mv address@hidden $@
>>
>> This question isn't about autotest, but rather about the two commands in
>> this rule - why generate the output into address@hidden and then mv 
>> address@hidden into
>> address@hidden
>> Is there some power mv has over autom4te that allows it better access to
>> the target under some conditions?
>>
>
> Atomicity.  autom4te generates its output piecemeal (while it is running,
> you can see an incomplete version of address@hidden); such an incomplete file 
> will
> probably fail miserably but in an unpredictable manner when run as a shell
> script.  As an example of the damage possible with an incomplete script?
> Suppose you are generating a shell script that states:
>
> rm -rf *.tmp
>
> but due to buffering constraints on stdio, the kernel happens to have
> flushed "rm -rf *" to disk (perhaps because it hit a 64k boundary, or so),
> but still has ".tmp" buffered up for a later write.  In the common case,
> executing an incomplete script will hopefully cause a syntax error; but in
> cases like I just described, it can cause data loss.
>
> Thus, we generate into a temporary file (which no one will read in an
> intermediate state), then use mv to the final location (which has atomic
> semantics - anyone open()ing the file will see either the old inode which
> was presumably complete, or the new inode that we just completed), so that
> the target file is always a valid and complete file for reading or
> execution.
>
> In fact, the notion of generate to a temporary then move into place is a
> rather common idiom.
>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3266
> Virtualization:  qemu.org | libvirt.org
>