Re: Atomic update of destination in cp and install

[Pádraig Brady]

On 03/23/2014 10:47 AM, Markus Kuhn wrote:
> It appears the coreutils "install" tool makes no effort to update
> the destination file atomically. In other words, "install" is
> currently not a safe tool to update an existing file if the
> destination might be in use concurrently.
> 
> A destination file D can be updated atomically by
> 
>   1) first writing the new content of the destination file
>      into a temporary file T (e.g. created with mkostemp()),
>      located in the same directory as D, then
> 
>   2) rename T into D (which atomically replaces D on most
>      Unix-style file systems)
> 
> This way, concurrent users of D will only be able to open
> complete versions of either the old or the new D, but never
> see any intermediate state, such as a missing D or a half
> written one. Likewise, if the operation fails, the old D
> will remain intact.
> 
> The "install" command on Mac OSX 10.9 has an option -S to
> make the overwriting of the destination atomic.
> 
> It would be nice if the GNU variants of "cp", "install"
> and "mv" (the latter also has to copy if the destination
> is on a different device) would also offer the ability to
> update destination files atomically.
> 
> I'm not even sure why this should just be an option:
> can you think of a good reason why such atomic
> updates should not be the default behaviour?
> 
> Surely the increased robustness far outweighs the cost of
> an additional rename.

So replacement of files is tricky, and I summarized some
considerations for the general case here:
http://www.pixelbeat.org/docs/unix_file_replacement.html

install(1) will unlink() an existing destination first,
before writing the new version. That should handle atomicity
for existing processes that have the file open as they'll be
using the original linked data.  You're correct though that
new openers of the file may get a partially written version.
Now if the opener is a process trying to exec the program,
that will get ETXTBSY, so at least that case is deterministic.
(BTW some systems will give ETXTBSY if you rename() to a file of
a running process, so you'd have to have multiple renames to handle that).

The BSD functionality triggered with the -S (safe copy) option
(which is different unfortunately to the GNU -S (--suffix) option),
is probably an option due to the potential extra space requirements
for large data files for example that aren't currently open.

The GNU install --backup option implements the safe copy behavior
of the BSD -S option, but not the atomicity behavior.

There are various edge cases with different (remote) file systems
support for (atmoic) rename(), link() etc. but it would be good
for GNU install to support this atomic operation, ideally without options.

Pádraig.