bug#34713: Files vanishing when moving to different FS

[Bob Proulx]

tags 34713 notabug
close 34713
thanks

Hello Christoph,

Christoph Michelbach wrote:
> To reproduce this bug, you need two different file systems. Adapt the
> paths to fit your system.

Thank you for making this bug report.  However what you are
experiencing is due to the race condition created by the non-atomic
nature of copying files from one file system to another, removing
files, and renaming files.  This is not a bug in mv but is an
intrinsic behavior.

> Set the experimental file structure up like this:
> 
> mkdir exp
> cd exp
> mkdir a
> cd a
> touch a{1..100000}
> cd ..
> mkdir b
> cd b
> touch b{1..10000}
> mkdir /t/ae # /t has to be on a different file system

Thank you for the very nice test case.

> Then have two terminals open in the exp directory created above.

This is a clue to the nature of the problem being a race condition.
It describes simultaneous parallel processes.

> In one, execute this command:
> 
> mv a /t/ae

Because /t is on a different file system mv cannot simply rename the
files but must perform the action in two steps.  It copies the file
from source to destination.  It removes source file.  This is
documented in the mv with:

    'mv' can move any type of file from one file system to another.
    Prior to version '4.0' of the fileutils, 'mv' could move only
    regular files between file systems.  For example, now 'mv' can
    move an entire directory hierarchy including special device files
    from one partition to another.  It first uses some of the same
    code that's used by 'cp -a' to copy the requested directories and
    files, then (assuming the copy succeeded) it removes the
    originals.  If the copy fails, then the part that was copied to
    the destination partition is removed.  If you were to copy three
    directories from one partition to another and the copy of the
    first directory succeeded, but the second didn't, the first would
    be left on the destination partition and the second and third
    would be left on the original partition.

The mv a /t/ae action is similar to cp -a a t/ae && rm -r a when the
action is successful.  Similar because there are two steps happening.
A first step with the copy and a second step with the removal and
there is a time skew between those actions.

> In the other, execute this one while the one in the first terminal
> still is running (hence the large number of files so you have time to
> do this):
> 
> mv b/* a

This is the second part of the race condition.  It it moving files
into the a directory at the same time that files are being copied out
of the directory and the directory itself being removed.

> You will end up with 100 000 files in /t/ae. The 10 000 files beginning
> with the letter b will be gone.

Look at the two actions explicitly:

Process 1:
  cp -a a /t/ae
  rm -rf a

Process 2:
  mv b/* a

Now it is more obvious that as soon as the first process copy finishes
that it will remove the source location, that is having files moved
into it by the second process, that the directory will be deleted by
the first process.

Does that make it easier to understand what is happening?

The copy and remove two actions do not occur when both the source and
destination are on the same file system.  In that case the file can be
renamed atomically without doing a copy.  But when the action is
across two file systems this is not possible and it is simulated (or
perhaps emulated) by the copy and remove two step action.

Whenever tasks are moving files into and out of the same directory at
the same time this is always something to be aware of regardless
because they may be an overlap of actions in that directory.

In this particular example the problem can be avoided by renaming "a"
first and then transfering the files to the other file system.
Because it was removed then the second process can create it without
collision.  Something like this pseudo-code.  However to safely use
temporary file names will require more code than this.  This is simply
for illustration purposes.

Process 1:
  mv a tmpdirname
  cp -a tmpdirname /t/ae
  rm -rf tmpdirname

Process 2:
  mkdir a
  mv b/* a

I hope this helps.

Since this is not a bug in mv I am going to close the ticket in our
bug database.  But please we would like to hear back from you in this
ticket for any further discussion.

Bob