bug#9500: [PATCH]: use posix_fallocate where supported

[Pádraig Brady]

-------- Original Message --------
Subject: bug#9500: [PATCH]: use posix_fallocate where supported
Date: Fri, 25 Nov 2011 10:35:34 +0100
From: Goswin von Brederlow <address@hidden>
To: Padraig Brady <address@hidden>

Hi,

On 09/14/2011 03:46 PM, Pádraig Brady wrote:

> I thought a little about this today.
>
> fallocate() is a feature to quickly allocate space in a file system.
> It's useful for 3 things as far as I can see:
>
>   1. Improved file layout for subsequent access
>   2. Immediate indication of ENOSPC
>   3. Efficient writing of NUL portions
>
> Note 1. is somewhat moot with newer file systems that do "delayed allocation".
> So what do we need to consider when using fallocate on the destination file?
> Considering just cp for the moment, its inputs impacting this are the options:

Not every filesystem does delayed allocation and delayed allocation only
works for file considerably smaller than the amount of cache.


Also a note on sparse files:

If you have fiemap then you can fallocate the used chunks in the
destination. But there might be zero filled blocks that would become
unallocated with current cp. Those would remain allocated (but not zero
filles, which might be a difference) in the destination. Same if the
file is fully allocated but has empty blocks. So a bit more has to be
done.

So here is what I would do for sparse files:

- get fiemap of file, if it fails the fiemap is just one chunk for the
  whole file

- For each chunk in the fiemap:
  1 read block from source
  2 if (block == ZERO) goto 7
  3 fallocate(offset, end of chunk)
  4 write block
  5 read block from source
  6 if (block != ZERO) goto 4
  7 fruncate(offset)
  8 read block from source
  9 if (block == ZERO) goto 8
 10 goto 3

Failure of fallocate should probably be ignored to preserve current
behaviour and because if empty blocks are found less space will be used
and might actually fit.

The above would result in the same allocation for the file as current cp
creates for sparse files.

If you want to copy files while preserving the allocation of the source
then I think cp needs a new --sparse=source mode. In that mode setp 7
would change from ftruncate() to write(ZERO) to preserve the empty
blocks and you would only fallocate once per chunk for the full chunk.

Hope that helps.

MfG
        Goswin

PS: Feel free to forward to the mailinglist, gmane doesn't show its
address or I would cc.