Re: [Bug-tar] [PATCH] Remove nonportable check for files containing only zeroes

[Andreas Dilger]

On Jan 23, 2018, at 2:49 PM, Mark H Weaver <address@hidden> wrote:
> 
> Joerg Schilling <address@hidden> writes:
> 
>> Mark H Weaver <address@hidden> wrote:
>> 
>>>> Now many bytes have been written past the hole?
>>> 
>>> Did you read my entire message?  The answer to your question was just a
>>> few lines beyond the excerpt that you quoted above.  I wrote:
>>> 
>>>>> Yes, on Btrfs I reliably see (st_blocks == 0) on a recently written,
>>>>> mostly sparse file with size > 8G, using linux-libre-4.14.14.  More
>>>>> specifically, the "storing sparse files > 8G" test in tar's test suite
>>>>> reliably fails on my system:
>>>>> 
>>>>>  140: storing sparse files > 8G                       FAILED 
>>>>> (sparse03.at:29)
>>>>> 
>>>>> The test creates a sparse file consisting of 8 gibibytes of hole
>>>>> followed by 512 bytes of 'A's at the end.  [...]
>> 
>> Sorry, I did not see this.
>> 
>> Well then it would be of interest whether btrfs is able to keep 512 bytes of
>> data in the meta data space.
> 
> A hypothetical file system might store the entire file contents
> *compressed* in the inode.  Large holes might be stored quite compactly
> in such a scheme.  It's not hard to imagine a realistic compression
> method that could successfully store 8 gigibytes of zeroes followed by
> 512 bytes of 'A's, completely within the inode.
> 
> In such a file system, even in the absence of delayed allocation,
> (st_blocks == 0) would be justifiable, no?

Depends...  ZFS always reports the space used by the dnode itself (512 bytes,
or possibly more if there are embedded xattrs).  Ext4 doesn't report the inode
usage, since this is statically allocated at format time, but it does report
the space used by external xattr blocks and inline data (since it is stored in
an xattr itself, even if packed into the inode).

IMHO, that is also a valid approach to report some value:

    st_blocks = ((inline data size + 511) / 512)

to userspace for inline files, since they will _always_ be using some space on
disk (even if highly compressed), regardless of where it is being stored.

Cheers, Andreas

signature.asc
Description: Message signed with OpenPGP