Re: [RFC PATCH] curl: Allow reading after EOF

[Eric Blake]

On 3/17/21 11:43 AM, Kevin Wolf wrote:
>>> It is not entirely clear to me if this is something we want to do. If we
>>> do care about consistency between protocol drivers, something like this
>>> should probably be done in block/io.c eventually - but that would
>>> require converting bs->total_sectors to byte granularity first.
>>>
>>> Any opinions on what the most desirable semantics would be and whether
>>> we should patch individual drivers until we can have a generic solution?
>>
>> What valid scenarios are there for wanting to read beyond the bounds
>> of the protocol driver storage ? Why was file-posix allowing this
>> so far ?
>>

Our block driver already filters all reads larger than the image size
rounded to the nearest sector; so this discussion is ONLY about the 511
bytes possible in an unaligned file at the protocol layer and its
rounded-up size at the block layer.

>> If I've given file-posix a 10 GB plain file or device and something
>> requests a read from the 11 GB offset, IMHO, that is a sign of serious
>> error somewhere and possible impending doom.

The block layer won't permit that; it's too far beyond the 511 bytes of
rounding up a sector-unaligned image.

>>
>> For writable storage, I would think that read + write should be
>> symmetric, by which I mean if a read() at a particular offset
>> succeeds, then I would also expect a write() at the same offset to
>> succeed, and have its data later returned by a read().
>>
>> We generally can't write at an offset beyond the storage (unless we
>> are intending to auto-enlarge a plain file), so I think we shouldn't
>> allow reads either.
> 
> It is definitely related to format drivers that grow their image files.
> I think the reason for allowing this may have been that with O_DIRECT,
> you need aligned requests and when format drivers write just a few
> bytes, we actually do a RMW - and you don't want to get an error during
> the read part just because the image file will only be resized by the
> write.

I like the nbdkit behavior for symmetry: since we can read the tail as
zero, allowing write as zero does not change the size but also avoids an
ENOSPC, while allowing the guest full control over the bytes prior to
the unaligned tail.  But I can also live with the symmetry of reads from
the final sector see zero, but writes to the final sector fail
(basically, the final sector becomes read-only, even if the rest of the
image is writable).

> 
> Since curl is a read-only protocol driver (at the moment, I actually
> have an experimental branch that adds write support so we can run
> iotests for http), this reason doesn't really apply. At the moment, it
> would be just for consistency.


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org