Re: [Qemu-devel] 4k seq read splitting for virtio-blk - possible workarounds?

[Andrey Korolyov]

On Mon, Oct 26, 2015 at 8:03 PM, Paolo Bonzini <address@hidden> wrote:
>
>
> On 26/10/2015 17:43, Andrey Korolyov wrote:
>> On Mon, Oct 26, 2015 at 7:37 PM, Paolo Bonzini <address@hidden> wrote:
>>> On 26/10/2015 17:31, Andrey Korolyov wrote:
>>>>> the virtio block device is always splitting a single read
>>>>> range request to 4k ones, bringing the overall performance of the
>>>>> sequential reads far below virtio-scsi.
>>>>>
>>>>> How does the blktrace look like in the guest?
>>>>
>>>> Yep, thanks for suggestion. It looks now like a pure driver issue:
>>>>
>>>>  Reads Queued:       11008,    44032KiB  Writes Queued:           0,       
>>>>  0KiB
>>>>  Read Dispatches:    11008,    44032KiB  Write Dispatches:        0,       
>>>>  0KiB
>>>>
>>>> vs
>>>>
>>>>  Reads Queued:      185728,   742912KiB  Writes Queued:           0,       
>>>>  0KiB
>>>>  Read Dispatches:     2902,   742912KiB  Write Dispatches:        0,       
>>>>  0KiB
>>>>
>>>> Because guest virtio-blk driver lacks *any* blk scheduler management,
>>>> this is kinda logical. Requests for scsi backend are dispatched in
>>>                                                        ^^^^^^^^^^
>>>
>>> queued you mean?
>>>
>>>> single block-sized chunks as well, but they are mostly merged by a
>>>> scheduler before being passed to the device layer. Could be there any
>>>> improvements over the situation except writing an underlay b/w virtio
>>>> emulator backend and the real storage?
>>>
>>> This is probably the fall-out of converting the virtio-blk to use
>>> blk-mq, which was premature to say the least.  Jeff Moyer was working on
>>> it, but I'm not sure if this has been merged.  Andrey, what kernel are
>>> you using?
>>
>> Queued, sorry for honest mistype, guest kernel is a 3.16.x from
>> jessie, so regular blk-mq is there. Any point of interest for trying
>> something newer? And of course I didn`t thought about something older,
>> will try against 3.10 now.
>
> Yes, it makes sense to try both something older and something newer.  I
> found this:
>
>     commit e6c4438ba7cb615448492849970aaf0aaa1cc973
>     Author: Jeff Moyer <address@hidden>
>     Date:   Fri May 8 10:51:30 2015 -0700
>
>     blk-mq: fix plugging in blk_sq_make_request
>
> Looking at the meat of the patch, we have:
>
>         const int is_sync = rw_is_sync(bio->bi_rw);
>         const int is_flush_fua = bio->bi_rw & (REQ_FLUSH | REQ_FUA);
> -       unsigned int use_plug, request_count = 0;
> +       struct blk_plug *plug;
> +       unsigned int request_count = 0;
>         struct blk_map_ctx data;
>         struct request *rq;
>
> -       /*
> -        * If we have multiple hardware queues, just go directly to
> -        * one of those for sync IO.
> -        */
> -       use_plug = !is_flush_fua && !is_sync;
>
> For reads rw_is_sync returns true, hence use_plug is always false.  So
> 4.2 kernels could fix this issue.
>
> Paolo

Yes, both cases are positive, thanks for very detailed explanation and
for tips. Does this also mean that most current distros which are
using 'broken' >=3.13 <4.2 driver would bring sequential read
performance, especially on rotating media, or media with high request
latency like hybrid disk, down to knees for virtio, which almost
always is a default selection?