Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

[Dongli Zhang]

On 4/16/19 1:34 AM, Wei Li wrote:
> Hi @Paolo Bonzini & @Stefan Hajnoczi,
> 
> Would you please help confirm whether @Paolo Bonzini's multiqueue feature 
> change will benefit virtio-scsi or not? Thanks!
> 
> @Stefan Hajnoczi,
> I also spent some time on exploring the virtio-scsi multi-queue features via 
> num_queues parameter as below, here are what we found:
> 
> 1. Increase number of Queues from one to the same number as CPU will get 
> better IOPS increase.
> 2. Increase number of Queues to the number (e.g. 8) larger than the number of 
> vCPU (e.g. 2) can get even better IOPS increase.

As mentioned in below link, when the number of hw queues is larger than
nr_cpu_ids, the blk-mq layer would limit and only use at most nr_cpu_ids queues
(e.g., /sys/block/sda/mq/).

That is, when the num_queus=4 while vcpus is 2, there should be only 2 queues
available /sys/block/sda/mq/

https://lore.kernel.org/lkml/address@hidden/

I am just curious how increasing the num_queues from 2 to 4 would double the
iops, while there are only 2 vcpus available...

Dongli Zhang

> 
> In addition, It seems Qemu can get better IOPS while the attachment uses more 
> queues than the number of vCPU, how could it possible? Could you please help 
> us better understand the behavior? Thanks a lot!
> 
> 
> Host CPU Configuration:
> CPU(s):                2
> Thread(s) per core:    2
> Core(s) per socket:    1
> Socket(s):             1
> 
> Commands for multi queue Setup:
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=1 id=test1
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=2 id=test2
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=4 id=test4
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=8 id=test8
> 
> 
> Result:
>       |  8 Queues   |  4 Queues     |      2 Queues    |      Single Queue
> IOPS  |   +29%         |      27%           |        11%           |      
> Baseline
> 
> Thanks,
> Wei
> 
> On 4/5/19, 2:09 PM, "Wei Li" <address@hidden> wrote:
> 
>     Thanks Stefan for your quick response!
>     
>     Hi Paolo,
>     Could you please send us a link related to the multiqueue feature which 
> you are working on so that we could start getting some details about the 
> feature.
>     
>     Thanks again,
>     Wei 
>     
>     On 4/1/19, 3:54 AM, "Stefan Hajnoczi" <address@hidden> wrote:
>     
>         On Fri, Mar 29, 2019 at 08:16:36AM -0700, Wei Li wrote:
>         > Thanks Stefan for your reply and guidance!
>         > 
>         > We spent some time on exploring the multiple I/O Threads approach 
> per your feedback. Based on the perf measurement data, we did see some IOPS 
> improvement for multiple volumes, which is great. :)
>         > 
>         > In addition, IOPS for single Volume will still be a bottleneck, it 
> seems like multiqueue block layer feature which Paolo is working on may be 
> able to help improving the IOPS for single volume.
>         > 
>         > @Paolo, @Stefan, 
>         > Would you mind sharing the multiqueue feature code branch with us? 
> So that we could get some rough idea about this feature and maybe start doing 
> some exploration? 
>         
>         Paolo last worked on this code, so he may be able to send you a link.
>         
>         Stefan
>         
>     
> 
> 
>