Re: [Qemu-devel] converting the block layer from coroutines to threads

[Anthony Liguori]

On 02/24/2012 02:43 PM, Paolo Bonzini wrote:
> On 02/24/2012 08:22 PM, Anthony Liguori wrote:
> > Virtio really wants each virtqueue to be processed in a separate
> > thread.  On a multicore system, there's considerable improvement doing
> > this.  I think that's where we ought to start.
>
> Well, that's where we ought to *get*.  Stefan's work is awesome but with
> the current feature set it would be hard to justify it upstream.
>
> To get it upstream we need to generalize it and make it work well with
> the block layer.  And vice versa make the block layer work well with
> threads, which is what I care about here.
>
> > We really just need the block layer to be re-entrant, we don't
> > actually need qcow2 or anything else that uses coroutines to use full
> > threads.
>
> Once you can issue I/O from two threads at the same-time (such as
> streaming in the iothread and guest I/O in the virtqueue thread),
> everything already needs to be thread-safe.  It is a pretty short step
> from there to thread pools for everything.

If you start with a thread safe API for submitting block requests, that could be 
implemented as

bapi_aiocb *bapi_submit_readv(bapi_driver *d, struct iovec *iov, int iovcnt,
                              off_t offset)
{
   bapi_request *req = make_bapi_request(BAPI_READ, iov, iovcnt, offset);

   return bapi_queue_add_req(req);
}

Which would schedule the I/O thread to actually implement the operation.  You 
could then start incrementally refactoring specific drivers to be re-entrant 
(like linux-aio).  But anything that already needs to use a thread pool to do 
its I/O probably wouldn't benefit from threading virtio.

More importantly, the above would give you good performance to start with, 
instead of refactoring a bunch of code hoping to eventually get to good performance.

> > Or at least, as far as I know, we don't have any performance data to
> > suggest that we do.
>
> No, it's not about speed, though of course it only works if there is no
> performance dip.  It is just an enabling step.
>
> That said, my weekend officially begins now. :)

Enjoy!!

Regards,

Anthony Liguori

> Paolo