Re: [Qemu-devel] qcow2 performance plan

[Kevin Wolf]

Am 14.09.2010 15:43, schrieb Anthony Liguori:
> Hi Avi,
> 
> On 09/14/2010 08:07 AM, Avi Kivity wrote:
>>  Here's a draft of a plan that should improve qcow2 performance.  It's 
>> written in wiki syntax for eventual upload to wiki.qemu.org; lines 
>> starting with # are numbered lists, not comments.
> 
> Thanks for putting this together.  I think it's really useful to think 
> through the problem before anyone jumps in and starts coding.
> 
>> = Basics =
>>
>> At the minimum level, no operation should block the main thread.  This
>> could be done in two ways: extending the state machine so that each
>> blocking operation can be performed asynchronously 
>> (<code>bdrv_aio_*</code>)
>> or by threading: each new operation is handed off to a worker thread.
>> Since a full state machine is prohibitively complex, this document
>> will discuss threading.
> 
> There's two distinct requirements that must be satisfied by a fast block 
> device.  The device must have fast implementations of aio functions and 
> it must support concurrent request processing.
> 
> If an aio function blocks in the process of submitting the request, it's 
> by definition slow.  But even if you may the aio functions fast, you 
> still need to be able to support concurrent request processing in order 
> to achieve high throughput.
> 
> I'm not going to comment in depth on your threading proposal.  When it 
> comes to adding concurrency, I think any approach will require a rewrite 
> of the qcow2 code and if the author of that rewrite is more comfortable 
> implementing concurrency with threads than with a state machine, I'm 
> happy with a threaded implementation.
> 
> I'd suggest avoiding hyperbole like "a full state machine is 
> prohibitively complex".  QED is a full state machine.  qcow2 adds a 
> number of additional states because of the additional metadata and sync 
> operations but it's not an exponential increase in complexity.

It will be quite some additional states that qcow2 brings in, but I
suspect the really hard thing is getting the dependencies between
requests right.

I just had a look at how QED is doing this, and it seems to take the
easy solution, namely allowing only one allocation at the same time. So
this is roughly equivalent to Avi's worker thread that runs today's
qcow2 code and is protected by a global mutex.

No doubt compared to real concurrency this is relatively easy to achieve
in either model (but probably easier with threads).

Kevin