Re: [Qemu-devel] [PATCH v2] posix-aio-compat: fix latency issues

[Stefan Hajnoczi]

On Mon, Aug 22, 2011 at 6:29 PM, Jan Kiszka <address@hidden> wrote:
> On 2011-08-14 06:04, Avi Kivity wrote:
>> In certain circumstances, posix-aio-compat can incur a lot of latency:
>>  - threads are created by vcpu threads, so if vcpu affinity is set,
>>    aio threads inherit vcpu affinity.  This can cause many aio threads
>>    to compete for one cpu.
>>  - we can create up to max_threads (64) aio threads in one go; since a
>>    pthread_create can take around 30μs, we have up to 2ms of cpu time
>>    under a global lock.
>>
>> Fix by:
>>  - moving thread creation to the main thread, so we inherit the main
>>    thread's affinity instead of the vcpu thread's affinity.
>>  - if a thread is currently being created, and we need to create yet
>>    another thread, let thread being born create the new thread, reducing
>>    the amount of time we spend under the main thread.
>>  - drop the local lock while creating a thread (we may still hold the
>>    global mutex, though)
>>
>> Note this doesn't eliminate latency completely; scheduler artifacts or
>> lack of host cpu resources can still cause it.  We may want pre-allocated
>> threads when this cannot be tolerated.
>>
>> Thanks to Uli Obergfell of Red Hat for his excellent analysis and 
>> suggestions.
>
> At this chance: What is the state of getting rid of the remaining delta
> between upstream's version and qemu-kvm?

That would be nice.  qemu-kvm.git uses a signalfd to handle I/O
completion whereas qemu.git uses a signal, writes to a pipe from the
signal handler, and uses qemu_notify_event() to break the vcpu.  Once
the force iothread patch is merged we should be able to move to
qemu-kvm.git's signalfd approach.

Stefan