Re: [Qemu-devel] [RFC 0/9] QContext: QOM class to support multiple event loops

[liu ping fan]

On Sat, May 4, 2013 at 12:03 AM, Michael Roth <address@hidden> wrote:
> These patches apply on top of qemu.git master, and can also be obtained from:
> git://github.com/mdroth/qemu.git qcontext
>
> OVERVIEW
>
> This series introduces a set of QOM classes/interfaces for event
> registration/handling: QContext and QSource, which are based closely on
> their GMainContext/GSource GLib counterparts.
>
> QContexts can be created via the command-line via -object, and can also be
> intructed (via -object params/properties) to automatically start a
> thread/event-loop to handle QSources we attach to them.
>
> The reference implementation of QContext is GlibQContext, which uses
> GMainContext/GSource interfaces underneath the covers, thus we can
> also attach GSource (and as a result, AioContexts) to it.
>
> As part of this series we also port the QEMU main loop to using QContext
> and drop virtio-blk's dataplane thread in favor of a GlibQContext thread,
> which virtio-blk dataplane attaches to via it's AioContext's GSource.
>
> With these patches in place we can do virtio-blk dataplane assignment
> like so:
>
>   qemu ... \
>     -object glib-qcontext,id=ctx0,threaded=yes
>     -drive file=file.raw,id=drive0,aio=native,cache=none \
>     -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=ctx0
>
> And also gain the ability to assign a virtio-blk dataplane to the default
> QContext driven by the QEMU main iothread:
>
>   qemu ... \
>     -drive file=file.raw,id=drive0,aio=native,cache=none \
>     -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=main
>
> The latter likely isn't particularly desirable, and the series is in rough
> shape in general, but the goal of this RFC to demonstrate the approach and
> get some feedback on how we might handle thread assignments for things like
> virtio-blk/virtio-net dataplane, and multi-threaded device models general.
>
> Input on this would be greatly appreciated.
>
> BACKGROUND
>
> There has been an outgoing discussion on qemu-devel about what event loop
> interface to consolidate around for virtio-blk dataplane, threaded virtio-net,
> and offloading device workloads to seperate threads in general.
>
> Currently the main candidates seem to be GLib GSources and AioContext, with
> virtio-blk/virtio-net dataplane being the use-cases under consideration.
>
> virtio-blk:
>
> In the case of virtio-blk dataplane, we need to drive virtqueue and AIO 
> events.
> Since AioContext is used extensively throughout the block layer to drive AIO,
> it makes sense to re-use it here as we look toward pushing dataplane
> functionality deeper into the block layer to benefit from things like image
> format support/snapshots/etc.
>
> virtio-net:
>
> In the case of Ping Fan's virtio-net dataplane patches
> (http://thread.gmane.org/gmane.comp.emulators.qemu/196111/focus=196111), we
> need to drive virtqueue and NetClient peer events (such as TAP devices, or
> possibly things like slirp in the future). Currently NetClient events rely on
> IOHandler interfaces such as qemu_set_fd_handler(). These interfaces are 
> global
> ones that rely on a single IOHandler list serviced by QEMU's main loop. An
> effort is currently underway to port these to GSources so that can be more
> easilly attached to other event loops (as opposed to the hooks used for the
> virtio-net dataplane series).
>
> Theoretically, much of the latter (such as TAP devices) can also be done 
> around
> AioContext with some minor changes, but other NetClient backends such as slirp
> lend themselves to more open-ended event loop interfaces like GSources. Other
> devices might also find themselves needing something more open-ended (a 
> somewhat
> silly but present example being virtio-serial + GSource-driven chardev)
>
> QContext:
>
> Since the direction for the forseeable future will likely continue to be
> GSources for some things, AioContext for others, a way to reconcile these
> approaches would be the age-old approach of adding a layer of abstration on
> top of the 2 so that we can handle device/backend thread assignments using
> a general mechanism. Building around this abstration also leaves open the
> ability to deal with things like locking considerations for real-time support
> in the future.
>
> A reasonable start to modeling abstraction layer this would be the open-ended
> GMainContext/GSource approach that QEMU relies on already, which is what
> the QContext/QSource interfaces do (with some minor differences/additions
> such as QSources storing and opaque instead of the GSource-subclassing 
> approach
> for GLib).
>
I think, custom-ed function for readable or not , ex, tap_can_send()
should be passed into QSource, but I failed to find such things in
[PATCH 3/9] QSource: QEMU event source object.  Do I miss it?

Thanks and regards,
Pingfan
> TODO/KNOWN ISSUES
>
>  - GTK UI causes a crash during certain window refresh events. works fine with
>    VNC though, and no problems with other GSources.
>  - slirp isn't working (will work with Ping Fan's slirp->GSource conversion)
>  - add proper locking
>  - integrate timers into a QSource to make timer-event driveable in seperate
>    QContext event loops
>  - consider a command-line wrapper to -object, such as:
>      -context <id>,[threaded=<yes|no>],[backend=<default|glib>]
>
>  Makefile.objs                    |    6 +-
>  async.c                          |   16 +++
>  hw/block/dataplane/virtio-blk.c  |   46 ++-----
>  include/block/aio.h              |    6 +
>  include/hw/virtio/virtio-blk.h   |    7 +-
>  include/qcontext/glib-qcontext.h |   59 ++++++++
>  include/qcontext/qcontext.h      |   76 +++++++++++
>  include/qcontext/qsource.h       |   63 +++++++++
>  include/qemu/main-loop.h         |   31 ++++-
>  include/qom/object.h             |   35 +++++
>  iohandler.c                      |  238 ++++++++++++++++++++++----------
>  main-loop.c                      |   96 ++++++-------
>  qcontext/Makefile.objs           |    1 +
>  qcontext/glib-qcontext.c         |  280 
> ++++++++++++++++++++++++++++++++++++++
>  qcontext/qcontext.c              |  252 ++++++++++++++++++++++++++++++++++
>  qcontext/qsource.c               |  143 +++++++++++++++++++
>  qom/Makefile.objs                |    6 +-
>  qom/object.c                     |   46 +++++++
>  tests/Makefile                   |    7 +
>  tests/test-qcontext.c            |  259 +++++++++++++++++++++++++++++++++++
>  vl.c                             |    2 +
>  21 files changed, 1512 insertions(+), 163 deletions(-)
>