Re: [Qemu-devel] RFC: async commands with QMP

[Daniel P. Berrange]

On Wed, Jul 29, 2015 at 04:49:56PM +0200, Marc-André Lureau wrote:
> Hi
> 
> It seems there is no support for async commands with QAPI/QMP other
> than having to poll for status, or a "sync" command followed
> eventually by an event. But, there is no direct relation between the
> command and the event.
> 
> As a monitor command shouldn't block, it would be useful to have async
> commands support, if possible with a common scheme.
> 
> Here is a simple proposal, let's take an existing command:
> 
>                 -> { "execute": "drive-backup", "arguments": {
> "device": "drive0",
>                                                                "sync": "full",
> 
> "target": "backup.img" },
>                       "id": 1234 }
>                 <- { "return": {}, "id": 1234 }
> 
> 
> You can then check regularly the state with query-block-jobs.. Suppose
> instead of returning immediately, a similar function would return when
> the task is completed. It would be nice if you wouldn't have to
> duplicate existing blocking functions to an _async variant. Could we
> introduce an additional "async" member? (rightfully rejected on qemu
> today)
> 
>                 -> { "execute": "drive-backup", "arguments": {
> "device": "drive0",
>                                                                "sync": "full",
> 
> "target": "backup.img" },
>                       "id": 1234,
>                       "async": true }
> Here, with "async": true, the caller knows he shouldn't expect an
> immediate return, and he can exchange other messages:
>                 -> { "execute"...
>                 <- { "event", ...
> And when "drive-backup" finishes:
>                 <- { "return": {}, "id": 1234 }
> 
> In order to make "async" variant possible, "id" would have to be
> provided and unique. I think the async support should also be
> announced in the qmp_capabilities. Commands would have to opt-in for
> async support in the schema. An async return wouldn't be allowed when
> a blocking command is ongoing.
> 
> There are many variations possible on the same idea. We could
> introduce new _async functions instead, and keep the "id"
> requirements. Or alternatively, an "async_id" could be generated by
> qemu and returned immediately. Then the reply of the command could be
> an event instead. Ex:
> 
>                 -> { "execute": "drive-backup-async", "arguments": {
> "device": "drive0",
>                                                                "sync": "full",
> 
> "target": "backup.img" },
>                       "id": 1234 }
>                 <- { "return-async": {}, "async_id": 42 }
> ... later on:
>                 <- {"event": "ASYNC_COMPLETED", "async_id": 42,
> "data": {.return values could go there..}, "id" 1234}
> 
> I haven't looked much on the implementation side, but I can try to
> implement a proof-of-concept. Let see if this threads brings some
> discussion first.

When QMP was originally written there was some infrastructure for doing
async commands, but over time this has all been ripped out because it
was never really used, complicated the code and didn't work too well
either. It seems we pretty much settled on the approach that all
commands should be fast to execute, and where there is a long term
job being run, we have commands to query its status, cancel it, and
sometimes events too.  One of the benefits of this is that it means
that libvirt can determine current status of ongoing background jobs
when it restarts and reconnects to a previously launched QEMU, where
as an async command approach is tied to the specific monitor connection
that is open.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|