Re: [Qemu-devel] [PATCH v2 00/40] Generic background jobs

[Dr. David Alan Gilbert]

* Marc-André Lureau (address@hidden) wrote:
> Hi
> 
> On Tue, May 22, 2018 at 1:01 PM, Kevin Wolf <address@hidden> wrote:
> > Am 18.05.2018 um 20:41 hat Dr. David Alan Gilbert geschrieben:
> >> * Kevin Wolf (address@hidden) wrote:
> >> > Before we can make x-blockdev-create a background job, we need to
> >> > generalise the job infrastructure so that it can be used without any
> >> > associated block node.
> >>
> >> Is there any relationship between what this does, and what
> >> Marc-André's 'monitor: add asynchronous command type' tries to do?
> >> (See address@hidden 26th March)
> >
> > Depends on who you ask. :-)
> >
> > In a way, yes. Marc-André's async commands are for long-running
> > operations and jobs are for long-running operations. However, they are
> > for a different kind of "long-running".
> >
> > Basically, Marc-André's work is for commands that are generally quick,
> > but perform some blocking operation. Usually commands that are currently
> > implemented synchronously, but always bothered us because they block the
> > VM for a while. They take maybe a few seconds at most, but you really
> > don't want them to block the guest even for short time.
> >
> > The important part here is you don't want to modify the operations while
> > they're running, they just send their return value when they are done.
> > (In the latest version, Marc-André even made sure not to modify the wire
> > protocol, so IIUC the commands aren't really async any more in the sense
> > that you can have multiple commands running, but they are just
> > non-blocking in the sense that the guest can keep running while they are
> > doing their work.)
> >
> > In comparison, jobs are typically really long running, like several
> > minutes (like copying a whole disk image). Some of them can even run
> > indefinitely (like mirroring) until you explicitly tell them to stop.
> > You want to continue using the monitor while they are running, and to be
> > able to manage them at runtime (pause/resume/set speed/cancel/...).
> >
> > So I think both address different problems.
> 
> I agree with Kevin that both address different needs and complement
> each other. Right now, QMP commands are handled synchronous, so they
> block the main thread (or the "OOB" thread). I needed a simple way to
> handle them asynchronously, *without* breaking or changing an existing
> QMP command API. Job, on the other hand, provides a lot of facilities
> that are unnecessary for most short living commands, while adding
> complexity, at the protocol level (for instance, by using an extra
> "job-id" space and not being tied to the qmp session). Depending on
> the facilities Job provide, and the guarantee that must hold when
> using it, it may make sense or help to transform an existing command
> to an asynchronous version, but I doubt many of the existing commands
> will need it. However, if there is a need to introduce job-like
> facilities to QMP async (such as running concurrent commands, listing
> running commands, being able to cancel them, being notified on
> progression etc), then I think we should be careful to use Job. For
> now, this is unneeded, as QMP async is internal to qemu, the client
> can't run anything  to manage an ongoing async command, so I think
> both are improvements and will coexist harmoniously.

OK; it seems we are carving stuff up into a whole bunch of different
categories based on their longevity and blockableness.
(Normal, OOB, Async, Job)

I do worry a bit that pretty much anything that's not 'instant' means
it's waiting for something somewhere, and because you're waiting
for something that means you've got to be able to fail or cancel
it in case that thing never happens.

Dave
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK