Re: [PATCH][RESEND v3 3/3] Add a Hyper-V Dynamic Memory Protocol driver (hv-balloon)

[Maciej S. Szmigiero]

On 28.02.2023 18:34, Daniel P. Berrangé wrote:
> On Fri, Feb 24, 2023 at 10:41:16PM +0100, Maciej S. Szmigiero wrote:
>
> > Hot-adding additional memory is done by creating a new memory backend (for
> > example by executing HMP command
> > "object_add memory-backend-ram,id=mem1,size=4G"), then executing a new
> > "hv-balloon-add-memory" QMP command, providing the id of that memory
> > backend as the "id" parameter.
>
> [snip]
>
> > After a VM reboot each previously hot-added memory backend gets released.
> > A "HV_BALLOON_MEMORY_BACKEND_UNUSED" QMP event is emitted in this case so
> > the software controlling QEMU knows that it either needs to delete that
> > memory backend (if no longer needed) or re-insert it.
>
> IIUC you're saying that the 'hv-balloon-add-memory' command needs
> to be re-run after a guest reset ?

Yes.

> If so I feel that is a rather> undesirable job to punt over the mgmt app. The 
> 'reset' event can
> be missed if the mgmt app happend to be restarting and reconnecting
> to existing running QMP console.

See the answer below the next paragraph.

> > In the future, the guest boot memory size might be changed on reboot
> > instead, taking into account the effective size that VM had before that
> > reboot (much like Hyper-V does).
>
> Is that difficult to do right now ?  It isn't too nice to make the
> mgmt apps implement the workaround now if we're going to make it
> redundant later.

The v1 of this driver did re-add memory backends automatically after
a reboot, so if that's something that is desirable it can be re-introduced
without much difficulty.

The issue here is that the guest might never re-connect to the DM protocol
interface after a reboot (perhaps because the VM was rebooted from
a Windows to a Linux guest).
In this case the driver would wait endlessly, not letting the
underlying memory backends to be removed.

virtio-mem also seems to unplug all blocks unconditionally when the VM is
rebooted.

On the other hand, actually resizing the guest boot memory is definitely
not trivial - for sure that's something for future work
(virtio-mem might also benefit from it).

> > The above design results in much better ballooning performance than when
> > using virtio-balloon with the same guest: 230 GB / minute with this driver
> > versus 70 GB / minute with virtio-balloon.
>
> snip
>
> > The unballoon operation is also pretty much instantaneous:
> > thanks to the merging of the ballooned out page ranges 200 GB of memory can
> > be returned to the guest in about 1 second.
> > With virtio-balloon this operation takes about 2.5 minutes.
>
> That's pretty impressive !

Thanks!

> > These tests were done against a Windows Server 2019 guest running on a
> > Xeon E5-2699, after dirtying the whole memory inside guest before each
> > balloon operation.
>
>
>
> > Since the required GTree operations aren't present in every Glib version
> > a check for them was added to "configure" script, together with new
> > "--enable-hv-balloon" and "--disable-hv-balloon" arguments.
> > If these GTree operations are missing in the system's Glib version this
> > driver will be skipped during QEMU build.
>
> Funnily enough there's a patch posted recently that imports the glib
> GTree impl into QEMU calling it QTree. This was to workaround a problem
> with GSlice not being async signal safe, but if we take that patch, then
> you wouldn't need to skip the build you could rely on this in-tree copy
> instead.
>
> https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg01225.html

Thanks for the pointer, however the {G,Q}Tree import explicitly excludes
these tree operations that this driver needs (as they currently don't
have any callers in QEMU).

So in this case either they would have to be imported too or the driver
would need QEMU being built with the upstream Glib (as far as I can see,
[1] says this will still be possible with glib >= 2.76.0).

Thanks,
Maciej

[1]: https://gitlab.com/qemu-project/qemu/-/issues/285