[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v3 00/14] Dynamically switch to vhost shadow virtqueues at vdpa n
From: |
Eugenio Pérez |
Subject: |
[PATCH v3 00/14] Dynamically switch to vhost shadow virtqueues at vdpa net migration |
Date: |
Wed, 15 Feb 2023 18:38:36 +0100 |
It's possible to migrate vdpa net devices if they are shadowed from the
start. But to always shadow the dataplane is to effectively break its host
passthrough, so its not efficient in vDPA scenarios.
This series enables dynamically switching to shadow mode only at
migration time. This allows full data virtqueues passthrough all the
time qemu is not migrating.
In this series only net devices with no CVQ are migratable. CVQ adds
additional state that would make the series bigger and still had some
controversy on previous RFC, so let's split it.
Successfully tested with vdpa_sim_net with patch [1] applied and with the qemu
emulated device with vp_vdpa with some restrictions:
* No CVQ. No feature that didn't work with SVQ previously (packed, ...)
* VIRTIO_RING_F_STATE patches implementing [2].
* Expose _F_SUSPEND, but ignore it and suspend on ring state fetch like
DPDK.
Previous versions were tested by many vendors. Not carrying Tested-by because
of code changes, so re-testing would be appreciated.
Comments are welcome.
v3:
- Start datapatch in SVQ in device started while migrating.
- Properly register migration blockers if device present unsupported features.
- Fix race condition because of not stopping the SVQ until device cleanup.
- Explain purpose of iova tree in the first patch message.
- s/dynamycally/dynamically/ in cover letter.
v2:
- Check for SUSPEND in vhost_dev.backend_cap, as .backend_features is empty at
the check moment.
- at 20230208094253.702672-12-eperezma@redhat.com/T/">https://lore.kernel.org/all/20230208094253.702672-12-eperezma@redhat.com/T/
v1:
- Omit all code working with CVQ and block migration if the device supports
CVQ.
- Remove spurious kick.
- Move all possible checks for migration to vhost-vdpa instead of the net
backend. Move them to init code from start code.
- Suspend on vhost_vdpa_dev_start(false) instead of in vhost-vdpa net backend.
- Properly split suspend after geting base and adding of status_reset patches.
- Add possible TODOs to points where this series can improve in the future.
- Check the state of migration using migration_in_setup and
migration_has_failed instead of checking all the possible migration status in
a switch.
- Add TODO with possible low hand fruit using RESUME ops.
- Always offer _F_LOG from virtio/vhost-vdpa and let migration blockers do
their thing instead of adding a variable.
- RFC v2 at https://lists.gnu.org/archive/html/qemu-devel/2023-01/msg02574.html
RFC v2:
- Use a migration listener instead of a memory listener to know when
the migration starts.
- Add stuff not picked with ASID patches, like enable rings after
driver_ok
- Add rewinding on the migration src, not in dst
- RFC v1 at https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg01664.html
[1] https://lore.kernel.org/lkml/20230203142501.300125-1-eperezma@redhat.com/T/
[2] https://lists.oasis-open.org/archives/virtio-comment/202103/msg00036.html
Eugenio Pérez (14):
vdpa net: move iova tree creation from init to start
vdpa: stop svq at vhost_vdpa_dev_start(false)
vdpa: Negotiate _F_SUSPEND feature
vdpa: add vhost_vdpa_suspend
vdpa: move vhost reset after get vring base
vdpa: rewind at get_base, not set_base
vdpa net: allow VHOST_F_LOG_ALL
vdpa: add vdpa net migration state notifier
vdpa: disable RAM block discard only for the first device
vdpa net: block migration if the device has CVQ
vdpa: block migration if device has unsupported features
vdpa: block migration if dev does not have _F_SUSPEND
vdpa: block migration if SVQ does not admit a feature
vdpa: return VHOST_F_LOG_ALL in vhost-vdpa devices
include/hw/virtio/vhost-backend.h | 4 +
include/hw/virtio/vhost-vdpa.h | 1 +
hw/virtio/vhost-vdpa.c | 130 +++++++++++++++-----
hw/virtio/vhost.c | 3 +
net/vhost-vdpa.c | 195 +++++++++++++++++++++++++-----
hw/virtio/trace-events | 1 +
6 files changed, 274 insertions(+), 60 deletions(-)
--
2.31.1
- [PATCH v3 00/14] Dynamically switch to vhost shadow virtqueues at vdpa net migration,
Eugenio Pérez <=
- [PATCH v3 01/14] vdpa net: move iova tree creation from init to start, Eugenio Pérez, 2023/02/15
- [PATCH v3 02/14] vdpa: stop svq at vhost_vdpa_dev_start(false), Eugenio Pérez, 2023/02/15
- [PATCH v3 03/14] vdpa: Negotiate _F_SUSPEND feature, Eugenio Pérez, 2023/02/15
- [PATCH v3 04/14] vdpa: add vhost_vdpa_suspend, Eugenio Pérez, 2023/02/15
- [PATCH v3 05/14] vdpa: move vhost reset after get vring base, Eugenio Pérez, 2023/02/15
- [PATCH v3 06/14] vdpa: rewind at get_base, not set_base, Eugenio Pérez, 2023/02/15
- [PATCH v3 07/14] vdpa net: allow VHOST_F_LOG_ALL, Eugenio Pérez, 2023/02/15
- [PATCH v3 08/14] vdpa: add vdpa net migration state notifier, Eugenio Pérez, 2023/02/15