[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL v4 76/83] vhost-user: Support vhost_dev_start
From: |
Michael S. Tsirkin |
Subject: |
[PULL v4 76/83] vhost-user: Support vhost_dev_start |
Date: |
Mon, 7 Nov 2022 17:53:54 -0500 |
From: Yajun Wu <yajunw@nvidia.com>
The motivation of adding vhost-user vhost_dev_start support is to
improve backend configuration speed and reduce live migration VM
downtime.
Today VQ configuration is issued one by one. For virtio net with
multi-queue support, backend needs to update RSS (Receive side
scaling) on every rx queue enable. Updating RSS is time-consuming
(typical time like 7ms).
Implement already defined vhost status and message in the vhost
specification [1].
(a) VHOST_USER_PROTOCOL_F_STATUS
(b) VHOST_USER_SET_STATUS
(c) VHOST_USER_GET_STATUS
Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
device start and reset(0) for device stop.
On reception of the DRIVER_OK message, backend can apply the needed setting
only once (instead of incremental) and also utilize parallelism on enabling
queues.
This improves QEMU's live migration downtime with vhost user backend
implementation by great margin, specially for the large number of VQs of 64
from 800 msec to 250 msec.
[1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html
Signed-off-by: Yajun Wu <yajunw@nvidia.com>
Acked-by: Parav Pandit <parav@nvidia.com>
Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
hw/virtio/vhost-user.c | 74 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 73 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index d256ce589b..abe23d4ebe 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -81,6 +81,7 @@ enum VhostUserProtocolFeature {
VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
/* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
+ VHOST_USER_PROTOCOL_F_STATUS = 16,
VHOST_USER_PROTOCOL_F_MAX
};
@@ -126,6 +127,8 @@ typedef enum VhostUserRequest {
VHOST_USER_GET_MAX_MEM_SLOTS = 36,
VHOST_USER_ADD_MEM_REG = 37,
VHOST_USER_REM_MEM_REG = 38,
+ VHOST_USER_SET_STATUS = 39,
+ VHOST_USER_GET_STATUS = 40,
VHOST_USER_MAX
} VhostUserRequest;
@@ -1452,6 +1455,43 @@ static int vhost_user_set_u64(struct vhost_dev *dev, int
request, uint64_t u64,
return 0;
}
+static int vhost_user_set_status(struct vhost_dev *dev, uint8_t status)
+{
+ return vhost_user_set_u64(dev, VHOST_USER_SET_STATUS, status, false);
+}
+
+static int vhost_user_get_status(struct vhost_dev *dev, uint8_t *status)
+{
+ uint64_t value;
+ int ret;
+
+ ret = vhost_user_get_u64(dev, VHOST_USER_GET_STATUS, &value);
+ if (ret < 0) {
+ return ret;
+ }
+ *status = value;
+
+ return 0;
+}
+
+static int vhost_user_add_status(struct vhost_dev *dev, uint8_t status)
+{
+ uint8_t s;
+ int ret;
+
+ ret = vhost_user_get_status(dev, &s);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if ((s & status) == status) {
+ return 0;
+ }
+ s |= status;
+
+ return vhost_user_set_status(dev, s);
+}
+
static int vhost_user_set_features(struct vhost_dev *dev,
uint64_t features)
{
@@ -1460,6 +1500,7 @@ static int vhost_user_set_features(struct vhost_dev *dev,
* backend is actually logging changes
*/
bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
+ int ret;
/*
* We need to include any extra backend only feature bits that
@@ -1467,9 +1508,18 @@ static int vhost_user_set_features(struct vhost_dev *dev,
* VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
* features.
*/
- return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
+ ret = vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
features | dev->backend_features,
log_enabled);
+
+ if (virtio_has_feature(dev->protocol_features,
+ VHOST_USER_PROTOCOL_F_STATUS)) {
+ if (!ret) {
+ return vhost_user_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
+ }
+ }
+
+ return ret;
}
static int vhost_user_set_protocol_features(struct vhost_dev *dev,
@@ -2620,6 +2670,27 @@ void vhost_user_cleanup(VhostUserState *user)
user->chr = NULL;
}
+static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
+{
+ if (!virtio_has_feature(dev->protocol_features,
+ VHOST_USER_PROTOCOL_F_STATUS)) {
+ return 0;
+ }
+
+ /* Set device status only for last queue pair */
+ if (dev->vq_index + dev->nvqs != dev->vq_index_end) {
+ return 0;
+ }
+
+ if (started) {
+ return vhost_user_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+ VIRTIO_CONFIG_S_DRIVER |
+ VIRTIO_CONFIG_S_DRIVER_OK);
+ } else {
+ return vhost_user_set_status(dev, 0);
+ }
+}
+
const VhostOps user_ops = {
.backend_type = VHOST_BACKEND_TYPE_USER,
.vhost_backend_init = vhost_user_backend_init,
@@ -2654,4 +2725,5 @@ const VhostOps user_ops = {
.vhost_backend_mem_section_filter = vhost_user_mem_section_filter,
.vhost_get_inflight_fd = vhost_user_get_inflight_fd,
.vhost_set_inflight_fd = vhost_user_set_inflight_fd,
+ .vhost_dev_start = vhost_user_dev_start,
};
--
MST
- [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files, (continued)
- [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files, Michael S. Tsirkin, 2022/11/07
- [PULL v4 67/83] hw/arm/virt: Enable HMAT on arm virt machine, Michael S. Tsirkin, 2022/11/07
- [PULL v4 68/83] tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators, Michael S. Tsirkin, 2022/11/07
- [PULL v4 69/83] tests: virt: Update expected *.acpihmatvirt tables, Michael S. Tsirkin, 2022/11/07
- [PULL v4 70/83] vfio: move implement of vfio_get_xlat_addr() to memory.c, Michael S. Tsirkin, 2022/11/07
- [PULL v4 71/83] intel-iommu: don't warn guest errors when getting rid2pasid entry, Michael S. Tsirkin, 2022/11/07
- [PULL v4 72/83] intel-iommu: drop VTDBus, Michael S. Tsirkin, 2022/11/07
- [PULL v4 73/83] intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function, Michael S. Tsirkin, 2022/11/07
- [PULL v4 74/83] intel-iommu: PASID support, Michael S. Tsirkin, 2022/11/07
- [PULL v4 75/83] vhost: Change the sequence of device start, Michael S. Tsirkin, 2022/11/07
- [PULL v4 76/83] vhost-user: Support vhost_dev_start,
Michael S. Tsirkin <=
- [PULL v4 77/83] hw/smbios: add core_count2 to smbios table type 4, Michael S. Tsirkin, 2022/11/07
- [PULL v4 78/83] bios-tables-test: teach test to use smbios 3.0 tables, Michael S. Tsirkin, 2022/11/07
- [PULL v4 79/83] tests/acpi: allow changes for core_count2 test, Michael S. Tsirkin, 2022/11/07
- [PULL v4 80/83] bios-tables-test: add test for number of cores > 255, Michael S. Tsirkin, 2022/11/07
- [PULL v4 81/83] tests/acpi: update tables for new core count test, Michael S. Tsirkin, 2022/11/07
- [PULL v4 82/83] hw/virtio: introduce virtio_device_should_start, Michael S. Tsirkin, 2022/11/07
- [PULL v4 83/83] checkpatch: better pattern for inline comments, Michael S. Tsirkin, 2022/11/07
- Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups, Michael S. Tsirkin, 2022/11/08
- Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups, Philippe Mathieu-Daudé, 2022/11/15