So-called "internal" virtio-fs migration refers to transporting the
back-end's (virtiofsd's) state through qemu's migration stream. To do
this, we need to be able to transfer virtiofsd's internal state to and
from virtiofsd.
Because virtiofsd's internal state will not be too large, we believe it
is best to transfer it as a single binary blob after the streaming
phase. Because this method should be useful to other vhost-user
implementations, too, it is introduced as a general-purpose addition to
the protocol, not limited to vhost-user-fs.
These are the additions to the protocol:
- New vhost-user protocol feature VHOST_USER_PROTOCOL_F_MIGRATORY_STATE:
This feature signals support for transferring state, and is added so
that migration can fail early when the back-end has no support.
- SET_DEVICE_STATE_FD function: Front-end and back-end negotiate a pipe
over which to transfer the state. The front-end sends an FD to the
back-end into/from which it can write/read its state, and the back-end
can decide to either use it, or reply with a different FD for the
front-end to override the front-end's choice.
The front-end creates a simple pipe to transfer the state, but maybe
the back-end already has an FD into/from which it has to write/read
its state, in which case it will want to override the simple pipe.
Conversely, maybe in the future we find a way to have the front-end
get an immediate FD for the migration stream (in some cases), in which
case we will want to send this to the back-end instead of creating a
pipe.
Hence the negotiation: If one side has a better idea than a plain
pipe, we will want to use that.
- CHECK_DEVICE_STATE: After the state has been transferred through the
pipe (the end indicated by EOF), the front-end invokes this function
to verify success. There is no in-band way (through the pipe) to
indicate failure, so we need to check explicitly.
Once the transfer pipe has been established via SET_DEVICE_STATE_FD
(which includes establishing the direction of transfer and migration
phase), the sending side writes its data into the pipe, and the reading
side reads it until it sees an EOF. Then, the front-end will check for
success via CHECK_DEVICE_STATE, which on the destination side includes
checking for integrity (i.e. errors during deserialization).
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
---
include/hw/virtio/vhost-backend.h | 24 +++++
include/hw/virtio/vhost.h | 79 ++++++++++++++++
hw/virtio/vhost-user.c | 147 ++++++++++++++++++++++++++++++
hw/virtio/vhost.c | 37 ++++++++
4 files changed, 287 insertions(+)
diff --git a/include/hw/virtio/vhost-backend.h
b/include/hw/virtio/vhost-backend.h
index ec3fbae58d..5935b32fe3 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -26,6 +26,18 @@ typedef enum VhostSetConfigType {
VHOST_SET_CONFIG_TYPE_MIGRATION = 1,
} VhostSetConfigType;
+typedef enum VhostDeviceStateDirection {
+ /* Transfer state from back-end (device) to front-end */
+ VHOST_TRANSFER_STATE_DIRECTION_SAVE = 0,
+ /* Transfer state from front-end to back-end (device) */
+ VHOST_TRANSFER_STATE_DIRECTION_LOAD = 1,
+} VhostDeviceStateDirection;
+
+typedef enum VhostDeviceStatePhase {
+ /* The device (and all its vrings) is stopped */
+ VHOST_TRANSFER_STATE_PHASE_STOPPED = 0,
+} VhostDeviceStatePhase;