[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Virtio-fs] Live migration support for virtio-fs
From: |
Vivek Goyal |
Subject: |
Re: [Virtio-fs] Live migration support for virtio-fs |
Date: |
Mon, 3 May 2021 09:07:16 -0400 |
Hi Xiongzi,
Try qemu monitor to hot device add/remove that works. I think libvirt
support is yet to be added.
One issue I faced was that I had to add a pcie root to move virtiofs
device off pcie.0 and then hot add remove functionality works.
So I had to add pcie root port by adding following to qemu command
line.
-device pcie-root-port,id=root1,slot=0
And now you can hot add a device using device_add.
(qemu) device_add
vhost-user-fs-pci,bus=root1,id=myvirtiofs,chardev=char0,tag=myfs,queue-size=1024
- To hot remove device, issue from qemu monitor.
(qemu) device_del <device_id>
ex. device_del myvirtiofs
Thanks
Vivek
On Mon, May 03, 2021 at 12:50:56PM +0000, Ge, Xiongzi wrote:
> Thanks, Stefan!
> It seems that libvirt does not support device-attach for filesystem type
> device, like virtiofs. Are there any other ways to do this hot-unplugging?
>
> Xiongzi
>
> On 4/8/21, 12:21 PM, "Stefan Hajnoczi" <stefanha@redhat.com> wrote:
>
> On Wed, Apr 07, 2021 at 06:05:29PM +0000, Ge, Xiongzi wrote:
> > Thanks Dr Gilbert, Vivek, Stefan, Greg!
> > I put together the discussion into this thread and CC
> qemu-devel@nongnu.org.
> >
> > Problem:
> > Current Virtio-FS does not support live migration. Even when the
> virtiofs directory is not mounted on the guest, VM cannot do live migration.
> Any suggestions/interest If we want to make this work so we can still have
> the live migration feature for other purpose on that VM?
> >
> > If we already have a shared file system like Ceph, does this make it
> different?
>
> (Marc-André: I CCed you regarding D-Bus VMState, see below.)
>
> During the last virtio-fs call on Wednesday we discussed how to allow
> live migration when the file system is not mounted. This means live
> migration of a mounted file system still wouldn't be supported, but at
> least it would be possible to migrate when the file system is not
> mounted.
>
> The easiest way to do that is to hot unplug the vhost-user-fs-pci device
> so that QEMU can live migrate the VM and then hot plug the
> vhost-user-fs-pci device again after migration has completed. This
> requires either manually performing the hot plug operations or some
> automation to do this around live migration.
>
> A smoother approach would be for QEMU to allow live migration when the
> file system is not mounted. Unfortunately it turns out this is actually
> rather involved because virtiofsd runs as a separate process from QEMU
> and the live migration interface for vhost-user devices is currently
> pretty basic
> (https://qemu.readthedocs.io/en/latest/interop/dbus-vmstate.html). There
> is currently no way for virtiofsd to communicate with QEMU to allow/deny
> live migration depending on whether virtiofsd currently has an active
> FUSE session (mount). It might be possible to extend the D-Bus VMState
> interface and implement support for "migration blockers" as they are
> known in QEMU, but I wonder whether it's not better to just work on
> full virtio-fs live migration instead. Either task is complex and I
> wouldn't recommend it as a first contribution to virtiofsd or QEMU.
>
> Can you make use of the hotplug approach to live migration?
>
> Stefan
>