[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: vhost-user (virtio-fs) migration: back end state
From: |
Juan Quintela |
Subject: |
Re: vhost-user (virtio-fs) migration: back end state |
Date: |
Tue, 07 Feb 2023 16:13:21 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) |
Hanna Czenczek <hreitz@redhat.com> wrote:
> On 06.02.23 22:02, Juan Quintela wrote:
>> Stefan Hajnoczi <stefanha@gmail.com> wrote:
>>> On Mon, 6 Feb 2023 at 07:36, Hanna Czenczek <hreitz@redhat.com> wrote:
>>>> Hi Stefan,
>>>>
>>>> For true virtio-fs migration, we need to migrate the daemon’s (back
>>>> end’s) state somehow. I’m addressing you because you had a talk on this
>>>> topic at KVM Forum 2021. :)
>>>>
>>>> As far as I understood your talk, the only standardized way to migrate a
>>>> vhost-user back end’s state is via dbus-vmstate. I believe that
>>>> interface is unsuitable for our use case, because we will need to
>>>> migrate more than 1 MB of state. Now, that 1 MB limit has supposedly
>>>> been chosen arbitrarily, but the introducing commit’s message says that
>>>> it’s based on the idea that the data must be supplied basically
>>>> immediately anyway (due to both dbus and qemu migration requirements),
>>>> and I don’t think we can meet that requirement.
>>> Yes, dbus-vmstate is the available today. It's independent of
>>> vhost-user and VIRTIO.
>> Once that we are here:
>> - typical size of your starte (either vhost-user or whatever)
>
> Difficult to say, completely depends on the use case. When
> identifying files by path and organizing them in a tree structure,
> probably ~48 bytes per indexed file, plus, say, 16 bytes per open
> file.
>
> So for a small shared filesystem, the state can be very small, but
> we’ll also have to prepare for cases where it is in the range of
> several MB.
That is not two bad. Anything below a few tens megabytes is easy to
manage. Anything in the hundred of megabytes or more really need thought.
> The main problem isn’t size but that (when identifying files by path)
> we’ll probably want to construct the paths when migrating, which won’t
> be done instantaneously.
>
>> - what are the posibilities that you can enter the iterative stage
>> negotiation (i.e. that you can create a dirty bitmap about your state)
>
> Very good. We should know when parts of the state (information about
> a specific indexed or open file) changes. (Exceptions apply, but they
> mostly come down to whether indexed files are identified by path or
> file handle, which is a choice the user will probably need to make.
> Either one comes with caveats.)
That is good.
>> How much is migration to file used in practice?
>> I would like to have some information here.
>> It could be necessary probably to be able to encrypt it. And that is a
>> (different) whole can of worms.
>
> I don’t think virtio-fs state needs to be encrypted any more than any
> other state. It’ll basically just map FUSE inode IDs to a file in the
> shared directory, either via path or file handle; and then also track
> open(2) flags for opened files. (At least that’s what’s currently on
> my radar.) That information should actually be replicated in the
> guest, too (because it too will have mapped the filesystem paths to
> FUSE inode IDs), so isn’t more security relevant than guest memory.
Oh, that was not about virtio-fs at all.
Is because you talked about file migration.
Right now, we need to use exec migration to do this. but it is clearly
suboptimal. Basically we just do a normal migration, but that means
that we have a lot of duplicated pages on the destination.
But we can do better. Just create a file that is as big as the memory,
and write everypage in its own place. So loading is going to be really
fast. (yes holes on RAM is a different issue, but we can ignore that
for now).
And the other thing is that we have to really encrypt it somehow, so I
guess that a block cipher should work, but encryption is not my field of
expertise at all.
In vhost-user-fs case, I fully agree with you that if you are
"exporting" part of the local filesystem, encryption don't buy you
anything.
Later, Juan.
- vhost-user (virtio-fs) migration: back end state, Hanna Czenczek, 2023/02/06
- Re: vhost-user (virtio-fs) migration: back end state, Hanna Czenczek, 2023/02/07
- Re: vhost-user (virtio-fs) migration: back end state, Stefan Hajnoczi, 2023/02/07
- Re: vhost-user (virtio-fs) migration: back end state, Stefan Hajnoczi, 2023/02/08
- Re: vhost-user (virtio-fs) migration: back end state, Michael S. Tsirkin, 2023/02/08
- Re: vhost-user (virtio-fs) migration: back end state, Hanna Czenczek, 2023/02/08
- Re: vhost-user (virtio-fs) migration: back end state, Stefan Hajnoczi, 2023/02/08