Re: vhost-user (virtio-fs) migration: back end state

[Juan Quintela]

Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Mon, 6 Feb 2023 at 16:02, Juan Quintela <quintela@redhat.com> wrote:
>> The last two bits are on my ToDo list for the near future, but not done.
>>
>> If we ended having lots of so big devices, we are going to have to think
>> about downtimes in the order of dozens of seconds, not subsecond.
>>
>> So, if you are planning doing this in the near future, this is a good
>> time to discuss this.
>
> Can you explain the dirty bitmap requirement you mentioned further, given 
> that:
> 1. vhost-user has dirty memory logging already, so DMA is covered.
> 2. An iterative interface allows the device to keep generating more
> state, so does QEMU really need to know if parts of the previously
> emitted binary blob have become dirty? It might allow QEMU to minimize
> the size of the savevm file, but only if the overwritten data has the
> same size.
>
> Is a dirty bitmap for the device state necessary?

Not for that.  My fault.  I was talking about a dirty bitmap because
that is how it is implemented in vfio.  Notice that qemu don't see that
bitmap.  But the device can enter the iterative stage.

The reason that I ask is if you can enter the iterative stage to send
stuff before the last stage.

If you don't have something similar to that, you need to send that in
one go, and that is going to take really a lot of time.

When we are talking about NVidia GPU's, they have two kinds of state:
- Frame buffer: they can (or at some point would have) a dirty bitmap
  and they can enter the iterative stage.
- They have internal state that is not visible to the user, that state
  is big (around 1GB) and they can't enter the iterative stage, because
  they can't know if the guest, the card, or whoever has changed that.

For vhost device:
- what is the "typical" amount of state.
- if it is more than a few megabytes, is there a way to send parts of it
  before the ending stage?  That is what I asked for a dirty bitmap for
  it, but it can be anything else.  Just that vhost-user keep track of
  what has sent to the other side and then have less state for the last
  stage?
  

Later, Juan.