Re: [Qemu-devel] [RFC] New thread for the VM migration

[Juan Quintela]

Avi Kivity <address@hidden> wrote:

>> Disabling hotplug should be enough?
>
> So is powering down the destination host.

O:-)  You see that I explained that later O:-)

>
>> Notice that hotplug/unplug during
>> migration don't make a lot of sense anyways.
>
> That's completely wrong.  Hotplug is a guest/end-user operation;
> migration is a host/admin operation.  The two don't talk to each other
> at all - if the admin (usually a program) wants to migrate at the same
> time the user wants to hotplug, which one gets the bad news?  Who will
> actually test the combination?

I am not sure if it makes sense, but to be able to allow hotplug during
migration we need to change lots of things.  It don't work today either,
so I think it is excesive to fix that on this patch.

> It's true that with the current setup we can't really do migration and
> hotplug together, since we can't synchronize the hotplug on the
> destination with the migration.  But we should be able to, and we
> should design migration with that in mind.

ok then.

>> Not all the bitmap syncying has proper locking now (copyng towards one
>> place), but rest of cade looks really thread safe to me (migration code
>> is only called from this thread, so it should be safe).
>>
>> My understanding on how this work:
>>
>>   vcpu thread modifies momery
>>   iothread (some times) modifies memory
>>
>>   migration thread: reads memory, and gets the lock before syncing its
>>   bitmap with kvm one and qemu one (clearing it on the time).
>>
>> Assume we disable hotplug/unplug (what we have to do anyways).  What is
>> the locking problem that we have?
>
> I didn't really grok the buffering of the migration bitmap.  It does
> look correct.  Would be best in a separate patch to point out the new
> mechanism (but that doesn't really excuse the bad review).

I agree that patch needs to be split in meaningful chunks.  As it is
today, it is difficult to review.

>> We do stage 3 with the iothread locked, i.e. at that point everything
>> else is stopped.  Before stage 3, can kvm or qemu modify a page and
>> _not_ modify the bitmap?  My understanding is not.
>>
>> Only real variable that we are sharing is ram_list, or I am losing
>> something obvious?
>
> You are right.  ram_list _is_ volatile though (but we can't really
> change it these days during migration).

I have think a little bit about hotplug & migration, and haven't arraive
to a nice solution.

- Disabling hotplug/unplug during migration: easy to do.  But it is not
  exactly user friendly (we are here).

- Allowing hotplug during migration. Solutions:
  * allow the transfer of hotplug events over the migration protocol
    (make it even more complicated, but not too much.  The big problem is
    if the hotplug succeeds on the source but not the destination, ...)
  * migrate the device list in stage 3: it fixes the hotplug problem
    nicely, but it makes the interesting problem that after migrating
    all the ram, we can find "interesting" problems like: disk not
    readable, etc.  Not funny.
  * <insert your nice idea here>

As far as I can see, if we sent the device list 1st, we can create the
full machine at destination, but hotplug is "interesting" to manage.
Sending the device list late, solve hotplug, but allows errors after
migrating all memory (also known as, why don't you told me *sooner*).

No clue about a nice & good solution.  For now, I think that we should
go with disabling hotplug/unplug during migration, until we get a better
idea.

Later, Juan.