Re: [Qemu-devel] [RFC] qmp interface for save vmstate to image

[Stefan Hajnoczi]

On Thu, Mar 21, 2013 at 12:48:35PM +0100, Pavel Hrdina wrote:
> On 03/21/2013 07:43 AM, Wenchao Xia wrote:
> > Hi, Pavel
> >    Sorry for late response.
> 
> np :)
> 
> >> Hi Wenchao,
> >>
> >> It seems the we are working on the same thing. You are trying to improve
> >> the size of vmstate if you want to save it to file or as an internal
> >> snapshot.
> >>
> >> I'm also working on that issue and I think that my solution could be
> >> also used for savevm to external file or for live backup.
> >>
> >> Here is my proposal how to do it:
> >>
> >> We will not have the fixed size of vmstate, we will have the possible
> >> minimal size of the vmstate. I will also use the migration code to save
> >> the vmstate.
> >>
> >    It is good if speed and size can be improved, but IMHO the size will
> > be a problem. Predictable or fixed size ensure management stack to
> > give assess and decision, preserve resource ahead, personally I
> > does not like a process continue to take resource without limit,
> > in most case I'll turn it off.... By using qcow2m vmstate will have a
> > fixed MAX size, ideal to be used to take it as a backup data.
> >    Above is my personal opinion, and I do want to know the maintainer's
> > opinion to decide whether to continue.
> 
> I mean that the vmstate size would by at max the same as the guest ram
> size, but could be smaller. I also dislike that actually the vmstate
> could be much more larger then the guest ram size.
> 
> > 
> >> In the qemu_savevm_state_begin we will create bitmap for all ram pages.
> >> Then we set all pages in bitmap to "1" and it means we need to save them
> >> all. Then we check all ram pages for duplicated pages and we will unset
> >> all duplicated pages from "savevm_bitmap".
> >>
> >> In the qemu_savevm_state_iterate we will start saving remaining ram
> >> pages according to "savevm_bitmap". Because the guest is running, it
> >> could change the data in ram pages which is still not saved. For this
> >> case we also have to create a priority queue. Into this priority queue
> >> we will copy every ram page before it will be changed and also remove
> >> this ram page from savevm_bitmap. In the iterate cycle we will at first
> >> handle the priority queue and then continue to other ram pages from the
> >> savevm_bitmap.
> >>
> >    OK, I got your idea: intercept the page writing before it changes.
> > I think this could reduce time in savevm. But some problems need to be
> > confirmed:
> > 1 is it workable when KVM is used? In my understanding KVM will directly
> > change the ram page before qemu can take over.
> 
> Yes, this is true. I'm now investigating any way how to do this, but I'm
> afraid that without support from kvm kernel module it cannot be done.
> 
> > 2 the performance sacrifice of running guest, need a test.
> 
> Surly I'll test it if there will be some solution how to copy the page
> before it is changed when kvm is used.

There already is a guest RAM cloning mechanism: fork the QEMU process.
Then you have a copy-on-write guest RAM.

In a little more detail:

1. save non-RAM device state
2. quiesce QEMU to a state that is safe for forking
3. create an EventNotifier for live savevm completion signal
4. fork and pass completion EventNotifier to child
5. parent continues running VM
6. child performs vmsave of copy-on-write guest RAM
7. child signals completion EventNotifier and terminates
8. parent raises live savevm completion QMP event

Stefan