[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty
From: |
Chunguang Li |
Subject: |
Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent |
Date: |
Mon, 26 Sep 2016 22:55:01 +0800 (GMT+08:00) |
> -----原始邮件-----
> 发件人: "Dr. David Alan Gilbert" <address@hidden>
> 发送时间: 2016年9月26日 星期一
> 收件人: "Chunguang Li" <address@hidden>
> 抄送: address@hidden, address@hidden, address@hidden, address@hidden,
> address@hidden
> 主题: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty
> after they have been sent
>
> * Chunguang Li (address@hidden) wrote:
> > Hi all!
> > I have some confusion about the dirty bitmap during migration. I have
> > digged into the code. I figure out that every now and then during
> > migration, the dirty bitmap will be grabbed from the kernel space through
> > ioctl(KVM_GET_DIRTY_LOG), and then be used to update qemu's dirty bitmap.
> > However I think this mechanism leads to resendness of some NON-dirty pages.
> >
> > Take the first iteration of precopy for instance, during which all the
> > pages will be sent. Before that during the migration setup, the
> > ioctl(KVM_GET_DIRTY_LOG) is called once, so the kernel begins to produce
> > the dirty bitmap from this moment. When the pages "that haven't been sent"
> > are written, the kernel space marks them as dirty. However I don't think
> > this is correct, because these pages will be sent during this and the next
> > iterations with the same content (if they are not written again after they
> > are sent). It only makes sense to mark the pages which have already been
> > sent during one iteration as dirty when they are written.
> >
> >
> > Am I right about this consideration? If I am right, is there some advice to
> > improve this?
>
> I think you're right that this can happen; to clarify I think the
> case you're talking about is:
>
> Iteration 1
> sync bitmap
> start sending pages
> page 'n' is modified - but hasn't been sent yet
> page 'n' gets sent
> Iteration 2
> sync bitmap
> 'page n is shown as modified'
> send page 'n' again
>
Yes,this is right the case I am talking about.
> So you're right that is wasteful; I guess it's more wasteful
> on big VMs with slow networks where the length of each iteration
> is large.
I think this is "very" wasteful. Assume the workload writes the pages dirty
randomly within the guest address space, and the transfer speed is constant.
Intuitively, I think nearly half of the dirty pages produced in Iteration 1 is
not really dirty. This means the time of Iteration 2 is double of that to send
only really dirty pages.
Thanks,
Chunguang
>
> Fixing it is not easy, because you have to be really careful
> never to miss a page modification, even if the page is sent
> about the same time it's dirtied.
>
> One way would be to sync the dirty log from the kernel
> in smaller chunks.
>
> Dave
>
>
> >
> > Thanks,
> > Chunguang Li
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK
--
Chunguang Li, Ph.D. Candidate
Wuhan National Laboratory for Optoelectronics (WNLO)
Huazhong University of Science & Technology (HUST)
Wuhan, Hubei Prov., China