Re: [Qemu-devel] [PATCH] migration: calculate expected_downtime with ram_bytes_remaining()

[Dr. David Alan Gilbert]

* Balamuruhan S (address@hidden) wrote:
> On 2018-04-04 13:36, Peter Xu wrote:
> > On Wed, Apr 04, 2018 at 11:55:14AM +0530, Balamuruhan S wrote:
> > 
> > [...]
> > 
> > > > too. So still I'll put aside the "which one is better" question.
> > > >
> > > > For your use case, you can have a look on either of below way to
> > > > have a converged migration:
> > > >
> > > > - auto-converge: that's a migration capability that throttles CPU
> > > >   usage of guests
> > > 
> > > I used auto-converge option before hand and still it doesn't help
> > > for migration to complete
> > 
> > Have you digged about why?  AFAIK auto-convergence will at last absort
> > merely the whole vcpu resource (99% of them maximum).  Maybe you are
> > not with the best throttle values?  Or do you think that could be a
> > auto-convergence bug too?
> 
> I am not sure, I will work on it to find why.

> > 
> > > 
> > > >
> > > > - postcopy: that'll let you start the destination VM even without
> > > >   transferring all the RAMs before hand
> > > 
> > > I am seeing issue in postcopy migration between POWER8(16M) ->
> > > POWER9(1G)
> > > where the hugepage size is different. I am trying to enable it but
> > > host
> > > start
> > > address have to be aligned with 1G page size in
> > > ram_block_discard_range(),
> > > which I am debugging further to fix it.
> > 
> > I thought the huge page size needs to be matched on both side
> > currently for postcopy but I'm not sure.
> 
> you are right! it should be matched, but we need to support
> POWER8(16M) -> POWER9(1G)
> 
> > CC Dave (though I think Dave's still on PTO).

There's two problems there:
  a) Postcopy with really big huge pages is a problem, because it takes
     a long time to send the whole 1G page over the network and the vCPU
     is paused during that time;  for example on a 10Gbps link, it takes
     about 1 second to send a 1G page, so that's a silly time to keep
     the vCPU paused.

  b) Mismatched pagesizes are a problem on postcopy; we require that the
     whole of a hostpage is sent continuously, so that it can be
     atomically placed in memory, the source knows to do this based on
     the page sizes that it sees.  There are some other cases as well 
     (e.g. discards have to be page aligned.)

Both of the problems are theoretically fixable; but neither case is
easy.
(b) could be fixed by sending the hugepage size back to the source,
so that it knows to perform alignments on a larger boundary to it's
own RAM blocks.

(a) is a much much harder problem; one *idea* would be a major
reorganisation of the kernels hugepage + userfault code to somehow
allow them to temporarily present as normal pages rather than a
hugepage.

Does P9 really not have a hugepage that's smaller than 1G?

Dave

--
Dr. David Alan Gilbert / address@hidden / Manchester, UK