Re: [PATCH RFC 14/15] migration: Postcopy preemption on separate channel

[Dr. David Alan Gilbert]

* Peter Xu (peterx@redhat.com) wrote:
> On Tue, Feb 08, 2022 at 11:24:14AM +0000, Dr. David Alan Gilbert wrote:
> > > The current model is we only have 1 postcopy channel and 1 precopy 
> > > channel, but
> > > it should be easier if we want to make it N post + 1 pre base on this 
> > > series.
> > 
> > It's not clear to me if we need to be able to do N post + M pre, or
> > whether we have a rule like always at least 1 post, but if there's more
> > pagefaults in the queue then you can steal all of the pre channels.
> 
> Right, >1 queue length should easily happen with workloads in real cloud
> environment.  Though even with only 1post channel we can already hit at least
> <~1ms with this series even if there're 16 pending requests per my test.  I
> think that may cover quite some real workloads.
> 
> One thing to mention is that we should always assume the pre-channels are
> filled up with tons of pages already in the NIC send buffer, so they won't be
> good candidate for postcopy requests, IMHO.  So I'm not sure whether we can
> mixly use the pre/post channels - we may need to leave the post channels idle.

No I'm not sure either; even with separate channels do we have problems
with contention on the NIC?

Dave

> Then, if we keep some of the multifd channels idle, then it will become some
> other thing rather than the existing multifd, since we will start to treat
> threads and channels differently and break the "equality" rule in the strict
> version of multifd world.
> 
> > > This also reminded me that, instead of a new capability, should I simply 
> > > expose
> > > a parameter "postcopy-channels=N" to CLI so that we can be prepared with 
> > > multi
> > > postcopy channels?
> > 
> > I'm not sure we know enough yet about what configuration it would have;
> > I'd be tempted to just make it work for the user by enabling both
> > multifd and preemption and then using this new mechanism rather than
> > having to add yet another parameter.
> 
> Let me stick with the current capability bit then, so as to make it 
> 1pre+1post.
> And we can leave Npre+1post for later.
> 
> Thanks,
> 
> -- 
> Peter Xu
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK