qemu-ppc
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-ppc] [Qemu-devel] [RFC PATCH v2 2/2] spapr: Memory hot-unplug


From: Michael Roth
Subject: Re: [Qemu-ppc] [Qemu-devel] [RFC PATCH v2 2/2] spapr: Memory hot-unplug support
Date: Wed, 27 Apr 2016 14:07:10 -0500
User-agent: alot/0.3.6

Quoting Igor Mammedov (2016-04-27 09:34:53)
> On Wed, 27 Apr 2016 15:59:52 +0200
> Thomas Huth <address@hidden> wrote:
> 
> > On 27.04.2016 15:37, Igor Mammedov wrote:
> > > On Tue, 26 Apr 2016 16:03:37 -0500
> > > Michael Roth <address@hidden> wrote:
> > >   
> > >> Quoting Igor Mammedov (2016-04-26 02:52:36)  
> > >>> On Tue, 26 Apr 2016 10:39:23 +0530
> > >>> Bharata B Rao <address@hidden> wrote:
> > >>>     
> > >>>> On Mon, Apr 25, 2016 at 11:20:50AM +0200, Igor Mammedov wrote:    
> > >>>>> On Wed, 16 Mar 2016 10:11:54 +0530
> > >>>>> Bharata B Rao <address@hidden> wrote:
> > >>>>>       
> > >>>>>> On Wed, Mar 16, 2016 at 12:36:05PM +1100, David Gibson wrote:      
> > >>>>>>> On Tue, Mar 15, 2016 at 10:08:56AM +0530, Bharata B Rao wrote:      
> > >>>>>>>   
> > >>>>>>>> Add support to hot remove pc-dimm memory devices.
> > >>>>>>>>
> > >>>>>>>> Signed-off-by: Bharata B Rao <address@hidden>        
> > >>>>>>>
> > >>>>>>> Reviewed-by: David Gibson <address@hidden>
> > >>>>>>>
> > >>>>>>> Looks correct, but again, needs to wait on the PAPR change.      
> > >>>>> [...]      
> > >>>>>>
> > >>>>>> While we are here, I would also like to get some opinion on the real
> > >>>>>> need for memory unplug. Is there anything that memory unplug gives us
> > >>>>>> which memory ballooning (shrinking mem via ballooning) can't give ?  
> > >>>>>>     
> > >>>>> Sure ballooning can complement memory hotplug but turning it on would
> > >>>>> effectively reduce hotplug to balloning as it would enable overcommit
> > >>>>> capability instead of hard partitioning pc-dimms provides. So one
> > >>>>> could just use ballooning only and not bother with hotplug at all.
> > >>>>>
> > >>>>> On the other hand memory hotplug/unplug (at least on x86) tries
> > >>>>> to model real hardware, thus removing need in paravirt ballooning
> > >>>>> solution in favor of native guest support.      
> > >>>>
> > >>>> Thanks for your views.
> > >>>>     
> > >>>>>
> > >>>>> PS:
> > >>>>> Guest wise, currently hot-unplug is not well supported in linux,
> > >>>>> i.e. it's not guarantied that guest will honor unplug request
> > >>>>> as it may pin dimm by using it as a non migratable memory. So
> > >>>>> there is something to work on guest side to make unplug more
> > >>>>> reliable/guarantied.      
> > >>>>
> > >>>> In the above scenario where the guest doesn't allow removal of certain
> > >>>> parts of DIMM memory, what is the expected behaviour as far as QEMU
> > >>>> DIMM device is concerned ? I seem to be running into this situation
> > >>>> very often with PowerPC mem unplug where I am left with a DIMM device
> > >>>> that has only some memory blocks released. In this situation, I would 
> > >>>> like
> > >>>> to block further unplug requests on the same device, but QEMU seems
> > >>>> to allow more such unplug requests to come in via the monitor. So
> > >>>> qdev won't help me here ? Should I detect such condition from the
> > >>>> machine unplug() handler and take required action ?    
> > >>> I think offlining is a guests task along with recovering from
> > >>> inability to offline (i.e. offline all + eject or restore original 
> > >>> state).
> > >>> QUEM does it's job by notifying guest what dimm it wants to remove
> > >>> and removes it when guest asks it (at least in x86 world).    
> > >>
> > >> In the case of pseries, the DIMM abstraction isn't really exposed to
> > >> the guest, but rather the memory blocks we use to make the backing
> > >> memdev memory available to the guest. During unplug, the guest
> > >> completely releases these blocks back to QEMU, and if it can only
> > >> release a subset of what's requested it does not attempt to recover.
> > >> We can potentially change that behavior on the guest side, since
> > >> partially-freed DIMMs aren't currently useful on the host-side...
> > >>
> > >> But, in the case of pseries, I wonder if it makes sense to maybe go
> > >> ahead and MADV_DONTNEED the ranges backing these released blocks so the
> > >> host can at least partially reclaim the memory from a partially
> > >> unplugged DIMM?  
> > > It's a little bit confusing, one asked to remove device but it's still
> > > there but not completely usable/available.
> > > What will happen when user wants that memory plugged back?  
> > 
> > As far as I've understood MADV_DONTNEED, you can use the memory again at
> > any time - just the previous contents will be gone, which is ok in this
> > case since the guest previously marked this area as unavailable.
> If host gave returned memory to someone else there might not be enough
> resources to give it back (what would happen I can't tell may be VM will
> stall or just get exception).

It's not really an issue for pseries, since once the LMB is released
it's totally gone as far as the guest is concerned, and there's no
way to plug it back in via the still-present DIMM until removal
completes after, say, reset time.

But, either way, I agree if we'll intend to let the guest recover, it
would be immediately upon being unable to satisfy the whole unplug and
not some future time.

> 
> Anyhow I'd suggest ballooning if one needs partial unplug and fix
> physical unplug to unplug whole pc-dimm or none instead of
> turning pc-dimm device model into some hybrid with balloon device
> and making users/mgmt even more confused.

That seems reasonable, I can see why recovering memory from partially
removed DIMMs overlaps a lot with the ballooning use case...

But I think that kind of leaves the question of how to make memory
unplug useful in practice? In practice, memory unplug seems quite
likely to fail in all-or-nothing scenarios. So if we expect
all-or-nothing removal in the guest, then it seems like some work
needs to be done with the balloon driver or elsewhere to provide the
sort of specificity management would need to know to determine if a
DIMM has become fully unpluggable, and let the guest make ballooning
decisions that complement eventual DIMM unplug more effectively.

> 
> > 
> >  Thomas
> > 
> 




reply via email to

[Prev in Thread] Current Thread [Next in Thread]