Re: [Qemu-devel] [PATCH for-2.10 22/23] numa: add '-numa cpu, ...' option for property based node mapping

[David Gibson]

On Wed, Mar 29, 2017 at 02:08:58PM +0200, Igor Mammedov wrote:
> On Wed, 29 Mar 2017 13:27:23 +1100
> David Gibson <address@hidden> wrote:
> 
> > On Tue, Mar 28, 2017 at 01:09:11PM +0200, Igor Mammedov wrote:
> > > On Tue, 28 Mar 2017 16:16:02 +1100
> > > David Gibson <address@hidden> wrote:
> > >   
> > > > On Wed, Mar 22, 2017 at 02:32:47PM +0100, Igor Mammedov wrote:  
> > > > > legacy cpu to node mapping is using cpu index values to map
> > > > > VCPU to node with help of '-numa node,nodeid=node,cpus=x[-y]'
> > > > > option. However cpu index is internal concept and QEMU users
> > > > > have to guess /reimplement qemu's logic/ to map it to
> > > > > a concrete cpu socket/core/thread to make sane CPUs
> > > > > placement across numa nodes.
> > > > > 
> > > > > This patch allows to map cpu objects to numa nodes using
> > > > > the same properties as used for cpus with -device/device_add
> > > > > (socket-id/core-id/thread-id/node-id).
> > > > > 
> > > > > At present valid properties/values to address CPUs could be
> > > > > fetched using hotpluggable-cpus monitor/qmp command, it will
> > > > > require user to start qemu twice when creating domain to fetch
> > > > > possible CPUs for a machine type/-smp layout first and
> > > > > then the second time with numa explicit mapping for actual
> > > > > usage. The first step results could be saved and reused to
> > > > > set/change mapping later as far as machine type/-smp stays
> > > > > the same.
> > > > > 
> > > > > Proposed impl. supports exact and wildcard matching to
> > > > > simplify CLI and allow to set mapping for a specific cpu
> > > > > or group of cpu objects specified by matched properties.
> > > > > 
> > > > > For example:
> > > > > 
> > > > >    # exact mapping x86
> > > > >    -numa cpu,node-id=x,socket-id=y,core-id=z,thread-id=n
> > > > > 
> > > > >    # exact mapping SPAPR
> > > > >    -numa cpu,node-id=x,core-id=y
> > > > > 
> > > > >    # wildcard mapping, all cpu objects that match socket-id=y
> > > > >    # are mapped to node-id=x
> > > > >    -numa cpu,node-id=x,socket-id=y
> > > > > 
> > > > > Signed-off-by: Igor Mammedov <address@hidden>  
> > > > 
> > > > What's the rationale for adding a new CLI, rather than adding node-id
> > > > properties to the appropriate objects with -device, -global or -set as
> > > > appropriate?  
> > >  '-global' applies to all cpus, while '-device,-set' applies to present
> > >  at boot time cpus only. So they do not work for the case of possible but
> > >  not present at boot time objects.  
> > 
> > Ah!  Of course.
> > 
> > > For ACPI based targets, we need to have
> > >  numa mapping at boot time to build ACPI SRAT table.
> > >  I don't know if it's important for spapr/fdt,  
> > 
> > Not in the same way.  For spapr the device tree fragment for the new
> > cpu is supplied to the guest at hotplug time rather than having to be
> > in the initial device tree.  So for us, node could be supplied with
> > device_add.
> I've implemented cpu.node-id check in the same way for all targets
> for spapr it's patch patch 06/23 which forces cpu.node-id to match
> whatever mapping has been provided with -numa cpu[s]
> OR
> with implied default /0/ if mapping for cpu hasn't been specified
> with -numa explicitly.
> 
> That way it won't break legacy machines and on compat code is necessary,
> I'd would leave it up to you with patch on top of this to lift 
> restriction/make
> it more relaxed for spapr if you think it won't break anything.
> 
> Although from libvirt pov, I'd prefer to treat all targets uniformly,
> which narrows choice down to '-numa' mapping approach that it uses
> now.

Yeah, that makes sense.  If we ever have a compelling reason to allow
node designation at device_add time on Power, we can relax the
restrictions then.  I doubt it will ever happen.

> 
> > > but it uses current predefined
> > >  mapping with -numa node,cpus=x-y and new CLI hides from user internal
> > >  cpu_index and allows to use the same properties as we use for -device 
> > > cpu,...
> > >  to define mapping to numa nodes for present/possible cpus.
> ...
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature