Re: [Qemu-devel] [PATCH v4 0/9] enable numa configuration before machine_init() from QMP

[David Gibson]

On Tue, Apr 17, 2018 at 04:13:34PM +0200, Markus Armbruster wrote:
> Igor Mammedov <address@hidden> writes:
> 
> [...]
> > Series allows to configure NUMA mapping at runtime using QMP
> > interface. For that to happen it introduces a new '-preconfig' CLI option
> > which allows to pause QEMU before machine_init() is run and
> > adds new set-numa-node QMP command which in conjunction with
> > query-hotpluggable-cpus allows to configure NUMA mapping for cpus.
> >
> > Later we can modify other commands to run early, for example device_add.
> > I recall SPAPR had problem when libvirt started QEMU with -S and, while it's
> > paused, added CPUs with device_add. Intent was to coldplug CPUs (but at that
> > stage it's considered hotplug already), so SPAPR had to work around the 
> > issue.
> 
> That instance is just stupidity / laziness, I think: we consider any
> plug after machine creation a hot plug.  Real machines remain cold until
> you press the power button.  Our virtual machines should remain cold
> until they start running, i.e. with -S until the first "cont".

Makes sense to me.  As I recall, the chief problem I had here with
PAPR hotplug was that -S stopped *after* the machine_reset, which
meant the only mechanisms we really had to inform the guest of the
hardware were the hotplug mechanisms.  Sort of.  It got complicated
because of the feature negotiation system that PAPR guests have.

At present -S doesn't really operate like a "stop before hitting the
power", it instead acts more like a breakpoint on the first
instruction of the firmware.

If we were to move the -S stop to before the machine_reset, I believe
that might simplify several things in our hotplug handling code for
papr.

> I vaguely remember me asking this before, but your answer didn't make it
> into this cover letter, which gives me a pretext to ask again instead of
> looking it up in the archives: what exactly prevents us from keeping the
> machine cold enough for numa configuration until the first "cont"?
> 
> >
> > Example of configuration session:
> > $QEMU -smp 2 -preconfig ...
> >
> > QMP:
> > # get CPUs layout for current target/machine/CLI
> > -> {'execute': 'query-hotpluggable-cpus' }  
> > <- {'return': [
> >        {'props': {'core-id': 0, 'thread-id': 0, 'socket-id': 1}, ... },
> >        {'props': {'core-id': 0, 'thread-id': 0, 'socket-id': 0}, ... }
> >    ]}
> >
> > # configure 1st node
> > -> {'execute': 'set-numa-node', 'arguments': { 'type': 'node', 'nodeid': 0 
> > } }  
> > <- {'return': {}}
> > -> {'execute': 'set-numa-node', 'arguments': { 'type': 'cpu',   
> >        'node-id': 0, 'core-id': 0, 'thread-id': 0, 'socket-id': 1, }
> >    }
> > <- {'return': {}}
> >
> > # configure 2nd node
> > -> {'execute': 'set-numa-node', 'arguments': { 'type': 'node', 'nodeid': 1 
> > } }
> > -> {'execute': 'set-numa-node', 'arguments': { 'type': 'cpu',  
> >        'node-id': 1, 'core-id': 0, 'thread-id': 0, 'socket-id': 0 }
> >    }
> > <- {'return': {}}
> >
> > # [optional] verify configuration
> > -> {'execute': 'query-hotpluggable-cpus' }  
> > <- {'return': [
> >        {'props': {'core-id': 0, 'thread-id': 0, 'node-id': 0, 'socket-id': 
> > 1}, ... },
> >        {'props': {'core-id': 0, 'thread-id': 0, 'node-id': 1, 'socket-id': 
> > 0}, ... }
> >    ]}
> >
> >
> > Git tree:
> >     https://github.com/imammedo/qemu.git qmp_preconfig_v3
> >
> > Ref to v1:
> >     https://lists.gnu.org/archive/html/qemu-devel/2017-10/msg03583.html
> >     Message-Id: <address@hidden>
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature