Re: [Qemu-devel] [RFC v2 0/4] enable numa configuration before machine is running from HMP/QMP

[David Gibson]

On Thu, Dec 28, 2017 at 06:22:55PM +0100, Igor Mammedov wrote:
> 
> As were suggested at (1) and at bof session where we discussed subj,
> I'm posting variant with late numa 'configuration' i.e. when QEMU is
> started with '-S' option in paused state and numa is configured via
> monitor/QMP before machine cpus are allowed to run.
> 
> Suggested idea was to try 'late' numa configuration as it might result in
> shortcut approach allowing us reuse current pause point (-S) versus adding
> another preconfig option with earlier pause point.
> So this series tries to show how feasible this approach.
> 
> Currently numa options mainly affect only firmware blobs (ACPI/FDT tables),
> it should have been possible to regenerate those blobs right before we start
> CPUs, which would allow us setup numa configuration at first pause point and
> get firmware blobs with updated numa information.
> 
> Series implements idea for x86 ans spapr machines and uses machine reset,
> to reconfigure firmware and other machine structures after each numa
> configuration command (HMP or QMP).
> 
> It was relatively not hard to implement for above machines as they already
> rebuild firmware blobs at reset time. But it still was a pain as QEMU isn't
> written with dynamic reconfiguration in mind and one need to update device
> state with new data (I think I've got it right but not 100% sure)
> 
> However when it comes to the last target supporting NUMA, ARM
> all simplification versus v1 goes down the drain, since FDT blob is build
> incrementally during machine_init(), -device, machine_done() time, and
> it turns out into huge refactoring to isolate scattered FDT pieces into
> single FDT build function (like we do for ACPI). It's job that we would need
> to do anyways for hotplug to work properly on ARM,

Kind of irrelevant to this series, but I agree.  pseries started out
with the FDT being almost static created at init time, with a few tiny
adjustments later on.  But as the platform developed we needed to move
more and more of the FDT generation to later on (reset time, roughly).
For a long time we had an ugly split between the "skeleton" built at
init time and the stuff built at reset time, until I eventually moved
it all to reset time.

I'm pretty sure ARM will want the same thing, for hotplug as you
mention, but also for other things.  I also think it'll save effort
over all to do it sooner rather than later.

I had stuff in the works for ages to make DT building easier,
including a full "live" DT model for qemu (fdt is a good format for
passing the DT from one unit to another, but it gets clunky to do lots
of manipulation with it).  Unfortunately I've been sufficiently busy
with other things that I haven't really gotten anywhere with that for
the last year or more.

> but I don't think it
> should get in the way of numa refactoring.
> So that was the point where I gave up and decided to post only x86/spapr
> pieces for demo purposes.

Fair enough.

> 
> I'm inclined towards avoiding 'v2 shortcut' and going in direction of v1,
> as I didn't see v2 as the right way in general, since one would have to:
>   - build machine / connect / initalize / devices one way and then find out
>     devices / connections that need to be fixed/updated with new 
> configuration,
>     it's very fragile and easy break.
> 
> If I remember correctly the bof session, consensus was that we would like to 
> have
> early configuration interface (like v1) in the end, so I'd rather send time
> on addressing v1 drawbacks instead of hacking machine init order to make numa 
> work
> in backwards way.
> 
> CC: address@hidden
> CC: address@hidden
> CC: address@hidden
> CC: address@hidden
> CC: address@hidden
> CC: address@hidden
> CC: address@hidden
> 
> [1]
> v1 for reference:
> [Qemu-devel] [RFC 0/6] enable numa configuration before machine_init() from 
> HMP/QMP
>     https://lists.nongnu.org/archive/html/qemu-devel/2017-10/msg03583.html
> 
> PS:
> exercise wasn't waste as it resulted in cleanups that were already merged.
> 
> 
> Igor Mammedov (4):
>   numa: split out NumaOptions parsing into parse_NumaOptions()
>   HMP: add set-numa-node command
>   QMP: add set-numa-node command
>   numa: pc: reset machine if numa config has changed in prelaunch time
> 
>  hmp.h                 |  1 +
>  include/hw/boards.h   |  1 +
>  include/sysemu/numa.h |  1 +
>  hmp-commands.hx       | 13 +++++++++++
>  hmp.c                 | 23 +++++++++++++++++++
>  hw/core/machine.c     |  3 ++-
>  hw/i386/pc.c          |  1 +
>  numa.c                | 63 
> +++++++++++++++++++++++++++++++++++----------------
>  qapi-schema.json      | 13 +++++++++++
>  vl.c                  |  4 ++++
>  10 files changed, 102 insertions(+), 21 deletions(-)
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature