Re: [Qemu-devel] [PATCH for 2.13 v3 1/2] spapr: Add ibm, max-associativity-domains property

[David Gibson]

On Tue, Apr 17, 2018 at 09:28:42AM +0530, Bharata B Rao wrote:
> On Mon, Apr 16, 2018 at 07:47:29PM +0300, Serhii Popovych wrote:
> > Bharata B Rao wrote:
> > > On Wed, Apr 11, 2018 at 02:41:59PM -0400, Serhii Popovych wrote:
> > >> Now recent kernels (i.e. since linux-stable commit a346137e9142
> > >> ("powerpc/numa: Use ibm,max-associativity-domains to discover possible 
> > >> nodes")
> > >> support this property to mark initially memory-less NUMA nodes as 
> > >> "possible"
> > >> to allow further memory hot-add to them.
> > >>
> > >> Advertise this property for pSeries machines to let guest kernels detect
> > >> maximum supported node configuration and benefit from kernel side change
> > >> when hot-add memory to specific, possibly empty before, NUMA node.
> > >>
> > >> Signed-off-by: Serhii Popovych <address@hidden>
> > >> ---
> > >>  hw/ppc/spapr.c | 10 ++++++++++
> > >>  1 file changed, 10 insertions(+)
> > >>
> > >> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > >> index a81570e..c05bbad 100644
> > >> --- a/hw/ppc/spapr.c
> > >> +++ b/hw/ppc/spapr.c
> > >> @@ -910,6 +910,13 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, 
> > >> void *fdt)
> > >>          0, cpu_to_be32(SPAPR_MEMORY_BLOCK_SIZE),
> > >>          cpu_to_be32(max_cpus / smp_threads),
> > >>      };
> > >> +    uint32_t maxdomains[] = {
> > >> +        cpu_to_be32(4),
> > >> +        cpu_to_be32(0),
> > >> +        cpu_to_be32(0),
> > >> +        cpu_to_be32(0),
> > >> +        cpu_to_be32(nb_numa_nodes - 1),
> > >> +    };
> > >>
> > >>      _FDT(rtas = fdt_add_subnode(fdt, 0, "rtas"));
> > >>
> > >> @@ -946,6 +953,9 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, 
> > >> void *fdt)
> > >>      _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points",
> > >>                       refpoints, sizeof(refpoints)));
> > >>
> > >> +    _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains",
> > >> +                     maxdomains, sizeof(maxdomains)));
> > >> +
> > >>      _FDT(fdt_setprop_cell(fdt, rtas, "rtas-error-log-max",
> > >>                            RTAS_ERROR_LOG_MAX));
> > >>      _FDT(fdt_setprop_cell(fdt, rtas, "rtas-event-scan-rate",
> > > 
> > > This commit causes hash guest with latest guest kernel to hang at early 
> > > boot.
> > 
> > I use v4.16 tag from stable and can't reproduce on P8 machine reported
> > issue.
> > 
> > Could you please share more details about your setup, kernel commit id
> > you spot problem?
> 
> I am on 4.16.0-rc7 (commit id: 0b412605ef5f)
> 
> BTW this happens only for non-NUMA guest.

Ah, that might explain it.  With no NUMA nodes specified, I think this
code will put a -1 into the max-associativity-domains property, which
is probably causing the mess.  If we don't have NUMA (nb_numa_nodes ==
0) we probably want to either omit the property entirely, or clamp
that 5th cell to 0.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature