Re: [PATCH v8 00/21] Introduce smp.modules for x86 in QEMU

[Daniel P . Berrangé]

On Fri, Feb 02, 2024 at 12:10:58AM +0800, Zhao Liu wrote:
> Hi Daniel,
> 
> On Thu, Feb 01, 2024 at 09:21:48AM +0000, Daniel P. Berrangé wrote:
> > Date: Thu, 1 Feb 2024 09:21:48 +0000
> > From: "Daniel P. Berrangé" <berrange@redhat.com>
> > Subject: Re: [PATCH v8 00/21] Introduce smp.modules for x86 in QEMU
> > 
> > On Thu, Feb 01, 2024 at 10:57:32AM +0800, Zhao Liu wrote:
> > > Hi Daniel,
> > > 
> > > On Wed, Jan 31, 2024 at 10:28:42AM +0000, Daniel P. Berrangé wrote:
> > > > Date: Wed, 31 Jan 2024 10:28:42 +0000
> > > > From: "Daniel P. Berrangé" <berrange@redhat.com>
> > > > Subject: Re: [PATCH v8 00/21] Introduce smp.modules for x86 in QEMU
> > > > 
> > > > On Wed, Jan 31, 2024 at 06:13:29PM +0800, Zhao Liu wrote:
> > > > > From: Zhao Liu <zhao1.liu@intel.com>
> > > 
> > > [snip]
> > > 
> > > > > However, after digging deeper into the description and use cases of
> > > > > cluster in the device tree [3], I realized that the essential
> > > > > difference between clusters and modules is that cluster is an 
> > > > > extremely
> > > > > abstract concept:
> > > > >   * Cluster supports nesting though currently QEMU doesn't support
> > > > >     nested cluster topology. However, modules will not support 
> > > > > nesting.
> > > > >   * Also due to nesting, there is great flexibility in sharing 
> > > > > resources
> > > > >     on clusters, rather than narrowing cluster down to sharing L2 (and
> > > > >     L3 tags) as the lowest topology level that contains cores.
> > > > >   * Flexible nesting of cluster allows it to correspond to any level
> > > > >     between the x86 package and core.
> > > > > 
> > > > > Based on the above considerations, and in order to eliminate the 
> > > > > naming
> > > > > confusion caused by the mapping between general cluster and x86 module
> > > > > in v7, we now formally introduce smp.modules as the new topology 
> > > > > level.
> > > > 
> > > > What is the Linux kernel calling this topology level on x86 ?
> > > > It will be pretty unfortunate if Linux and QEMU end up with
> > > > different names for the same topology level.
> > > > 
> > > 
> > > Now Intel's engineers in the Linux kernel are starting to use "module"
> > > to refer to this layer of topology [4] to avoid confusion, where
> > > previously the scheduler developers referred to the share L2 hierarchy
> > > collectively as "cluster".
> > > 
> > > Looking at it this way, it makes more sense for QEMU to use the
> > > "module" for x86.
> > 
> > I was thinking specificially about what Linux calls this topology when
> > exposing it in sysfs and /proc/cpuinfo. AFAICT, it looks like it is
> > called 'clusters' in this context, and so this is the terminology that
> > applications and users are going to expect.
> 
> The cluster related topology information under "/sys/devices/system/cpu/
> cpu*/topology" indicates the L2 cache topology (CPUID[0x4]), not module
> level CPU topology (CPUID[0x1f]).
> 
> So far, kernel hasn't exposed module topology related sysfs. But we will
> add new "module" related information in sysfs. The relevant patches are
> ready internally, but not posted yet.
> 
> In the future, we will use "module" in sysfs to indicate module level CPU
> topology, and "cluster" will be only used to refer to the l2 cache domain
> as it is now.

So, if they're distinct concepts both relevant to x86 CPUs, then from
the QEMU POV, should this patch series be changing the -smp arg to
allowing configuration of both 'clusters' and 'modules' for x86 ?

An earlier version of this series just supported 'clusters', and this
changed to 'modules', but your description of Linux reporting both
suggests QEMU would need both.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|