Re: [PATCH 0/3] Couple of HMAT fixes

[Michal Privoznik]

On 5/29/20 3:33 PM, Michal Privoznik wrote:
> I've started working on libvirt side of this feature. WIP patches can be
> found here:
>
> https://github.com/zippy2/libvirt/commits/hmat
>
> I've gotten to a point where libvirt generates cmd line but QEMU refuses
> it. Problem is that I was looking into qemu-options.hx instead of
> qapi/machine.json and thus found some irregularities between these two.
>
> I'm not necessarily stating that all these patches are correct (I have
> some doubts about 3/3 because nearly identical code can be found in
> machine_set_cpu_numa_node(), but I have no idea if it's a coincidence).
>
> Michal Privoznik (3):
>    qapi: Make @associativity, @policy and @line of NumaHmatCacheOptions
>      optional
>    numa: Allow HMAT cache to be defined before HMAT latency/bandwidth
>    numa: Initialize node initiator with respect to .has_cpu
>
>   hw/core/numa.c    | 22 +++++++++-------------
>   qapi/machine.json |  6 +++---
>   2 files changed, 12 insertions(+), 16 deletions(-)

Hey, so as I'm experimenting with this, I have couple of questions. 
Hopefully, you have answers.

1) How can I read HMAT from inside the guest? More specifically, I can 
see cache exposed under sysfs, but not latency/bandwidth. I mean, there is:

/sys/devices/system/node/node0/memory_side_cache/

which appears to contain interesting bits. But there seem to be nothing 
like that for latency/bandwidth. There is:

/sys/devices/system/node/node0/access0/initiators

containing:
read_bandwidth  read_latency  write_bandwidth  write_latency

but they all contain "0".


2) I still don't quite understand what initiator is. The way I read the 
documentation is that if a NUMA node has CPUs, it is initiator to 
itself. But, when I try to start the following command line, I get an error:

-machine 
pc-q35-2.12,accel=kvm,usb=off,vmport=off,dump-guest-core=off,hmat=on \
-cpu host,vmx=off \
-m 4096 \
-overcommit mem-lock=off \
-smp 4,sockets=4,cores=1,threads=1 \
-object memory-backend-ram,id=ram-node0,size=2147483648 \
-numa node,nodeid=0,cpus=0-1,initiator=0,memdev=ram-node0 \
-object memory-backend-ram,id=ram-node1,size=2147483648 \
-numa node,nodeid=1,cpus=2-3,initiator=1,memdev=ram-node1 \
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=1 
\
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=read-latency,latency=2 
\
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=write-latency,latency=4 
\
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=1048576K 
\
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=read-bandwidth,bandwidth=2097152K 
\
-numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=write-bandwidth,bandwidth=4194304K 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=10 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=read-latency,latency=20 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=write-latency,latency=40 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=1024K 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=read-bandwidth,bandwidth=2048K 
\
-numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=write-bandwidth,bandwidth=4096K 
\
-numa 
hmat-cache,node-id=0,size=256K,level=1,associativity=direct,policy=write-back,line=8 
\
-numa 
hmat-cache,node-id=0,size=128K,level=2,associativity=complex,policy=write-through,line=16 
\


qemu-system-x86_64: -numa 
node,nodeid=1,cpus=2-3,initiator=1,memdev=ram-node1: The initiator of 
CPU NUMA node 1 should be itself

Firstly, why does "CPU" even appear in the message? Initiator is an 
attribute of a NUMA node not CPU, right? Secondly, as specified on the 
command line, the initiator of the node 1 *is* node 1.


Michal