[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug 1856335] Re: Cache Layout wrong on many Zen Arch CPUs
From: |
Heiko Sieger |
Subject: |
[Bug 1856335] Re: Cache Layout wrong on many Zen Arch CPUs |
Date: |
Sun, 26 Apr 2020 10:43:45 -0000 |
It could be an issue of how the kernel presents the CPU topology.
Hardware: AMD Ryzen 3900X 12 core 24 threads (SMT)
Host: Kernel 5.6.6, QEMU 4.2
virsh capabilities | grep "cpu id"
<cpu id='0' socket_id='0' core_id='0' siblings='0,12'/>
<cpu id='1' socket_id='0' core_id='1' siblings='1,13'/>
<cpu id='2' socket_id='0' core_id='2' siblings='2,14'/>
<cpu id='3' socket_id='0' core_id='4' siblings='3,15'/>
<cpu id='4' socket_id='0' core_id='5' siblings='4,16'/>
<cpu id='5' socket_id='0' core_id='6' siblings='5,17'/>
<cpu id='6' socket_id='0' core_id='8' siblings='6,18'/>
<cpu id='7' socket_id='0' core_id='9' siblings='7,19'/>
<cpu id='8' socket_id='0' core_id='10' siblings='8,20'/>
<cpu id='9' socket_id='0' core_id='12' siblings='9,21'/>
<cpu id='10' socket_id='0' core_id='13' siblings='10,22'/>
<cpu id='11' socket_id='0' core_id='14' siblings='11,23'/>
<cpu id='12' socket_id='0' core_id='0' siblings='0,12'/>
<cpu id='13' socket_id='0' core_id='1' siblings='1,13'/>
<cpu id='14' socket_id='0' core_id='2' siblings='2,14'/>
<cpu id='15' socket_id='0' core_id='4' siblings='3,15'/>
<cpu id='16' socket_id='0' core_id='5' siblings='4,16'/>
<cpu id='17' socket_id='0' core_id='6' siblings='5,17'/>
<cpu id='18' socket_id='0' core_id='8' siblings='6,18'/>
<cpu id='19' socket_id='0' core_id='9' siblings='7,19'/>
<cpu id='20' socket_id='0' core_id='10' siblings='8,20'/>
<cpu id='21' socket_id='0' core_id='12' siblings='9,21'/>
<cpu id='22' socket_id='0' core_id='13' siblings='10,22'/>
<cpu id='23' socket_id='0' core_id='14' siblings='11,23'/>
See how cpu id=3 gets core id=4, and cpu id=6 gets core id=8, etc.
cat /sys/devices/system/cpu/cpu2/topology/core_id
2
cat /sys/devices/system/cpu/cpu3/topology/core_id
4
However, the association of CPU IDs to L3 caches seems to be correct:
echo "Level CPU list";for file in /sys/devices/system/cpu/cpu*/cache/index3;
do echo $(cat $file/id) " " $(cat $file/shared_cpu_list); done | sort
--version-sort
Level CPU list
0 0-2,12-14
0 0-2,12-14
0 0-2,12-14
0 0-2,12-14
0 0-2,12-14
0 0-2,12-14
1 3-5,15-17
1 3-5,15-17
1 3-5,15-17
1 3-5,15-17
1 3-5,15-17
1 3-5,15-17
2 6-8,18-20
2 6-8,18-20
2 6-8,18-20
2 6-8,18-20
2 6-8,18-20
2 6-8,18-20
3 9-11,21-23
3 9-11,21-23
3 9-11,21-23
3 9-11,21-23
3 9-11,21-23
3 9-11,21-23
There are 4 L3 caches with the correct CPU lists (6 CPUs/threads each).
Is it possible that this weird CPU ID enumeration is causing the
confusion?
Haven't had a chance to check out QEMU 5.0, but hope to do that today.
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1856335
Title:
Cache Layout wrong on many Zen Arch CPUs
Status in QEMU:
New
Bug description:
AMD CPUs have L3 cache per 2, 3 or 4 cores. Currently, TOPOEXT seems
to always map Cache ass if it was an 4-Core per CCX CPU, which is
incorrect, and costs upwards 30% performance (more realistically 10%)
in L3 Cache Layout aware applications.
Example on a 4-CCX CPU (1950X /w 8 Cores and no SMT):
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>EPYC-IBPB</model>
<vendor>AMD</vendor>
<topology sockets='1' cores='8' threads='1'/>
In windows, coreinfo reports correctly:
****---- Unified Cache 1, Level 3, 8 MB, Assoc 16, LineSize 64
----**** Unified Cache 6, Level 3, 8 MB, Assoc 16, LineSize 64
On a 3-CCX CPU (3960X /w 6 cores and no SMT):
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>EPYC-IBPB</model>
<vendor>AMD</vendor>
<topology sockets='1' cores='6' threads='1'/>
in windows, coreinfo reports incorrectly:
****-- Unified Cache 1, Level 3, 8 MB, Assoc 16, LineSize 64
----** Unified Cache 6, Level 3, 8 MB, Assoc 16, LineSize 64
Validated against 3.0, 3.1, 4.1 and 4.2 versions of qemu-kvm.
With newer Qemu there is a fix (that does behave correctly) in using the dies
parameter:
<qemu:arg value='cores=3,threads=1,dies=2,sockets=1'/>
The problem is that the dies are exposed differently than how AMD does
it natively, they are exposed to Windows as sockets, which means, that
if you are nto a business user, you can't ever have a machine with
more than two CCX (6 cores) as consumer versions of Windows only
supports two sockets. (Should this be reported as a separate bug?)
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1856335/+subscriptions