Re: [PULL 70/73] hw/pxb-cxl: Support passthrough HDM Decoders unless overridden

[Peter Maydell]

On Wed, 8 Mar 2023 at 01:14, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>
> The CXL r3.0 specification allows for there to be no HDM decoders on CXL
> Host Bridges if they have only a single root port. Instead, all accesses
> directed to the host bridge (as specified in CXL Fixed Memory Windows)
> are assumed to be routed to the single root port.

Hi; in issue https://gitlab.com/qemu-project/qemu/-/issues/1586
it's been pointed out that this commit causes an assertion
failure during 'make check' if you configure with
--enable-qom-cast-debug. You can repro by doing that and running:

 qemu-system-i386 -display none -machine q35,cxl=on -device pxb-cxl,bus=pcie.0

Here's a backtrace:

Thread 1 "qemu-system-i38" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6,
threadid=140737217810816) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6,
threadid=140737217810816) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737217810816) at
./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737217810816,
signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff4b1c476 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#4  0x00007ffff4b027f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x0000555555cecfab in object_dynamic_cast_assert
    (obj=obj@entry=0x555557a70b60, typename=0x555555f80406 "pxb",
file=0x555555f80357 "../../hw/pci-bridge/pci_expander_bridge.c",
line=line@entry=55, func=0x555555f8040a "PXB_DEV") at
../../qom/object.c:890
#6  0x00005555559c7bbd in PXB_DEV (obj=0x555557a70b60) at
../../hw/pci-bridge/pci_expander_bridge.c:54
#7  pxb_cxl_dev_reset (dev=0x555557a70b60) at
../../hw/pci-bridge/pci_expander_bridge.c:314
#8  0x00005555559bd624 in pci_qdev_realize (qdev=0x555557a70b60,
errp=0x7fffffffdd28) at ../../hw/pci/pci.c:2098
#9  0x0000555555ce8ada in device_set_realized (obj=<optimised out>,
value=true, errp=0x7fffffffdea8) at ../../hw/core/qdev.c:510
#10 0x0000555555cf0219 in property_set_bool
     (obj=obj@entry=0x555557a70b60, v=v@entry=0x555557a727b0,
name=name@entry=0x55555601db04 "realized", opaque=0x55555687b780,
errp=errp@entry=0x7fffffffdea8) at ../../qom/object.c:2285
#11 0x0000555555cee4e5 in object_property_set
     (obj=obj@entry=0x555557a70b60, name=name@entry=0x55555601db04
"realized", v=0x555557a727b0, errp=errp@entry=0x7fffffffdea8) at
../../qom/object.c:1420
#12 0x0000555555cf23cd in object_property_set_qobject
    (obj=obj@entry=0x555557a70b60, name=name@entry=0x55555601db04
"realized", value=<optimised out>, errp=errp@entry=0x7fffffffdea8) at
../../qom/qom-qobject.c:28
#13 0x0000555555cee93b in object_property_set_bool
(obj=0x555557a70b60, name=0x55555601db04 "realized", value=<optimised
out>, errp=0x7fffffffdea8)
    at ../../qom/object.c:1489
#14 0x0000555555a6ae42 in qdev_device_add_from_qdict
(opts=0x555557a6fb40, from_json=false, errp=0x7fffffffdea8,
errp@entry=0x555556765830 <error_fatal>)
    at ../../softmmu/qdev-monitor.c:714
#15 0x0000555555a6b202 in qdev_device_add
(opts=opts@entry=0x5555568776f0, errp=errp@entry=0x555556765830
<error_fatal>) at ../../softmmu/qdev-monitor.c:733
#16 0x0000555555a7367f in device_init_func (opaque=opaque@entry=0x0,
opts=0x3cd16a, opts@entry=0x5555568776f0, errp=0x6,
errp@entry=0x555556765830 <error_fatal>)
    at ../../softmmu/vl.c:1140
#17 0x0000555555e78331 in qemu_opts_foreach
    (list=<optimised out>, func=0x555555a73670 <device_init_func>,
opaque=opaque@entry=0x0, errp=0x555556765830 <error_fatal>) at
../../util/qemu-option.c:1135
#18 0x0000555555a6dd61 in qemu_create_cli_devices () at ../../softmmu/vl.c:2542
#19 qmp_x_exit_preconfig (errp=<optimised out>) at ../../softmmu/vl.c:2610
#20 0x0000555555a7177b in qemu_init (argc=<optimised out>,
argv=<optimised out>) at ../../softmmu/vl.c:3612
#21 0x000055555587b656 in main (argc=3985770, argv=0x3cd16a) at
../../softmmu/main.c:47

The problem is here:

> -static void pxb_dev_reset(DeviceState *dev)
> +static void pxb_cxl_dev_reset(DeviceState *dev)

This function is called from  pxb_cxl_dev_realize(),
which is the realize function for TYPE_PXB_CXL_DEVICE.
That type's parent is TYPE_PCI_DEVICE.

>  {
>      CXLHost *cxl = PXB_CXL_DEV(dev)->cxl.cxl_host_bridge;
>      CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
> +    PCIHostState *hb = PCI_HOST_BRIDGE(cxl);
>      uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
>      uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
> +    int dsp_count = 0;
>
>      cxl_component_register_init_common(reg_state, write_msk, CXL2_ROOT_PORT);
> -    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, 8);
> +    /*
> +     * The CXL specification allows for host bridges with no HDM decoders
> +     * if they only have a single root port.
> +     */
> +    if (!PXB_DEV(dev)->hdm_for_passthrough) {

However, here we try to cast the device pointer to PXB_DEV.
That is not permitted because dev is not of type TYPE_PXB_DEVICE
(either directly or as a parent class). So if you have the QOM
debugging enabled then the attempt to cast causes an assertion
failure.

> +        dsp_count = pcie_count_ds_ports(hb->bus);
> +    }
> +    /* Initial reset will have 0 dsp so wait until > 0 */
> +    if (dsp_count == 1) {
> +        cxl->passthrough = true;
> +        /* Set Capability ID in header to NONE */
> +        ARRAY_FIELD_DP32(reg_state, CXL_HDM_CAPABILITY_HEADER, ID, 0);
> +    } else {
> +        ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT,
> +                         8);
> +    }
>  }

What was the intention here with the type hierarchy?
Should TYPE_PXB_CXL_DEVICE be a subclass of TYPE_PXB_DEVICE,
or should the cxl-related functions not be trying to treat
it as a PXB device ?

thanks
-- PMM