Re: PCIe with Designware RC.

[BALATON Zoltan]

On Sun, 24 Dec 2023, Shlomo Pongratz wrote:
> Hi,
> I'm working on a AARCH64 project that uses the designeware
> (hw/pci-host/designware.c).
> I've copied the designware initialization from hw/arm/fsl-imx7.c and I
> hope I've updated the dtsi correctly.
> After fixing an issue with the iATU windows (see patch
> https://lists.gnu.org/archive/html/qemu-devel/2023-12/msg02643.html)
> I've tried to add virtualized NVMe controller.
> When I added the lines:
>        -device nvme,serial=deadbeef,drive=nvme0,bus=pcie \  (Or without bus=)
>        -drive file=/home/pliops/disk-1.img,if=none,id=nvme1 \

You define drive with if=none,id=nvme1 but have drive=nvme0 in your 
device. You should refer to the drive you want the device to use so I 
think it should either be -device nvme,drive=nvme1 or the if of drive 
should be nvme0. I don't know how this works for nvme but for CD drives 
for example adding a device would add it without disk and drive defines 
the disk to use. Not sure this makes sense for hard disks or nvme device 
but maybe the command line options don't consider that.

> I could see in QEMU monitor that the NVMe device was preset i.e.
> (qemu) info pci
>  Bus  0, device   0, function 0:
>    PCI bridge: PCI device 16c3:abcd
>      IRQ 0, pin A
>      BUS 0.
>      secondary bus 1.
>      subordinate bus 255.
>      IO range [0xf000, 0x0fff]
>      memory range [0xfff00000, 0x000fffff]
>      prefetchable memory range [0xfff00000, 0x000fffff]
>      id ""
>  Bus  0, device   1, function 0:
>    Class 0264: PCI device 1b36:0010
>      PCI subsystem 1af4:1100
>      IRQ 0, pin A
>      BAR0: 64 bit memory at 0xffffffffffffffff [0x00003ffe].
>      id ""
> However in lspci it was missing
> # lspci
> 00:00.0 Class 0604: 16c3:abcd
>
> If I used the following command
>        -drive file=/home/pliops/disk.img,if=none,id=nvme0 \
>        -device nvme,serial=deadbeef,drive=nvme0,bus=dw-pcie \

Here you correctly define both media and drive so it works as expected. 
There are some shortcuts for -drive with media=disk or media=cdrom and 
if=ide or scsi that don't need a separate drive option as if=none does but 
not sure if that supports nvme. You probably have to check documentation 
or code to find out.

> Then in the monitor I see:
> (qemu) info pci
>  Bus  0, device   0, function 0:
>    PCI bridge: PCI device 16c3:abcd
>      IRQ 0, pin A
>      BUS 0.
>      secondary bus 1.
>      subordinate bus 255.
>      IO range [0xf000, 0x0fff]
>      memory range [0x40000000, 0x401fffff]
>      prefetchable memory range [0xfff00000, 0x000fffff]
>      id ""
>  Bus  1, device   0, function 0:
>    Class 0264: PCI device 1b36:0010
>      PCI subsystem 1af4:1100
>      IRQ 1, pin A
>      BAR0: 64 bit memory at 0xffffffffffffffff [0x00003ffe].
>      id ""
> That is the NVMe is on BUS 1.
> And in lspci I can now see the device but on bus 1.
> # lspci
> 01:00.0 Class 0108: 1b36:0010
> 00:00.0 Class 0604: 16c3:abcd
>
> Is this expected?
>
> But the main problem is that during the initialization of the
> controller registers in BAR0 all the read and writes are actually done
> into the config space.

I don't know what this is but don't think it's related to the above.

Regards,
BALATON Zoltan

> Any ideas?
>
> Thank you
> Shlomo Pongratz.