Re: [Qemu-devel] [PATCH v5 0/5] Connect a PCIe host and graphics support to RISC-V

[Stephen Bates]

>    So it looks like you at least got to the point where the guest OS
>    would find PCIe devices...

Yes and in fact NVMe IO against those devices do succeed (I can write and read 
the NVMe namespaces). It is just slow because the interrupts are not getting to 
the OS and hence NVMe timeouts are how the completions are discovered.

>  Can you share the output of 'lspci' as well as the configuration you used 
> when building your bbl?

Below is lspci -vvv for the qemu command I sent earlier. The kernel source is 
here [1] and the .config is here [2].

00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
        Subsystem: Red Hat, Inc QEMU PCIe Host bridge
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
lspci: Unable to load libkmod resources: error -12 <--[Note this is an error 
due to poor kmod support in riscv Linux at this time]

00:01.0 Non-Volatile memory controller: Intel Corporation QEMU NVM Express 
Controller (rev 02) (prog-if 02 [NVM Express])
        Subsystem: Red Hat, Inc QEMU Virtual Machine
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx+
        Latency: 0
        Interrupt: pin A routed to IRQ 1
        Region 0: Memory at 45000000 (64-bit, non-prefetchable) [size=8K]
        Region 2: Memory at 44000000 (64-bit, prefetchable) [size=16M]
        Capabilities: [80] Express (v2) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- 
Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- 
TransPend-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, 
OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, 
OBFF Disabled
        Kernel driver in use: nvme

00:02.0 Non-Volatile memory controller: Intel Corporation QEMU NVM Express 
Controller (rev 02) (prog-if 02 [NVM Express])
        Subsystem: Red Hat, Inc QEMU Virtual Machine
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx+
        Latency: 0
        Interrupt: pin A routed to IRQ 1
        Region 0: Memory at 45002000 (64-bit, non-prefetchable) [size=8K]
        Region 2: Memory at 40000000 (64-bit, prefetchable) [size=64M]
        Capabilities: [80] Express (v2) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- 
Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- 
TransPend-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, 
OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, 
OBFF Disabled
        Kernel driver in use: nvme

    
Stephen 

[1] https://github.com/sbates130272/linux-p2pmem/tree/riscv-p2p-sifive
[2] 
https://github.com/Eideticom/kernel-configs/blob/master/riscv-good-config-updated-p2pdma