[Qemu-devel] [PATCH RFC 0/4] ARM SMMUv3 Emulation

[pmallapp]

From: Prem Mallappa <address@hidden>

Implementation Notes:

        - SMMUv3 model, as per ARM SMMUv3 11.0 spec
        - Works with Linux Kernel 4.4 SMMUv3 Driver By Will Deacon.
        - Stage1 only
        - only LPAE Translation tables supported
        - BE for translation tables is not supported.
        - Save/Restore not supported "YET".
        - Broadcom variant added, not much different at the moment

Untested
        - Stage2 only
        - Stage1+Stage2 support

Future planned work:
        - MSI(x) support

Test Case setup:
-------------------
 - DPDK + VFIO (DPDK from git master)
 - QEMU (master)
 - Linux Kernel 4.5
 - Bridge interface in Linux

1. Create Bridge interface
- create br0
        sudo brctl addbr br0
- create tap0 tap1 
        sudo ip tuntap add dev tap0 mode tap user ${USER}
        sudo ip tuntap add dev tap1 mode tap user ${USER}

- assign IP address to br0
        sudo ip addr add 20.40.60.80/24 dev br0

2. Launch QEMU (3 ethernet, one unused, the virtio-net is used for NFS)

     pmallapp> aarch64-softmmu/qemu-system-aarch64 -machine type=virt -cpu 
cortex-a57 \
                -m 4192 -smp 4 -nographic \
                -append 'console=ttyAMA0,38400n8 mem=4192M  root=/dev/nfs \
                        nfsroot=10.0.2.2:/srv/nfsroot/v8,vers=3,nolock rw 
e1000e.IntMode=2 \
                        ip=10.0.2.15:10.0.2.2:10.0.2.2:255.255.255.0:prems:eth0 
 ' \
                -kernel 
/home/pmallapp/work/vulcan/armsw/linux/build/arch/arm64/boot/Image \
                -chardev 
socket,id=serial0,host=localhost,port=5000,server,telnet \
                -serial chardev:serial0 \
                -chardev 
socket,id=mon0,host=localhost,port=6001,server,telnet,nowait \
                -monitor chardev:mon0 \
                -device 
pci-bridge,bus=pcie.0,multifunction=on,chassis_nr=2,msi=on,id=pcie.2 \
                -netdev user,id=eth0 \
                -device virtio-net-device,netdev=eth0 \
                -netdev tap,id=eth1,ifname=tap0,script=no,downscript=no \
                -device e1000,netdev=eth1,bus=pcie.2,addr=3 \
                -netdev tap,id=eth2,ifname=tap1,script=no,downscript=no \
                -device virtio-net-pci,netdev=eth2,vectors=0

3. Remove Device from Linux, Add to VFIO, create some reserved Huge pages
        # lspci -nn
        00:00.0 Host bridge [0600]: Red Hat, Inc. Device [1b36:0008]
        00:01.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCI-PCI bridge [1b36:0001]
        00:02.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device 
[1af4:1000]
        01:03.0 Ethernet controller [0200]: Intel Corporation 82540EM Gigabit 
Ethernet Controller [8086:100e] (rev 03)
        
        echo 0000:01:03.0 > /sys/bus/pci/devices/0000:01:03.0/driver/unbind
        echo 8086 100e > /sys/bus/pci/drivers/vfio-pci/new_id

        echo 300 > /proc/sys/vm/nr_hugepages (my case HugePageSize is 2MB)

4. Launch DPDK app, testpmd 
        ./testpmd -c 0xf  -w 0000:01:03.0 -- --rxq=1 --txq= rxd=32  \
                --txd=32 --mbuf-size=32 --total-num-mbufs=1025 
--max-pkt-len=512 -i
        testpmd> set fwd icmpecho
        testpmd> start

5. From host ping any address in 20.40.60.80/24 range
        ping 20.40.60.90 -I br0 -c 10 -r

Prem Mallappa (4):
  arm: smmu: ARM SMMUv3 emulation
  hw: arm: Added smmuv3 files for build
  hw: arm: Add SMMUv3 to virt platform
  devicetree: Added new APIs to make use of more fdt functions

 default-configs/aarch64-softmmu.mak |    1 +
 device_tree.c                       |   35 +
 hw/arm/Makefile.objs                |    1 +
 hw/arm/smmuv3-internal.h            |  343 ++++++++
 hw/arm/smmuv3.c                     | 1530 +++++++++++++++++++++++++++++++++++
 hw/arm/virt.c                       |   62 ++
 include/hw/arm/smmuv3.h             |   39 +
 include/hw/arm/virt.h               |    2 +
 include/sysemu/device_tree.h        |   18 +
 9 files changed, 2031 insertions(+)
 create mode 100644 hw/arm/smmuv3-internal.h
 create mode 100644 hw/arm/smmuv3.c
 create mode 100644 include/hw/arm/smmuv3.h

-- 
2.6.4