qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] VFIO / PCI-ASSIGN VGA Issues


From: Rafal Kupiec
Subject: [Qemu-devel] VFIO / PCI-ASSIGN VGA Issues
Date: Sun, 27 Apr 2014 13:29:50 +0200

Hello!

I would like to pass through my VGA card to KVM and use to accelerate 3D, however everytime Windows complains about the assigned device showing error 43 in device manager. No matter I try to assign it using q35 and VFIO or pc and legacy pci-assign, the result is the same all the time.
The VGA adapter i try to assign is NVIDIA GeForce GTX 770M:

01:00.0 VGA compatible controller: NVIDIA Corporation GK106M [GeForce GTX 770M] (rev a1)
01:00.0 0300: 10de:11e0 (rev a1)

Used CPU (i7-4800mq) and mainboard support VT-d, what can be check bu launching dmesg | grep -e DMAR -e IOMMU:

[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.000000] ACPI: DMAR 00000000cb967040 0000B8 (v01 INTEL      HSW  00000001 INTL 00000001)
[    0.000000] Intel-IOMMU: enabled
[    0.020208] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.020213] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
[    0.020279] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.250772] DMAR: No ATSR found
[    0.250795] IOMMU 0 0xfed90000: using Queued invalidation
[    0.250796] IOMMU 1 0xfed91000: using Queued invalidation
[    0.250797] IOMMU: Setting RMRR:
[    0.250807] IOMMU: Setting identity map for device 0000:00:02.0 [0xcd000000 - 0xcf1fffff]
[    0.251040] IOMMU: Setting identity map for device 0000:00:1d.0 [0xcbea6000 - 0xcbeb4fff]
[    0.251065] IOMMU: Setting identity map for device 0000:00:1a.0 [0xcbea6000 - 0xcbeb4fff]
[    0.251085] IOMMU: Setting identity map for device 0000:00:14.0 [0xcbea6000 - 0xcbeb4fff]
[    0.251100] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.251107] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.290141] [drm] DMAR active, disabling use of stolen memory


What is more, I have not found any errors in qemu debug output and/or dmesg (or I miss something). The only things showing, in dmesg, while starting up VM are those:

[   95.516726] pci-stub 0000:01:00.0: claimed by stub
[   95.747926] pci-stub 0000:01:00.0: enabling device (0006 -> 0007)
[   97.489931] pci-stub 0000:01:00.0: kvm assign device
[  103.287508] kvm: zapping shadow pages for mmio generation wraparound

Similar output can be found there while using VFIO, while qemu outputs:

Start bios (version ?-20140426_124946-NoteBelliash)
No Xen hypervisor found.
Ram Size=0xc0000000 (0x0000000000000000 high)
Relocating init from 0x000e2241 to 0xbffe0900 (size 63032)
Found QEMU fw_cfg
CPU Mhz=2696
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 8 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: map device bdf=00:04.0  bar 5, addr 0000c000, size 00000080 [io]
PCI: map device bdf=00:03.0  bar 1, addr 0000c080, size 00000040 [io]
PCI: map device bdf=00:01.2  bar 4, addr 0000c0c0, size 00000020 [io]
PCI: map device bdf=00:01.1  bar 4, addr 0000c0e0, size 00000010 [io]
PCI: map device bdf=00:04.0  bar 0, addr fd000000, size 01000000 [mem]
PCI: map device bdf=00:03.0  bar 6, addr fe000000, size 00040000 [mem]
PCI: map device bdf=00:04.0  bar 6, addr fe040000, size 00040000 [mem]
PCI: map device bdf=00:03.0  bar 0, addr fe080000, size 00020000 [mem]
PCI: map device bdf=00:02.0  bar 6, addr fe0a0000, size 00010000 [mem]
PCI: map device bdf=00:02.0  bar 1, addr fe0b0000, size 00001000 [mem]
PCI: map device bdf=00:04.0  bar 1, addr e0000000, size 10000000 [prefmem]
PCI: map device bdf=00:02.0  bar 0, addr f0000000, size 02000000 [prefmem]
PCI: map device bdf=00:04.0  bar 3, addr f2000000, size 02000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.2 id=8086:7020                                                                                                                                                                                                                                            
PCI: init bdf=00:01.3 id=8086:7113                                                                                                                                                                                                                                            
Using pmtimer, ioport 0xb008, freq 3579 kHz                                                                                                                                                                                                                                   
PCI: init bdf=00:02.0 id=1013:00b8                                                                                                                                                                                                                                            
PCI: init bdf=00:03.0 id=8086:100e                                                                                                                                                                                                                                            
PCI: init bdf=00:04.0 id=10de:11e0                                                                                                                                                                                                                                            
PCI: Using 00:02.0 for primary VGA
Found 4 cpu(s) max supported 4 cpu(s)
MP table addr=0x000f1790 MPC table addr=0x000f17a0 size=224
SMBIOS ptr=0x000f1770 table=0x000f15b0 size=439
ACPI DSDT=0xbfffe360
ACPI tables: RSDP=0x000f1580 RSDT=0xbfffe320
Scan for VGA option rom
WARNING! Found unaligned PCI rom (vd=1013:00b8)
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version ?-20140426_124946-NoteBelliash)
UHCI init on dev 00:01.2 (io=c0c0)
Found 1 lpt ports
Found 1 serial ports
Searching bootorder for: /address@hidden/address@hidden/address@hidden/address@hidden
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
ata0-0: QEMU HARDDISK ATA-7 Hard-Disk (146 GiBytes)
Searching bootorder for: /address@hidden/address@hidden,1/address@hidden/address@hidden
DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
Searching bootorder for: /address@hidden/address@hidden,1/address@hidden/address@hidden
PS2 keyboard initialized
All threads complete.
Scan for option roms
Running option rom at c900:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /address@hidden/address@hidden
Searching bootorder for: /address@hidden/kvmvapic.bin

Press F12 for boot menu.

Searching bootorder for: HALT
drive 0x000f1500: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=307200000
Running option rom at ca00:0003
Space available for UMB: cc800-ee800, f0000-f14a0
Returned 57344 bytes of ZoneHigh
e820 map has 7 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 00000000bfffe000 = 1 RAM
  4: 00000000bfffe000 - 00000000c0000000 = 2 RESERVED
  5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
  6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00


I have already applied 2 patches on kernel 3.14.1 - vga arbiter and acs. After that, nvidia gpu is the only device in group, so I'm assigning just it. Also lspci does not show any NVIDIA HDMI Audio device. While trying to solve this problem, I have came across patch prepared by Alex Williamson that aims to fix error 43. I have applied it on top of qemu, but this have not made any difference at all. However lspci -vvv shows NoSnoop- when using VFIO.

In order to launch VM i use below script. Actually it contains both methods (legacy pci-assign and VFIO:


DEVICES="01:00.0"
modprobe -r kvm_intel
modprobe -r kvm

for DEV in $DEVICES; do
        echo $(lspci -n | grep ${DEV} | awk '{ split($3,a,":"); print(a[1],a[2]);}') > /sys/bus/pci/drivers/pci-stub/new_id || exit 1
        if [ -e /sys/bus/pci/devices/0000\:${DEV}/driver ]; then
                echo "0000:${DEV}" > /sys/bus/pci/devices/0000\:${DEV}/driver/unbind || exit 1
        fi
        echo "0000:${DEV}" > /sys/bus/pci/drivers/pci-stub/bind
done

modprobe kvm ignore_msrs=1
modprobe kvm_intel emulate_invalid_guest_state=0

qemu-system-x86_64 \
        -enable-kvm -M pc -m 3072 -cpu host,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -smp 4,sockets=1,cores=2,threads=2 -bios /usr/share/seabios/bios.bin \
        -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios \
        -vga cirrus -device pci-assign,host=01:00.0,romfile=/path/to/vga.rom,prefer_msi=off \
        -hda /path/to/disk.img \
        -boot menu=on -usb -usbdevice tablet -vnc 127.0.0.1:1

exit 0
for DEV in $DEVICES; do
        if [ -e /sys/bus/pci/devices/0000\:${DEV}/driver ]; then
                echo "0000:${DEV}" > /sys/bus/pci/devices/0000\:${DEV}/driver/unbind || exit 1
                echo "Device ${DEV} unbinded"
        fi
        echo $(lspci -n | grep ${DEV} | awk '{ split($3,a,":"); print(a[1],a[2]);}') > /sys/bus/pci/drivers/vfio-pci/new_id || exit 1
        echo "Device ${DEV} binded to vfio-pci"
done

qemu-system-x86_64 \
        -enable-kvm -M q35 -m 3072 -cpu host,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -smp 4,sockets=1,cores=2,threads=2 -bios /usr/share/seabios/bios.bin \
        -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
        -device vfio-pci,host=01:00.0,bus=root.1,multifunction=on,x-vga=on \
        -device virtio-scsi-pci,id=scsi -vga none \
        -device piix4-ide,bus=pcie.0,id=piix4-ide \
        -drive file=/path/to/disk.img,id=disk,format=raw -device ide-hd,bus=piix4-ide.0,drive=disk \
        -drive file=/path/to/cd.iso,id=isocd -device ide-cd,bus=piix4-ide.1,drive=isocd \
        -boot menu=on -usb -usbdevice tablet -vnc 127.0.0.1:1


As you can see I have already tried many solutions, also tried to remove root.1 and assign nvidia gpu directly to pcie.0. Unfortunately, w/o any progress... All the time Windows displays exclamation mark and displays error 43.

What else can I check/try and how can i solve this problem?
Is it possible for me, to use my 2nd GPU with KVM-based virtual machine?

I'm looking forward for your help and I would like to thank you for any help and your time.


Best regards,
Rafal Kupiec

reply via email to

[Prev in Thread] Current Thread [Next in Thread]