Re: [Qemu-devel] VFIO VGA test branches

[Alex Williamson]

A few notes for anyone trying this...

      * I recommend the q35 machine type and using the default config
        file found in the docs directory.  This means your command line
        should include:

         -M q35 -nodefconfig -readconfig /path/to/qemu.git/docs/q35-chipset.cfg

      * You're likely passing through a graphics card that is attached
        to the host system below a root port, so make it appear that way
        to the guest too.  If your graphics card has a graphics function
        and audio function, assign them as:

        -device 
vfio-pci,host=2:00.0,x-vga=on,multifunction=on,bus=ich9-pcie-port-1,addr=0.0 \
        -device vfio-pci,host=2:00.1,bus=ich9-pcie-port-1,addr=0.1
        
        The bus name comes from the q35-chipset.cfg above.  If your
        graphics doesn't include a separate audio device, drop the
        second line and the multifunction option of the first (addr is
        also optional at that point, 0.0 will be the default).
        
      * If you follow both of the above, your VGA device is now below a
        root port, but the version of seabios in qemu doesn't support
        initializing VGA routing to that device.  To fix, use upstream
        seabios: git://git.seabios.org/seabios.git  The default config
        should work.  Then add the following to your qemu commandline:
        
        -L /path/to/seabios.git/out/ -L /path/to/qemu/bios/files/
        
        (the latter is likely /usr/local/share/qemu/)
        
      * You can use -nographic to prevent QEMU from trying to start SDL
        or need a vnc parameter.  You can also specify a -vnc option and
        use the window for mouse input.

      * Use -vga none.  At this point I'm not really interested in
        dual-headed VMs unless you're interested in working on it.
        Having an emulated VGA means we're not really testing VGA
        support through VFIO.

      * Do no use the vfio-pci romfile option unless you need it (ie.
        try w/o first).  Option ROMs check an internal signature against
        the hardware.  If they don't match, it isn't run.  If you
        download a ROM from the internet, you may get nowhere.  If you
        do need a ROM, it's best to scrape it off the device you're
        using.  You can do this through the "rom" file in sysfs for the
        device.  "echo 1 > rom" to enable it, the read it as "cat rom
        > /tmp/rom".  To do this, it should be a secondary graphics
        device and be untouched by host drivers.  You may have better
        luck booting from an install CD to get an environment where the
        device is untouched for this.

      * USB passthrough is handy for input and easier than figuring out
        which ports are connected to which USB controllers for vfio-pci
        assignment.  Use lsusb to find the devices, note the bus and
        device numbers, the use:

        -device usb-host,hostbus=8,hostaddr=2

I think that's it.  Feel free to reply with other best practices.
Thanks,

Alex

On Fri, 2013-05-03 at 16:56 -0600, Alex Williamson wrote:
> Hi folks,
> 
> A number of people have been trying VFIO's VGA support, a few have even
> been successful.  Resetting devices has been a problem and makes it
> very, very difficult to really use VGA assignment effectively.  The code
> in the branches below attempts to address this.  Discrete graphics
> devices are typically on their own bus, which we can reset so we
> theoretically get something pretty close to a power-on state for the GPU
> on each run (or after each guest reset).  With this I'm able to get
> multiple runs on my HD7850 with no need to reset the host.  Hopefully
> this will also cleanup after any host uses of the device so we can
> unload driver rather than blacklisting them.
> 
> If you've been playing with VFIO and VGA, please give the branches below
> a shot and report successes and failures.  Note that this new reset is
> only enable with the x-vga=on option, so should not do gratuitous bus
> resets for other devices.  Thanks,
> 
> Alex
> 
> git://github.com/awilliam/linux-vfio.git vfio-vga-reset
> git://github.com/awilliam/qemu-vfio.git vfio-vga-reset
> 
> PS - The above linux branch is v3.9 based which has a known kvm emulator
> bug.  If you're on Intel and nothing happens, try:
> 
> sudo modprobe -r kvm_intel
> sudo modprobe kvm_intel emulate_invalid_guest_state=0
> 
> This is required to execute the VGA BIOS on my HD7850.
> 
> If things still don't work, apply the following patch:
> 
> --- a/hw/misc/vfio.c
> +++ b/hw/misc/vfio.c
> @@ -40,7 +40,7 @@
>  #include "sysemu/kvm.h"
>  #include "sysemu/sysemu.h"
>  
> -/* #define DEBUG_VFIO */
> +#define DEBUG_VFIO
>  #ifdef DEBUG_VFIO
>  #define DPRINTF(fmt, ...) \
>      do { fprintf(stderr, "vfio: " fmt, ## __VA_ARGS__); } while (0)
> 
> And log the output (there will be lots).
> 
> Also, AMD/ATI and Nvidia are the only devices expected to have a
> reasonable shot at working.  I'm seeing reports of success on AMD/ATI HD
> 5xxx, 6xxx, and 7xxx, as well as Nvidia Geforce 7-series, 8-series, 4xx
> series, and 6xx series.  Older cards from those vendors probably aren't
> very interesting to support (honestly I wouldn't care much about 7/8
> series Nvidia or HD5xxx AMD, except I happen to have some for testing -
> use emulated VGA if you don't care about performance).
> 
> Intel IGD graphics has numerous issues since it's partially incorporated
> into the chipset.  Please don't bother to report IGD is broken unless
> you're interested in fixing it.