Re: [Qemu-devel] [PATCH] pci: honor PCI_COMMAND_MEMORY

[Dmitry Fleytman]

On 17 Jul 2017, at 19:59 PM, Michael S. Tsirkin <address@hidden> wrote:
> 
> On Mon, Jul 17, 2017 at 03:48:03PM +0300, Dmitry Fleytman wrote:
>> Am I understand correctly that there are no special cases for
>> IDE controllers, i.e. bus master bit must be set by SW same
>> way as for other PCI devices?
> 
> Bus mastering is typically enabled by the driver.
> E.g. under linux:
> 
> static int virtio_pci_restore(struct device *dev)
> {
>        struct pci_dev *pci_dev = to_pci_dev(dev);
>        struct virtio_pci_device *vp_dev = pci_get_drvdata(pci_dev);
>        int ret;
> 
>        ret = pci_enable_device(pci_dev);
>        if (ret)
>                return ret;
> 
>        pci_set_master(pci_dev);
>        return virtio_device_restore(&vp_dev->vdev);
> }
> 
> 
> As an exception, in case of BIOS booting using device ROM, it is set by
> the ROM. E.g. src/hw/virtio-pci.c:
> 
>    vp_reset(vp);
>    pci_enable_busmaster(pci);
>    vp_set_status(vp, VIRTIO_CONFIG_S_ACKNOWLEDGE |
>                  VIRTIO_CONFIG_S_DRIVER );


Thanks, Michael.

After some more investigations, there are additional interesting details.

First of all, resume from hibernation succeeds without USB host controller 
(-usbtablet).

I did tracing of PCI configuration space writes and here is what I see:

========= Booting with -usbtablet (piix3-usb-uhci) ================

++ pci_init_bus_master:93: i440FX, enabled: 0
++ pci_init_bus_master:93: PIIX3, enabled: 0
++ pci_init_bus_master:93: piix3-ide, enabled: 0
++ pci_init_bus_master:93: piix3-usb-uhci, enabled: 0
++ pci_init_bus_master:93: PIIX4_PM, enabled: 0
++ pci_init_bus_master:93: VGA, enabled: 0
++ pci_init_bus_master:93: e1000, enabled: 0

All devices start with bus master disabled

++ piix3_reset:121
++ pci_default_write_config:1363: i440FX, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: PIIX3, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 259, l 2, enabled: 0

Guest writes 259 (2 bytes) to PCI_COMMAND, i.e. PCI_COMMAND_IO | 
PCI_COMMAND_MEMORY | PCI_COMMAND_SERR.
This way it effectively disables PCI_COMMAND_MASTER event if it was enabled 
from the beginning.

++ pci_default_write_config:1363: piix3-usb-uhci, addr 4, val_in 259, l 2, 
enabled: 0
++ pci_default_write_config:1363: PIIX4_PM, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: VGA, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-usb-uhci, addr 4, val_in 259, l 2, 
enabled: 0
++ pci_default_write_config:1363: piix3-usb-uhci, addr 4, val_in 263, l 2, 
enabled: 4

At the same time it enables PCI_COMMAND_MASTER for piix3-usb-uhci.
After that it does a bunch of sector reads without DMA and then switches to DMA 
mode and eventually crashes

========= Booting without -usbtablet (piix3-usb-uhci) ================

++ pci_init_bus_master:93: i440FX, enabled: 0
++ pci_init_bus_master:93: PIIX3, enabled: 0
++ pci_init_bus_master:93: piix3-ide, enabled: 0
++ pci_init_bus_master:93: PIIX4_PM, enabled: 0
++ pci_init_bus_master:93: VGA, enabled: 0
++ pci_init_bus_master:93: e1000, enabled: 0

All devices start with bus master disabled

++ piix3_reset:121
++ pci_default_write_config:1363: i440FX, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: PIIX3, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: PIIX4_PM, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: VGA, addr 4, val_in 259, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 259, l 2, enabled: 0

Again it writes 259 to PCI_COMMAND
After that it does a bunch of sector reads without DMA as before.

++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1285, l 2, enabled: 
4

And after that it enables PCI bus master on piix3-ide,
starts DMA transfers and SUCEEDS resume from hibernation

++ pci_default_write_config:1363: VGA, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: VGA, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: VGA, addr 4, val_in 1287, l 2, enabled: 4
++ pci_default_write_config:1363: e1000, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 1287, l 2, enabled: 4
++ pci_default_write_config:1363: e1000, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: e1000, addr 4, val_in 1287, l 2, enabled: 4
++ pci_default_write_config:1363: e1000, addr 4, val_in 1287, l 2, enabled: 4
++ pci_default_write_config:1363: e1000, addr 4, val_in 263, l 2, enabled: 4
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1280, l 2, enabled: 0
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 1285, l 2, enabled: 
4
++ pci_default_write_config:1363: piix3-ide, addr 4, val_in 261, l 2, enabled: 4

It also enables bus master for all other devices at some point.

Do you have any idea for possible directions?

Regards,
Dmitry

> 
> 
> -- 
> MST