Re: [Qemu-ppc] [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Michael S. Tsirkin]

On Mon, Mar 16, 2015 at 02:16:40PM +0200, Marcel Apfelbaum wrote:
> On 03/10/2015 07:42 PM, Michael S. Tsirkin wrote:
> >On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
> >>On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
> >>>On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
> >>>>Signed-off-by: Marcel Apfelbaum <address@hidden>
> >>>>---
> >>>>  docs/pci_expander_bridge.txt | 52 
> >>>> ++++++++++++++++++++++++++++++++++++++++++++
> >>>>  1 file changed, 52 insertions(+)
> >>>>  create mode 100644 docs/pci_expander_bridge.txt
> >>>>
> >>>>diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
> >>>>new file mode 100644
> >>>>index 0000000..58bf7a8
> >>>>--- /dev/null
> >>>>+++ b/docs/pci_expander_bridge.txt
> >>>>@@ -0,0 +1,52 @@
> >>>>+PCI EXPANDER BRIDGE (PXB)
> >>>>+=========================
> >>>>+
> >>>>+Description
> >>>>+===========
> >>>>+PXB is a "light-weight" host bridge in the same PCI domain
> >>>>+as the main host bridge whose purpose is to enable
> >>>>+the main host bridge to support multiple PCI root buses.
> >>>>+It is implemented only for i440fx.
> >>>
> >>>BTW what makes it i440fx specific?
> >>>Also, what happens if you try to use it
> >>>with a different machine type?
> >>Is is i440fx specific, please look at patch 22/28.
> >>Also we have a specific check for i440fx, so CRS
> >>will not be emitted for other machine types.
> >>
> >>Thanks,
> >>Marcel
> >
> >In fact it won't work at all. Need to think about it,
> >maybe we can make it work more generally.
> >For CRS, should be possible to emit for q35 too?
> We can make it work, but not on the scope of this series.
> However, I'll add a IHostBridgeSnoop interface that will
> make the device work only with associated bus and
> this will make it less general.
> 
> Thanks,
> Marcel

OK, this works too. Do you mean PCIHostBridgeSnoop?

> >
> >>>
> >>>>+
> >>>>+As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
> >>>>+is a primary bus and can be associated with a NUMA node
> >>>>+(different from the main host bridge) allowing the guest OS
> >>>>+to recognize the proximity of a pass-through device to
> >>>>+other resources as RAM and CPUs.
> >>>>+
> >>>>+Usage
> >>>>+=====
> >>>>+A detailed command line would be:
> >>>>+
> >>>>+[qemu-bin + storage options]
> >>>>+-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
> >>>>+-m 2G
> >>>>+-object 
> >>>>memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa 
> >>>>node,nodeid=0,cpus=0,memdev=ram-node0
> >>>>+-object 
> >>>>memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 
> >>>>-numa node,nodeid=1,cpus=1,memdev=ram-node1
> >>>>+-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev 
> >>>>user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
> >>>>+-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device 
> >>>>e1000,bus=bridge2,addr=0x3
> >>>>+-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive 
> >>>>if=none,id=drive0,file=[img] -device 
> >>>>virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
> >>>>+
> >>>>+Here you have:
> >>>>+ - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA 
> >>>>node in host, but you can and should put it in different host NUMA nodes)
> >>>>+ - a pxb host bridge attached to NUMA 1 with an e1000 behind it
> >>>>+ - a pxb host bridge attached to NUMA 0 with an e1000 behind it
> >>>>+ - a pxb host bridge not attached to any NUMA with a hard drive behind 
> >>>>it.
> >>>>+
> >>>>+Implementation
> >>>>+==============
> >>>>+The PXB is composed by:
> >>>>+- HostBridge (TYPE_PXB_HOST)
> >>>>+  The host bridge allows to register and query the PXB's rPCI root bus 
> >>>>in QEMU.
> >>>>+- PXBDev(TYPE_PXB_DEVICE)
> >>>>+  It is a regular PCI Device that resides on the piix host-bridge bus 
> >>>>and its bus uses the same PCI domain.
> >>>>+  However, the bus behind is exposed through ACPI as a primary PCI bus 
> >>>>and starts a new PCI hierarchy.
> >>>>+  The interrupts from devices behind the PXB are routed through this 
> >>>>device the same as if it were a
> >>>>+  PCI-2-PCI bridge. The _PRT follows the i440fx model.
> >>>>+- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
> >>>>+  Created automatically as part of init sequence.
> >>>>+  When adding a device to PXB it is attached to the bridge for two 
> >>>>reasons:
> >>>>+  - Using the bridge will enable hotplug support
> >>>>+  - All the devices behind the bridge will use bridge's IO/MEM windows 
> >>>>compacting
> >>>>+    the PCI address space.
> >>>>+
> >>>>--
> >>>>2.1.0