Re: [Qemu-devel] Re: [PATCH] PCI: Bus number from the bridge, not the device

[Gleb Natapov]

On Sun, Nov 21, 2010 at 08:22:03PM +0200, Michael S. Tsirkin wrote:
> On Sun, Nov 21, 2010 at 06:01:11PM +0200, Gleb Natapov wrote:
> > > This FW is given to guest by qemu. It only assigns bus numbers
> > > because qemu told it to do so.
> > Seabios is just a guest qemu ships. There are other FW for qemu.
> 
> We don't support them though, do we?
> 
Who is "we"? As qemu community we support any guest code.

> > Bochs
> > bios, openfirmware, efi. All of them where developed outside of qemu
> > project and all of them are usable without qemu. You can't consider them
> > be part of qemu any more then Linux/Windows with virtio drivers.
> 
> You can also burn linuxbios onto your motherboard. If you do you
> void your warranty.
And? What is your point?

> 
> 
> > > 
> > > > > 
> > > > > > > And the spec says, e.g.:
> > > > > > > 
> > > > > > >     the memory mapped configuration base
> > > > > > >   address (always corresponds to bus number 0) for the PCI 
> > > > > > > Segment Group
> > > > > > >   of the host bridge is provided by _CBA and the bus range 
> > > > > > > covered by the
> > > > > > >   base address is indicated by the corresponding bus range 
> > > > > > > specified in
> > > > > > >   _CRS.
> > > > > > > 
> > > > > > Don't see how it is relevant. And _CBA is defined only for PCI 
> > > > > > Express. Lets
> > > > > > solve the problem for PCI first and then move to PCI Express. 
> > > > > > Jumping from one
> > > > > > to another destruct us from main discussion.
> > > > > 
> > > > > I think this is what confuses us.  As long as you are using cf8/cfc 
> > > > > there's no
> > > > > concept of a domain really.
> > > > > Thus:
> > > > >       /address@hidden
> > > > > 
> > > > > is probably enough for BIOS boot because we'll need to make root bus 
> > > > > numbers
> > > > > unique for legacy guests/option ROMs.  But this is not a hardware 
> > > > > requirement
> > > > > and might become easier to ignore eith EFI.
> > > > > 
> > > > You do not need MMCONFIG to have multiple PCI domains. You can have one
> > > > configured via standard cf8/cfc and another one on ef8/efc and one more
> > > > at mmio fce00000 and you can address all of them:
> > > > /address@hidden
> > > > /address@hidden
> > > > /address@hidden
> 
> Isn't the mmio one relocatable?
> 
No. If it is, there is a way to specify so.


> > > > 
> > > > And each one of those PCI domains can have 256 subbridges.
> > > 
> > > Will common guests such as windows or linux be able to use them? This
> > With proper drivers yes. There is HW with more then one PCI bus and I
> > think qemu emulates some of it (PPC MAC for instance). 
> 
> MAC is probably just poking in a couple of predefined places.  That's
> not enumeration.
> 
System but is not enumerable on PC too. That is why you need ACPI to
describe resources or you just poke into predefined places (0cf8-0cff in
case of PCI).

> > > seems to be outside the scope of the PCI Firmware specification, which
> > > says that bus numbers must be unique.
> > They must be unique per PCI segment group.
> 
> We've come full circle, didn't we? i am saying we should let users
> specify PCI Segment group+bus as opposed to the io port, which they
> don't use.
> 
When qemu creates device model there is no any "bus" number assigned to
pci bridges and thus "bus" number is meaningless for qemu.  PCI Segment
group has no HW meaning at all (you seems to ignore this completely). So
lets say you run qemu with -S and want to delete or add device. Where do
you get bus number an PCI Segment group number to specify it?

> > > 
> > > > > > > 
> > > > > > > > > 
> > > > > > > > > That should be enough for e.g. device_del. We do have the 
> > > > > > > > > need to
> > > > > > > > > describe the topology when we interface with firmware, e.g. 
> > > > > > > > > to describe
> > > > > > > > > the ACPI tables themselves to qemu (this is what Gleb's 
> > > > > > > > > patches deal
> > > > > > > > > with), but that's probably the only case.
> > > > > > > > > 
> > > > > > > > Describing HW topology is the only way to unambiguously 
> > > > > > > > describe device to
> > > > > > > > something or someone outside qemu and have persistent device 
> > > > > > > > naming
> > > > > > > > between different HW configuration.
> > > > > > > 
> > > > > > > Not really, since ACPI is a binary blob programmed by qemu.
> > > > > > > 
> > > > > > APCI is part of the guest, not qemu.
> > > > > 
> > > > > Yes it runs in the guest but it's generated by qemu. On real hardware,
> > > > > it's supplied by the motherboard.
> > > > > 
> > > > It is not generated by qemu. Parts of it depend on HW and other part 
> > > > depend
> > > > on how BIOS configure HW. _BBN for instance is clearly defined to return
> > > > address assigned bu the BIOS.
> > > 
> > > BIOS is supplied on the motherboard and in our case by qemu as well.
> > You can replace MB bios by coreboot+seabios on some of them.
> > Manufacturer don't want you to do it and make it hard to do, but
> > otherwise this is just software, not some magic dust.
> 
> They support common hardware but not all features will automatically work.
So what? The same can be said about Linux on a lot of HW.

> 
> > > There's no standard way for BIOS to assign bus number to the pci root,
> > > so it does it in device-specific way. Why should a management tool
> > > or a CLI user care about these? As far as they are concerned
> > > we could use some PV scheme to find root devices and assign bus
> > > numbers, and it would be exactly the same.
> > > 
> > Go write KVM userspace that does that. AFAIK there is project out there
> > that tries to do that. No luck so far. Your world view is very x86/Linux
> > centric. You need to broaden it a little bit. Next time you propose
> > something ask yourself will it work with qemu-sparc, qemu-ppc, qemu-amd.
> 
> Your view is very qemu centric. Ask yourself whether what you propose
> will work with libvirt. Better yet, ask libvirt developers.
My view is qemu centric because we are developing qemu here. Anything that will
work in qemu monitor will work with libvirt or any other management software.

> 
> > > > > > Just saying "not really" doesn't
> > > > > > prove much. I still haven't seen any proposition from you that 
> > > > > > actually
> > > > > > solve the problem. No, "lets use guest naming" is not it. There is 
> > > > > > no
> > > > > > such thing as "The Guest". 
> > > > > > 
> > > > > > --
> > > > > >                     Gleb.
> > > > > 
> > > > > I am sorry if I didn't make this clear.  I think we should use the 
> > > > > domain:bus
> > > > > pair to name the root device. As these are unique and 
> > > > > 
> > > > You forgot to complete the sentence :) But you made it clear enough and
> > > > it is incorrect. domain:bus pair not only not unique they do not exist
> > > > in qemu at all
> > > 
> > > Sure they do. domain maps to mcfg address for express. bus is used for
> > mcfg is optional as far as I can see. You can compile out MMCONFIG
> > support on Linux.
> 
> This doesn't mean that you will be able to address all devices
> on all systems if you do.
> 
Why qemu should care? Guest problem.

> > > cf8/cfc addressing. They are assigned by BIOS but since BIOS
> > > is supplied with hardware the point is moot.
> > Most PC hardware is supplied with Windows, so what? BIOS is a code that
> > runs in a guest. It is part of a guest. Every line of code executed by
> > vcpu belongs to a guest. No need to redefine things to prove you point.
> 
> Call it what you want. I care about the payload that the user supplies.
> BIOS and ACPI are implementation details of qemu needed to run said
> payload smoothly. Said payload might not know or care about
> IO address used to program host pci bridge.
And your point is? What "said payload" has to do with qemu CLI? Qemu
CLI behaviour should not depend on a guest code. This is path to
madness.

> 
> > > 
> > > > and as such can't be used to address device. They are
> > > > product of HW enumeration done by a guest OS just like eth0 or C:.
> > > > 
> > > > --
> > > >                         Gleb.
> > > 
> > > There's a huge difference between BIOS and guest OS,
> > Not true.
> > 
> > >                                            and between bus
> > > numbers of pci root and of nested bridges.
> > Really? What is it?
> 
> Look at the spec citations I provided. bus number for root must be set by
> firmware (or default to 0). The rest can be done by the OS.
> 
So there is not "huge difference" at all. This is just a good old bus
number. Two host bridges may have the same bus number. Some buses may be
not enumerated by a guest (neither FW nor OS) at all and should still be
addressable from qemu CLI.

> > > 
> > > Describing hardware io ports makes sense if you are trying to
> > > communicate data from qemu to the BIOS.  But the rest of the world might
> > > not care.
> > > 
> > The part of the world that manage HW cares.
> 
> You really want libvirt to deal with ioports?  Why only that?
No, I want them to deal with data that make sense. And I stated in first
mail after which you started this flame war that OF paths is not the only
way that make sense for qemu CLI. Using qdev device id makes perfect
sense too. So CLI may look like "devce_add pci.0:pci-bridge.1:ide.0"
or similar. They may even prefer it since they already deal with them on
the command line. Using bus addresses, on the other hand, that may not be even
assigned makes no sense.

> Let them emulate PCI while we are at it?
Don't be ridiculous. Actually that is you who wants to teach management
how bus numbers are assigned to PCI buses. I want to teach them about
device topology which they should know already since they control how
machine is created.

> 
> > You may need to add device
> > from monitor before first line of BIOS is event executed. How can you
> > rely on BIOS enumerate of devices in this case?
> 
> We can control the bus numbers that the BIOS will assign to pci root.
And if you need to insert device that sits behind pci-to-pci bridge? Do
you want to control that bus address too? And bus number do not unambiguously
describe pci root since two roots can have same bus number just fine.

> It's probably required to make them stable anyway.
> 
Why?

--
                        Gleb.