Re: [Qemu-devel] The maximum limit of virtual network device

[Wu, Jiaxin]

Thanks Laszlo and Marcel for the comments/suggestion. It is quite helpful 
against my usage case.

> >> - use Q35 (for a larger IO space),
> >> - plug a DMI-PCI bridge into the root bridge,
> >> - plug 5 PCI bridges into the DMI-PCI bridge,
> >> - plug 31 NICs per PCI bridge, each NIC into a separate slot.

Best regards!
Jiaxin


> -----Original Message-----
> From: Laszlo Ersek [mailto:address@hidden
> Sent: Thursday, July 6, 2017 5:46 PM
> To: Marcel Apfelbaum <address@hidden>; Wu, Jiaxin
> <address@hidden>; address@hidden; Alexander Bezzubikov
> <address@hidden>
> Subject: Re: [Qemu-devel] The maximum limit of virtual network device
> 
> On 07/06/17 11:24, Marcel Apfelbaum wrote:
> > On 06/07/2017 11:31, Laszlo Ersek wrote:
> 
> >> > Now, I would normally recommend sticking with i440fx for simplicity.
> >> However, each PCI bridge requires 4KB of IO space (meaning (1 + 5) * 4KB
> >> = 24KB),  and OVMF on the i440fx does not support that much (only
> >> 0x4000). So, I'll recommend Q35 for IO space purposes; OVMF on Q35
> >> provides 0xA000 (40KB).
> >
> > So if we use OVMF, going for Q35 gives us actually more IO space, nice!
> > However recommending Q35 for IO space seems odd :)
> 
> OVMF used to have only 0x4000 bytes for PCI IO aperture, since the
> beginning. In <https://bugzilla.redhat.com/show_bug.cgi?id=1333238> I
> investigated how much I could grow the aperture. On Q35, it was possible
> to grow it to 0xA000 bytes (but even then you have to disable vmport,
> which sort of sits in the middle otherwise). On i440fx, the IO ports in
> use by platform devices were so badly distributed that moving beyond
> 0x4000 was not possible. See in particular:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1333238#c16
> https://bugzilla.redhat.com/show_bug.cgi?id=1333238#c19
> 
> >> Therefore I guess the simplest example I can give now is:
> >> - use Q35 (for a larger IO space),
> >> - plug a DMI-PCI bridge into the root bridge,
> >> - plug 5 PCI bridges into the DMI-PCI bridge,
> >> - plug 31 NICs per PCI bridge, each NIC into a separate slot.
> >>
> >
> > The setup looks OK to me (assuming OVMF is needed, otherwise
> > PC + pci-bridges will result in more devices),
> 
> OVMF is quite needed; Jiaxin is one of the edk2 networking maintainers,
> and I think he's using QEMU and OVMF as a testbed for otherwise
> physically-oriented UEFI development.
> 
> > I do have a little concern.
> > We want to deprecate the dmi-pci bridge since it does not support
> > hot-plug (for itself or devices behind it).
> > Alexandr (CCed) is a GSOC student working on a generic
> > pcie-pci bridge that can (eventually) be hot-plugged
> > into a PCIe Root Port and keeps the machine cleaner.
> 
> Nice!
> 
> Please include an update to "docs/pcie.txt" in the scope :)
> 
> Thanks!
> Laszlo