Re: [Qemu-devel] [PATCH v2] x86: gigabyte alignment for ram

[Gerd Hoffmann]

On Mo, 2013-12-16 at 13:54 +0200, Michael S. Tsirkin wrote:
> On Mon, Dec 16, 2013 at 10:11:28AM +0100, Gerd Hoffmann wrote:
> > Map 3G (i440fx) or 2G (q35) of memory below 4G, so the RAM pieces
> > are nicely aligned to gigabyte borders.
> > 
> > Keep old memory layout for (a) old machine types and (b) in case all
> > memory fits below 4G and thus we don't have to split RAM into pieces
> > in the first place.  The later makes sure this change doesn't take
> > away memory from 32bit guests.
> > 
> > So, with i440fx and up to 3.5 GB of memory, all of it will be mapped
> > below 4G.  With more than 3.5 GB of memory 3 GB will be mapped below
> > 4G and the remaining amount will be mapped above 4G.
> > 
> > Signed-off-by: Gerd Hoffmann <address@hidden>
> 
> 
> OK the piix part looks ok to me.  For now I split this
> int two patches: for q35 and piix and parked this on my PCI tree.
> pushed, so pls check it out.
> I also added some comments - see patches I've sent on list.
> 
> However, I'm not sure why do we reserve so much memory for q35.
> 
> I re-checked the pci express spec, it explicitly says
> (Table 7-1: Enhanced Configuration Address Mapping)
> that address bits used for ECAM (aka MMCFG) are 20 + n - 1 to 0,
> wheren n is bits in the bus number field, so up to 8.
> Doing the math we need up to 28 bits that is 256 Megabytes
> of memory.

Correct.

> So what's the issue with using up to 3G for RAM?
> This makes me think the only issue is that it seems to conflict
> with MCH_HOST_BRIDGE_PCIEXBAR_DEFAULT, which we should just get rid of -
> it's never actually used.

Indeed, that has no use because the firmware initializes the xbar
anyway.

Problem is that the firmware places the xbar @ 0xb000000.
Hardcoded, assuming qemu will not map ram above 0xb0000000.

So, we must (a) fix firmware first and (b) get a ugly dependency
that older firmware will not run on latest qemu.

We also need to figure how we wanna fixup things.  So, current memory
layout looks like this:

   0x00000000 - 0xafffffff  --  RAM / unused
   0xb0000000 - 0xbfffffff  --  mmconfig xbar [256 pci busses]
   0xc0000000 - 0xfec00000  --  space for pci bars, almost 1g

Largest pci bar we can map below 4g is 512m, @ 0xc0000000.

If we wanna map 3G RAM we need to move the xbar somewhere else.  Big
question is where?

We can move it to 0xc0000000.  Then we can't map 512m pci bars any more.

We can move it to 0xe0000000.  Then we have to split the pci bar space,
mapping large bars below 0xe0000000 and small ones above 0xf0000000.
SeaBIOS pci init code isn't really up to it.  Could also become tricky
to declare it correctly in acpi / e820 due to the split.

We can move it to 0xf0000000 or 0xf8000000, then map all pci bars below
the xbar.  We have to make the xbar smaller then, reducing the number of
pci busses the mmconf area can handle.  My laptop looks like this.  Not
sure this a good move for qemu though, do we really want to limit the
number of busses we can handle, with everything moving to pci express?

cheers,
  Gerd