Re: [Qemu-devel] [PATCH v2 2/2] RAM API: Make use of it for x86 PC

[Anthony Liguori]

On 11/18/2010 09:57 AM, Avi Kivity wrote:
> On 11/18/2010 05:46 PM, Anthony Liguori wrote:
> > On 11/18/2010 09:22 AM, Avi Kivity wrote:
> > > On 11/18/2010 01:42 AM, Anthony Liguori wrote:
> > > > > Gack.  For the benefit of those that want to join the fun without
> > > > > digging up the spec, these magic flippable segments the i440fx can
> > > > > toggle are 12 fixed 16k segments from 0xc0000 to 0xeffff and a single
> > > > > 64k segment from 0xf0000 to 0xfffff.  There are read-enable and
> > > > > write-enable bits for each, so the chipset can be configured to read
> > > > > from the bios and write to memory (to setup BIOS-RAM caching), and 
> > > > > read
> > > > > from memory and write to the bios (to enable BIOS-RAM caching).  The
> > > > > other bit combinations are also available.
> > > >
> > > > Yup.  As Gleb mentions, there's the SDRAM register which controls 
> > > > whether 0xa0000 is mapped to PCI or whether it's mapped to RAM (but 
> > > > KVM explicitly disabled SMM support).
> > >
> > > KVM not supporting SMM is a bug (albeit one that is likely to remain 
> > > unresolved for a while).  Let's pretend that kvm smm support is not 
> > > an issue.
> > >
> > > IIUC, SMM means that there two memory maps when the cpu accesses 
> > > memory, one for SMM, one for non-SMM.
> >
> > No.  That's not what it means.  With the i440fx, when the CPU 
> > accesses 0xa0000, it gets forwarded to the PCI bus no different than 
> > an access to 0xe0000.
> >
> > If the CPU asserts the EXF4#/Ab7# signal, then the i440fx directs CPU 
> > accesses to 0xa0000 to RAM instead of the PCI bus.
>
> That's what "two memory maps" mean.  If you have one cpu in SMM and 
> another outside SMM, then those two maps are active simultaneously.

I'm not sure if more modern memory controllers do special things here, 
but for the i440fx, if any CPU asserts SMM mode, then any memory access 
to that space is going to access SMRAM.

> > Alternatively, if the SMRAM register is activated, then the i440fx 
> > will redirect 0xa0000 to RAM regardless of whether the CPU asserts 
> > that signal.  That means that even without KVM supporting SMM, this 
> > mode can happen.
>
> That's a single memory map that is modified under hardware control, 
> it's no different than BARs and such.

There is a single block of RAM.

The memory controller may either forward an address unmodified to the 
RAM block or it may forward the address to the PCI bus[1].  A non CPU 
access goes through a controller hierarchy and may be modified while it 
transverses the hierarchy.

So really, we should have a big chunk of RAM that we associate with a 
guest, with a list of intercepts that changes as the devices are 
modified.  Instead of having that list dispatch directly to a device, we 
should send all intercepted accesses to the memory controller and let 
the memory controller propagate out the access to the appropriate device.

[1] The except is access to the local APIC.  That's handled directly by 
the CPU (or immediately outside of the CPU before the access gets to the 
memory controller if the local APIC is external to the CPU).

> > > Things aren't that bad - a ram_addr_t and a physical address are 
> > > already different things, so we already have one level of translation.
> >
> > Yeah, but ram_addr_t doesn't model anything meaningful IRL.  It's an 
> > internal implementation detail.
>
> Does it matter?  We can say those are addresses on the memory bus.  
> Since they are not observable anyway, who cares if the correspond with 
> reality or not?

It matters a lot because the life cycle of RAM is different from the 
life cycle of ROM.

For instance, the original goal was to madvise(MADV_DONTNEED) RAM on 
reboot.  You can't do that to ROM because the contents matter.

But for PV devices, we can be loose in how we define the way the devices 
interact with the rest of the system.  For instance, we can say that 
virtio-pci devices are directly connected to RAM and do not go through 
the memory controllers.  That means we could get stable mappings of the 
virtio ring.

Regards,

Anthony Liguori