Re: An issue with x86 tcg and MMIO

[Peter Maydell]

On Thu, 2 Feb 2023 at 10:56, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 2/1/23 23:39, Jonathan Cameron wrote:
> > Not sure - if we can do the handling above then sure we could make that 
> > change.
> > I can see there is a path to register the callbacks but I'd kind of assumed
> > ROM meant read only...
>
> I think "romd" means "read mostly".
>
> In the case of flash, I believe that a write changes modes (block erase 
> something
> something) and the region changes state into MMIO.  But normal state is read 
> mode where
> read+execute go through unchallenged.

In QEMU a ROMD MemoryRegion (created by memory_region_init_rom_device())
is a memory region backed by RAM for reads and by callbacks for writes.
(I think ROMD stands for "ROM device".)

You can then use memory_region_device_set_romd() to put the ROMD into
either ROMD mode (the default, reads backed by RAM) or MMIO mode
(reads backed by MMIO callbacks). Writes are always callbacks regardless.
This is mainly meant for flash devices, which are usually reads-as-data
but have a programming mode where you write a command to it and then
read back command results. It's possible to use it for other tricks too.

When a ROMD is in ROMD mode then execution from it is as fast as execution
from any RAM; when it is in MMIO mode then execution from it is as slow
as execution from any other MMIO-backed MemoryRegion.

Note that AFAIK you can't execute from MMIO at all with KVM (either
ROMD-in-MMIO mode or a plain old MMIO device).

You might want to look at whether QEMU's iommu functionality is helpful
to you -- I'm assuming CXL doesn't do weird stuff on a less-than-page
granularity, and the iommu APIs will let you do "programmatically decide
where this address should actually go". The other option involves
mapping and unmapping MemoryRegions inside a container MR.

thanks
-- PMM