Re: [qemu-s390x] [PATCH RFC 0/2] s390: stop abusing memory_region_allocate_system_memory()

[Igor Mammedov]

On Fri, 2 Aug 2019 10:29:43 +0200
Christian Borntraeger <address@hidden> wrote:

> On 02.08.19 10:04, David Hildenbrand wrote:
> > On 29.07.19 16:52, Igor Mammedov wrote:  
> >> While looking into unifying guest RAM allocation to use hostmem backends
> >> for initial RAM (especially when -mempath is used) and retiring
> >> memory_region_allocate_system_memory() API, leaving only single hostmem 
> >> backend,
> >> I was inspecting how currently it is used by boards and it turns out 
> >> several
> >> boards abuse it by calling the function several times (despite documented 
> >> contract
> >> forbiding it).
> >>
> >> s390 is one of such boards where KVM limitation on memslot size got 
> >> propagated
> >> to board design and memory_region_allocate_system_memory() was abused to 
> >> satisfy
> >> KVM requirement for max RAM chunk where memory region alias would suffice.
> >>
> >> Unfortunately, memory_region_allocate_system_memory() usage created 
> >> migration
> >> dependency where guest RAM is transferred in migration stream as several 
> >> RAMBlocks
> >> if it's more than KVM_SLOT_MAX_BYTES.  
> > 
> > So if I understand it correctly, we only call
> > memory_region_allocate_system_memory() in case the guest initial memory
> > size exceeds KVM_SLOT_MAX_BYTES - ~8TB.  
> 
> We always call it. We just call it twice for > 8TB
> > 
> > Do we *really* care about keeping migration of systems running that most
> > probably nobody (except Christian ;) ) really uses? (especially not in
> > production).
> > 
> > I am fine keeping migration running if it's easy, but introducing hacks
> > (reading below) for such obscure use cases - I don't know.
> > 
> > @Christian: Please prove me wrong. :)  
> 
> For the time being we can block migration for guests > 8TB if that helps (it 
> should not
> fail in a guest killing fashion), but we should
> 1. continue to be able to migrate guests < 8TB
> 2. continue to be 
> 
> On the other hand I find "and suddenly it fails if you go beyond this" really
> unpleasant. So it would be interesting to see the next round of patches to 
> check how "hacky" those really are.

I've looked into cleaning up "migratable aliases",
so far it works fine (well in configurations I was able to test,
there were no regressions in x86 machines which use aliases quite a bit).
As I've written before, aliases could be used for x86 later but
that yet to be proved, so I'd prefer to delay this patch if possible.

However, I'd prefer to intentionally break migration with >8TB
guests to simpler code without keepeing around compat mode
that won't/isn't used in practice.

As for new round of patches (including missing KVM fixup),
I'm going to post them now-ish for you to check it out.

> 
> >   
> >>
> >> In order to replace these several RAM chunks with a single memdev and keep 
> >> it
> >> working with KVM memslot size limit and migration compatible, following 
> >> was done:
> >>    * [2/2] use memory region aliases to partition hostmem backend RAM on
> >>            KVM_SLOT_MAX_BYTES chunks, which should keep KVM side working
> >>    * [1/2] hacked memory region aliases (to ram memory regions only) to 
> >> have
> >>            its own RAMBlocks pointing to RAM chunks owned by aliased memory
> >>            region. While it's admittedly a hack, but it's relatively 
> >> simple and
> >>            allows board code rashape migration stream as necessary
> >>
> >>            I haven't tried to use migratable aliases on x86 machines, but 
> >> with it
> >>            it could be possible to drop legacy RAM allocation and compat 
> >> knob
> >>            (cd5ff8333a) dropping '-numa node,mem' completely even for old 
> >> machines.
> >>
> >> PS:
> >>    Tested with ping pong cross version migration on s390 machine 
> >>    (with reduced KVM_SLOT_MAX_BYTES since I don't have access to large
> >>     enough host)
> >>      
> >>
> >> Igor Mammedov (2):
> >>   memory: make MemoryRegion alias migratable
> >>   s390: do not call memory_region_allocate_system_memory() multiple
> >>     times
> >>
> >>  exec.c                     |  7 ++++---
> >>  hw/s390x/s390-virtio-ccw.c | 20 +++++++++++++++-----
> >>  memory.c                   |  5 +++++
> >>  3 files changed, 24 insertions(+), 8 deletions(-)
> >>  
> > 
> >   
>