Re: [qemu-s390x] [PATCH RFC 0/2] s390: stop abusing memory_region_allocate_system_memory()

[Christian Borntraeger]

On 02.08.19 10:04, David Hildenbrand wrote:
> On 29.07.19 16:52, Igor Mammedov wrote:
>> While looking into unifying guest RAM allocation to use hostmem backends
>> for initial RAM (especially when -mempath is used) and retiring
>> memory_region_allocate_system_memory() API, leaving only single hostmem 
>> backend,
>> I was inspecting how currently it is used by boards and it turns out several
>> boards abuse it by calling the function several times (despite documented 
>> contract
>> forbiding it).
>>
>> s390 is one of such boards where KVM limitation on memslot size got 
>> propagated
>> to board design and memory_region_allocate_system_memory() was abused to 
>> satisfy
>> KVM requirement for max RAM chunk where memory region alias would suffice.
>>
>> Unfortunately, memory_region_allocate_system_memory() usage created migration
>> dependency where guest RAM is transferred in migration stream as several 
>> RAMBlocks
>> if it's more than KVM_SLOT_MAX_BYTES.
> 
> So if I understand it correctly, we only call
> memory_region_allocate_system_memory() in case the guest initial memory
> size exceeds KVM_SLOT_MAX_BYTES - ~8TB.

We always call it. We just call it twice for > 8TB
> 
> Do we *really* care about keeping migration of systems running that most
> probably nobody (except Christian ;) ) really uses? (especially not in
> production).
> 
> I am fine keeping migration running if it's easy, but introducing hacks
> (reading below) for such obscure use cases - I don't know.
> 
> @Christian: Please prove me wrong. :)

For the time being we can block migration for guests > 8TB if that helps (it 
should not
fail in a guest killing fashion), but we should
1. continue to be able to migrate guests < 8TB
2. continue to be 

On the other hand I find "and suddenly it fails if you go beyond this" really
unpleasant. So it would be interesting to see the next round of patches to 
check how "hacky" those really are.


> 
>>
>> In order to replace these several RAM chunks with a single memdev and keep it
>> working with KVM memslot size limit and migration compatible, following was 
>> done:
>>    * [2/2] use memory region aliases to partition hostmem backend RAM on
>>            KVM_SLOT_MAX_BYTES chunks, which should keep KVM side working
>>    * [1/2] hacked memory region aliases (to ram memory regions only) to have
>>            its own RAMBlocks pointing to RAM chunks owned by aliased memory
>>            region. While it's admittedly a hack, but it's relatively simple 
>> and
>>            allows board code rashape migration stream as necessary
>>
>>            I haven't tried to use migratable aliases on x86 machines, but 
>> with it
>>            it could be possible to drop legacy RAM allocation and compat knob
>>            (cd5ff8333a) dropping '-numa node,mem' completely even for old 
>> machines.
>>
>> PS:
>>    Tested with ping pong cross version migration on s390 machine 
>>    (with reduced KVM_SLOT_MAX_BYTES since I don't have access to large
>>     enough host)
>>      
>>
>> Igor Mammedov (2):
>>   memory: make MemoryRegion alias migratable
>>   s390: do not call memory_region_allocate_system_memory() multiple
>>     times
>>
>>  exec.c                     |  7 ++++---
>>  hw/s390x/s390-virtio-ccw.c | 20 +++++++++++++++-----
>>  memory.c                   |  5 +++++
>>  3 files changed, 24 insertions(+), 8 deletions(-)
>>
> 
>