qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH/RFC] s390x: split memory into 4TB chunks


From: Christian Borntraeger
Subject: Re: [Qemu-devel] [PATCH/RFC] s390x: split memory into 4TB chunks
Date: Wed, 6 Dec 2017 13:15:04 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0


On 12/06/2017 01:12 PM, David Hildenbrand wrote:
> On 06.12.2017 13:06, Christian Borntraeger wrote:
>>
>>
>> On 12/06/2017 01:04 PM, David Hildenbrand wrote:
>>> On 06.12.2017 13:00, Christian Borntraeger wrote:
>>>> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
>>>> limiting the memory per slot to 7.999TB. Lets split the base memory
>>>> into 4TB chunks to allow go beyond 8TB for a guest. With that (and
>>>> optimistic overcommitment in the kernel) I was able to start  a 59TB
>>>> guest on a 1TB system.
>>>>
>>>> Signed-off-by: Christian Borntraeger <address@hidden>
>>>> ---
>>>>  hw/s390x/s390-virtio-ccw.c | 20 +++++++++++++++++---
>>>>  1 file changed, 17 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
>>>> index 8425534..6735bbb 100644
>>>> --- a/hw/s390x/s390-virtio-ccw.c
>>>> +++ b/hw/s390x/s390-virtio-ccw.c
>>>> @@ -157,11 +157,25 @@ static void virtio_ccw_register_hcalls(void)
>>>>  static void s390_memory_init(ram_addr_t mem_size)
>>>>  {
>>>>      MemoryRegion *sysmem = get_system_memory();
>>>> -    MemoryRegion *ram = g_new(MemoryRegion, 1);
>>>> +    ram_addr_t chunk, offset;
>>>> +    gchar *name;
>>>>  
>>>>      /* allocate RAM for core */
>>>> -    memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size);
>>>> -    memory_region_add_subregion(sysmem, 0, ram);
>>>> +    offset = 0;
>>>> +    while (mem_size) {
>>>> +        MemoryRegion *ram = g_new(MemoryRegion, 1);
>>>> +        chunk = mem_size;
>>>> +        /* KVM does not allow memslots >= 8 TB. Lets split into 4TB 
>>>> chunks */> +        if (chunk > 4UL * 1024 * 1024 * 1024 * 1024) {
>>>> +            chunk = 4UL * 1024 * 1024 * 1024 * 1024;
>>>> +        }
>>>> +        name = g_strdup_printf("s390.ram[0x%lx]", offset);
>>>> +        memory_region_allocate_system_memory(ram, NULL, name, chunk);
>>>> +        memory_region_add_subregion(sysmem, offset, ram);
>>>> +        mem_size -= chunk;
>>>> +        offset += chunk;
>>>> +        g_free(name);
>>>> +    }
>>>>  
>>>>      /* Initialize storage key device */
>>>>      s390_skeys_init();
>>>>
>>>
>>> This will most certainly break migration, no?
>>
>> Probably yes. Thats why the patch has RFC. I was looking for ideas.
> 
> As other architectures might also run into this problem, wonder if
> 
> a) bumping up KVM_MEM_MAX_NR_PAGES makes sense.

The original limitation comes from the fact that this define is used to limit
the number of bits in the dirty bitmap as some architectures do not provide
bitops beyond 2^32.

> b) as I said, transparently handle it in kvm slot handling code.

adding Paolo to check what he thinks.
> 
>>>
>>> 1. I remember the name being used for migration, I might be wrong.
>>> 2. Migration of guests > 4TB is certainly broken ;)
>>>
>>> I wonder if this should rather be handled in the kvm_slot code.
>>> (silently create an manage two slots)
>>
> 
> 




reply via email to

[Prev in Thread] Current Thread [Next in Thread]