Re: [Qemu-ppc] KVM memory slots limit on powerpc

[Thomas Huth]

On 08/09/15 09:11, Christian Borntraeger wrote:
> Am 08.09.2015 um 08:05 schrieb Thomas Huth:
>> On 07/09/15 16:31, Igor Mammedov wrote:
>>> On Fri, 4 Sep 2015 12:04:41 +0200
>>> Alexander Graf <address@hidden> wrote:
>>>> On 04.09.15 11:59, Christian Borntraeger wrote:
>>>>> Am 04.09.2015 um 11:35 schrieb Thomas Huth:
>>>>>>
>>>>>> now that we get memory hotplugging for the spapr machine on qemu-ppc,
>>>>>> too, it seems like we easily can hit the amount of KVM-internal memory
>>>>>> slots now ("#define KVM_USER_MEM_SLOTS 32" in
>>>>>> arch/powerpc/include/asm/kvm_host.h). For example, start
>>>>>> qemu-system-ppc64 with a couple of "-device secondary-vga" and "-m
>>>>>> 4G,slots=32,maxmem=40G" and then try to hot-plug all 32 DIMMs ... and
>>>>>> you'll see that it aborts way earlier already.
>>>>>>
>>>>>> The x86 code already increased the amount of KVM_USER_MEM_SLOTS to 509
>>>>>> already (+3 internal slots = 512) ... maybe we should now increase the
>>>>>> amount of slots on powerpc, too? Since we don't use internal slots on
>>>>>> POWER, would 512 be a good value? Or would less be sufficient, too?
>>>>>
>>>>> When you are at it, the s390 value should also be increased I guess.
>>>>
>>>> That constant defines the array size for the memslot array in struct kvm
>>>> which in turn again gets allocated by kzalloc, so it's pinned kernel
>>>> memory that is physically contiguous. Doing big allocations can turn
>>>> into problems during runtime.
>>>>
>>>> So maybe there is another way? Can we extend the memslot array size
>>>> dynamically somehow? Allocate it separately? How much memory does the
>>>> memslot array use up with 512 entries?
>>>
>>> KVM switched memslots allocation to kvm_kvzalloc(), so it would fallback to 
>>> vmalloc
>>>  commit 744961341d472db6272ed9b42319a90f5a2aa7c4
>>>  kvm: avoid page allocation failure in kvm_set_memory_region()
>>
>> Good hint, thanks for pointing that out! ... so increasing the array
>> size should not cause too much trouble :-)
> 
> Changing the allocation of the memslots to a growing structure seems like a
> good idea nevertheless. Any chance to do this as well when you are at it?

I'd like to finish some other stuff first, so if you want to have a try,
feel free to do so ... if not, I'll have a closer look at this later.

 Thomas