Re: [Qemu-ppc] KVM memory slots limit on powerpc

[Christian Borntraeger]

Am 08.09.2015 um 08:05 schrieb Thomas Huth:
> On 07/09/15 16:31, Igor Mammedov wrote:
>> On Fri, 4 Sep 2015 12:04:41 +0200
>> Alexander Graf <address@hidden> wrote:
>>
>>>
>>>
>>> On 04.09.15 11:59, Christian Borntraeger wrote:
>>>> Am 04.09.2015 um 11:35 schrieb Thomas Huth:
>>>>>
>>>>>  Hi all,
>>>>>
>>>>> now that we get memory hotplugging for the spapr machine on qemu-ppc,
>>>>> too, it seems like we easily can hit the amount of KVM-internal memory
>>>>> slots now ("#define KVM_USER_MEM_SLOTS 32" in
>>>>> arch/powerpc/include/asm/kvm_host.h). For example, start
>>>>> qemu-system-ppc64 with a couple of "-device secondary-vga" and "-m
>>>>> 4G,slots=32,maxmem=40G" and then try to hot-plug all 32 DIMMs ... and
>>>>> you'll see that it aborts way earlier already.
>>>>>
>>>>> The x86 code already increased the amount of KVM_USER_MEM_SLOTS to 509
>>>>> already (+3 internal slots = 512) ... maybe we should now increase the
>>>>> amount of slots on powerpc, too? Since we don't use internal slots on
>>>>> POWER, would 512 be a good value? Or would less be sufficient, too?
>>>>
>>>> When you are at it, the s390 value should also be increased I guess.
>>>
>>> That constant defines the array size for the memslot array in struct kvm
>>> which in turn again gets allocated by kzalloc, so it's pinned kernel
>>> memory that is physically contiguous. Doing big allocations can turn
>>> into problems during runtime.
>>>
>>> So maybe there is another way? Can we extend the memslot array size
>>> dynamically somehow? Allocate it separately? How much memory does the
>>> memslot array use up with 512 entries?
>>
>> KVM switched memslots allocation to kvm_kvzalloc(), so it would fallback to 
>> vmalloc
>>  commit 744961341d472db6272ed9b42319a90f5a2aa7c4
>>  kvm: avoid page allocation failure in kvm_set_memory_region()
> 
> Good hint, thanks for pointing that out! ... so increasing the array
> size should not cause too much trouble :-)

Changing the allocation of the memslots to a growing structure seems like a
good idea nevertheless. Any chance to do this as well when you are at it?

Christian