Re: [Qemu-devel] Qemu and heavily increased RSS usage

[Peter Lieven]

> Am 27.06.2016 um 14:39 schrieb Stefan Hajnoczi <address@hidden>:
> 
>> On Fri, Jun 24, 2016 at 12:45:49PM +0200, Peter Lieven wrote:
>>> Am 24.06.2016 um 11:58 schrieb Peter Maydell:
>>>> On 24 June 2016 at 10:37, Stefan Hajnoczi <address@hidden> wrote:
>>>>> On Wed, Jun 22, 2016 at 09:56:06PM +0100, Peter Maydell wrote:
>>>>>> On 22 June 2016 at 20:55, Peter Lieven <address@hidden> wrote:
>>>>>> What makes the coroutine pool memory intensive is the stack size of 1MB 
>>>>>> per
>>>>>> coroutine. Is it really necessary to have such a big stack?
>>>>> That reminds me that I was wondering if we should allocate
>>>>> our coroutine stacks with MAP_GROWSDOWN (though if we're
>>>>> not actually using 1MB of stack then it's only going to
>>>>> be eating virtual memory, not necessarily real memory.)
>>>> Yes, MAP_GROWSDOWN will not reduce RSS.
>>> Right, but then the 1MB of stack as currently allocated isn't
>>> going to be affecting RSS either I would have thought (except
>>> transiently, since we zero it on allocation which will
>>> bring it into the RSS until it falls back out again
>>> because we don't touch it after that).
>> 
>> What I observe regarding the coroutine pool is really strange. Under I/O load
>> while booting the vServer the RSS size is low as expected. If the vServer 
>> runs
>> for some time the RSS size suddenly explodes as if suddenly all the stack 
>> memory gets
>> mapped. This symptom definetely goes away if I disable the pool.
>> 
>> Regarding the coroutine pool I had the following thoughts:
>> - mmap the stack so its actually really freed if the coroutine is deleted 
>> (with MAP_GROWSDOWN or not?)
> 
> This might be an easy fix if malloc is holding memory and not reusing
> it.

it is reusing it, but its heavily fragmented as it seems. i am preparing a 
series to improve the rss usage. hopefully i have sth ready by tomorrow.

 Peter

> 
> Stefan