[Qemu-devel] High memory usage by multiqueue

[Edivaldo de Araujo Pereira]

Dear Jason,

In the last 2 weeks, I noticed a significant increase in the memory usage of 
qemu. Just to assess the stability of my system, I usually run a kind of stress 
test, every time I install a new kernel in my host or in a custom VM that I use 
in my classes, or when I compile a new snapshot of qemu. 

The test is quite simple, but somewhat brutal: 200 qemu guests (1 cpu, 256MB, 
no hd) running on a single host (4 core, 8 threads, 8GB). It's obvious that 
such thing doesn't serve any purpose other than to say it runs... 80% of all 
memory and 35% of all processing just to make 200 guests little more than 
idle... But the point is, when everything is ok, it runs very well, and I can 
even put some 240Mbits of traffic in the virtual net.

In my tests, each VM uses ~180MB (thanks to ksm, 200*180MB<8GB); but in recent 
weeks, the memory footprint of the MVs jumped to ~230MB, along whith some 
change in its behaviour that rends completely broken the magic of ksm (at least 
in my test); in other words, I can no more run 200 guests in my machine.

So I did a "git bisect", as Stefan told some time ago, and I found the problem 
is related to multiqueue support. More precisely, the constant MAX_QUEUE_NUM 
1024 in include/net/net.h.

As long as I understand, such a high number of queues is at least a little bit 
out of reality... Wouldn't it imply having possibly that number of threads 
runing network traffic? Problem is, I can not imagine the total number of 
logical processors a guest would need to have, just to take advantage of such a 
number of network traffic thhreads. By the way, that number is many times 
higher than supported by any physical network adapter I have read about.

So I've tampered a little with that number, and found that setting it to 4 or 8 
reduces drastically the memory usage, without disrupting any network function 
(in my test...); yet this is twice the numbers of some examples I've seen in 
wmware related texts. The good news: my stress test runs smoothly again... 
well, almost, for I'm not using multiqueue yet...

So, I think it would be good to find a better approach; dynamic allocation 
would be nice, in accordance with command line parameters, wouldn't it?

Thank you, very much

Edivaldo de Araújo Pereira