Re: [PATCH v2 06/12] accel/tcg: better handle memory constrained systems

[Daniel P . Berrangé]

On Wed, Jul 22, 2020 at 05:29:46PM +0100, Alex Bennée wrote:
> 
> Richard Henderson <richard.henderson@linaro.org> writes:
> 
> > On 7/21/20 11:28 PM, Alex Bennée wrote:
> >> +        size_t phys_mem = qemu_get_host_physmem();
> >> +        if (phys_mem > 0 && phys_mem < (2 * 
> >> DEFAULT_CODE_GEN_BUFFER_SIZE)) {
> >> +            tb_size = phys_mem / 8;
> >> +        } else {
> >> +            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
> >> +        }
> >
> > I don't understand the 2 * DEFAULT part.
> 
> I figured once you had at least twice as much memory you could use the
> full amount but...
> 
> 
> > Does this make more sense as
> >
> >     if (phys_mem == 0) {
> >         tb_size = default;
> >     } else {
> >         tb_size = MIN(default, phys_mem / 8);
> >     }
> 
> This is probably a less aggressive tapering off which still doesn't
> affect my 32gb dev machine ;-)

I still feel like this logic of looking at physmem is doomed, because
it makes the assumption that all of physical RAM is theoretically
available to the user, and this isn't the case if running inside a
container or cgroup with a memory cap set.

I don't really have any good answer here, but assuming we can use
1 GB for a cache just doesn't seem like a good idea, especially if
users are running multiple VMs in parallel.

OpenStack uses TCG in alot of their CI infrastructure for example
and runs multiple VMs. If there's 4 VMs, that's another 4 GB of
RAM usage just silently added on top of the explicit -m value.

I wouldn't be surprised if this pushes CI into OOM, even without
containers or cgroups being involved, as they have plenty of other
services consuming RAM in the CI VMs.

The commit 600e17b261555c56a048781b8dd5ba3985650013 talks about this
minimizing codegen cache flushes, but doesn't mention the real world
performance impact of eliminating those flushes ?

Presumably this makes the guest OS boot faster, but what's the before
and after time ?  And what's the time like for values in between the
original 32mb and the new 1 GB ?  Can we get some value that is
*significantly* smaller than 1 GB but still gives some useful benefit ?
what would 128 MB be like compared to the original 32mb ?

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|