Re: [Qemu-block] [PATCH] qcow2: do lazy allocation of the L2 cache

[Kevin Wolf]

Am 08.05.2015 um 11:00 hat Alberto Garcia geschrieben:
> On Fri 24 Apr 2015 03:04:06 PM CEST, Kevin Wolf <address@hidden> wrote:
> 
> >> >> I think it would be nice to have a way to free unused cache
> >> >> entries after a while.
> >> >
> >> > Do you think mmap plus a periodic timer would work?
> >> >
> >> > I'm hesitant about changes like this because they make QEMU more
> >> > complex, slow down the guest, and make the memory footprint
> >> > volatile.  But if a simple solution addresses the problem, then I'd
> >> > be happy.
> >> 
> >> I understand. One possible approach would be to use the timer only if
> >> the cache size requested by the user is large, so
> >> 
> >>  1) we only try to save memory when the amount of memory to save is
> >>     significant.
> >>  2) we don't make the default scenario slower.
> >> 
> >> I'll try to see if I can come up with a good solution (and with more
> >> numbers).
> >
> > I suspect that at this point, Anthony would have reminded us about
> > separation of policy and mechanism. The very least we need is some
> > option to control the policy (probably defaulting to no automatic
> > cache size changes).
> 
> There's also the problem that with a single chunk of memory for all
> cache tables it's not so easy to free individual entries.

That's one of the reasons why I suggested to fix the problem by using
mmap() for the individual entries rather than allocating one big chunk.

> One possible solution would be madvise() with MADV_DONTNEED, and that's
> rather simple to use, but I'm unsure about its portability.

Probably not portable at all, but the worst that can happen is that a
cache doesn't shrink, i.e. behaves like today. Seems good enough.

> > The other option would be to let the management tool change the cache
> > size options at runtime (and as it happens, my current bdrv_reopen()
> > overhaul allows just that - except that a QMP interface isn't planned
> > yet), but I'm not sure if it has enough information to make good
> > decisions. If we know what the criteria are, they could possibly be
> > exposed, though.
> 
> But what would you do there? Recreate the whole cache?

Recreating the cache is the easiest solution (and it is what my reopen
patches do) and the overhead is probably neglegible as long as you don't
resize the cache all the time.

If we really need it, resizing the cache while maintaining its content
would be possible, too. Extending shouldn't be a problem at all (with
your patches you'd have to rehash the entries, but that's it), and for
shrinking you would have to evict some entries (based on the algorithm
that is used in other cases as well). Neither sounds particularly
complicated, it's just some work that needs to be done if we ever get a
requirement like this.

Kevin