Re: [Qemu-devel] [PATCH] Introduce cache images for the QCOW2 format

[Kaveh Razavi]

On 08/14/2013 02:02 PM, Alex Bligh wrote:
>> > Not really. I meant different backing images, and not necessarily
>> > booting on the same host.
> So how does your cache solve the problem you mentioned in that
> para?
> 

If you have a fast network (think 10GbE), then qcow2 can easily boot
many VMs over many hosts from a single backing file without any
additional delay. If you don't, then you should do some sort of caching,
so that subsequent boots over many hosts do not hit the network again.

Regardless of the network, in a multi-user scenario, booting many VMs
with different backing files (possibly on different hosts) easily
creates a bottleneck at the storage device (i.e. disk) that hosts the
backing images due to random reads. Considering this, you would like to
keep cache images either on the VM hosts (in case of a slow network), or
on a tmpfs (in case of a fast network). In both cases, the small size of
the cache images help.

We have done a number of benchmarks when scaling to many hosts and I
just told you some of the results. If you want to know more, please
follow the link I sent earlier to a paper that we recently published on
the topic.

>> > This is essentially the same as what Eric mentioned in his email. As
>> > long as only one VM writes to the image and it is not simultaneously
>> > used by other VMs this should not happen.
> Correct. So you need one cache file per VM, not one per image.
> 

No, once the read-only cache is created, it can be used by different VMs
on the same host. But yes, it first needs to be created.

> Correct. So the patch would only speed up the first reboot of
> a VM that has already been booted on that node (as you have one
> cache file per VM). However, when the second VM boots with the
> same image, rather than the pages being hot in the page cache,
> it will be loading them from whatever device the cache is on
> (as per the above, we need one cache per VM), and whatever that
> is, it won't be faster than RAM.
> 
> So I fail to see how this speeds things up. Do you have measured
> numbers?

We did measure when the cache is either on memory/disk of the host or
when it is accessed over NFS (rwsize 64KB) and stored on remote
memory/disk. With a CentOS default installation, they all were booting
within 1% of each other. The idea behind this patch is not to make
booting faster, but to provide a mechanism to avoid scalability
bottlenecks that booting VMs (over many hosts) can create.

Kaveh