qemu-block
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-block] Performance impact of the qcow2 overlap checks


From: John Snow
Subject: Re: [Qemu-block] Performance impact of the qcow2 overlap checks
Date: Wed, 1 Feb 2017 04:58:36 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0


On 01/31/2017 05:28 PM, Max Reitz wrote:
> On 31.01.2017 13:23, John Snow wrote:
>>
>>
>> On 01/23/2017 11:29 AM, Max Reitz wrote:
> 
> [...]
> 
>>> The drawbacks with this approach would be the following:
>>> (1) Is printing a warning enough to make the user shut down the VM as
>>> fast as possible and run qemu-img check?
>>
>> No.
>>
>>> (2) It is legal to have a greater refcount than the number of internal
>>> snapshots plus one. qemu never produces such images, though (or does
>>> it?). Could there be existing images where users will be just annoyed by
>>> such warnings? Should we add a runtime option to disable them?
>>>
>>
>> I'd assume it's not legal? If you have a refcount > 1 and delete the
>> last reference, wouldn't that refcount be then incorrect...?
>>
>> Or I guess maybe an external tool may be playing games and using it as a
>> "sticky" bit of sorts?
> 
> As I said to Berto, it's perfectly legal because it's not forbidden and
> it could be useful for data deduplication.
> 

Wound up seeing that later. I still wonder if that's completely
accurate, but... oh well. Not too important.

> I don't think that's something anyone does currently, though, so we
> could disallow it as I have described. Question is, would it be worth it?
> 
>>> And of course another approach I already mentioned would be to scrap the
>>> overlap checks altogether once we have image locking (and I guess we can
>>> keep them around in their current form at least until then).
>>>
>>> Max
>>>
>>
>> I think it's worth having them.
> 
> Fair enough. On one hand I'm biased towards keeping them because I've
> written them. On the other hand, I just really don't like all of the
> *pre_write_overlap_check calls...
> 
>>                                 Perhaps if you use image locking they
>> can be disabled by default, but I wouldn't really advocate for removing
>> them.
> 
> If we disable them by default, nobody will use them. We shouldn't have
> dead code.
> 

Only if you use the image locking, which is -- oh, oh. Yeah, you do have
a point.

> We could think about disabling them if the image is locked and enabling
> them if it is not locked. This way, they would still get at least some
> usage...
> 
>> I think what you are pointing out with refcount blocks not being
>> essential to protect is probably pretty sufficient as an optimization,
>> too. Or I guess we can merge your ~real~ series to help optimize things.
> 
> Or Berto's idea works out to be good enough. :-)
> 

Follow your dreams!

>> I know in the little qcheck utility I wrote I use RBtrees of ranges that
>> get merged together; the tool doesn't really care which kinds of
>> metadata it is, it just knows that it has "metadata" that should be
>> protected. I think it's fast enough. If the qcow2 is kept fairly
>> defragmented, it's REALLY fast. IIRC, your patchset has something
>> similar, so it should be fast enough too.
> 
> Nothing fancy. It's based on a bitmap of metadata information (more like
> a nibble-map, though) that is RLE compressed for areas not currently in
> use. Simple, algorithmically speaking, but makes it harder to calculate
> the runtime complexity than using a "real" structure, I admit.
> 
> Max
> 

It's a real structure! It's just got some properties that are difficult
to analyze :) You just need to consider a load factor alpha and some
kind of probabilistic distribution describing sector usage for the
bitmap, right?

You could probably at least pick a few common cases and calculate that way.

Or don't. :)

--js



reply via email to

[Prev in Thread] Current Thread [Next in Thread]