Re: [PATCH 1/2] block/raw: added support of persistent dirty bitmaps

[Patrik Janoušek]

On 3/22/21 12:18 PM, Vladimir Sementsov-Ogievskiy wrote:
> 22.03.2021 13:46, Vladimir Sementsov-Ogievskiy wrote:
>> 22.03.2021 13:18, Patrik Janoušek wrote:
>>> On 3/22/21 9:41 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> 20.03.2021 12:32, Patrik Janoušek wrote:
>>>>> Current implementation of dirty bitmaps for raw format is very
>>>>> limited, because those bitmaps cannot be persistent. Basically it
>>>>> makes sense, because the raw format doesn't have space where could
>>>>> be dirty bitmap stored when QEMU is offline. This patch solves it
>>>>> by storing content of every dirty bitmap in separate file on the
>>>>> host filesystem.
>>>>>
>>>>> However, this only solves one part of the problem. We also have to
>>>>> store information about the existence of the dirty bitmap. This is
>>>>> solved by adding custom options, that stores all required metadata
>>>>> about dirty bitmap (filename where is the bitmap stored on the
>>>>> host filesystem, granularity, persistence, etc.).
>>>>>
>>>>> Signed-off-by: Patrik Janoušek<pj@patrikjanousek.cz>
>>>>
>>>>
>>>> Hmm. Did you considered other ways? Honestly, I don't see a reason for
>>>> yet another storing format for bitmaps.
>>>>
>>>> The task could be simply solved with existing features:
>>>>
>>>> 1. We have extenal-data-file feature in qcow2 (read
>>>> docs/interop/qcow2.txt). With this thing enabled, qcow2 file contains
>>>> only metadata (persistent bitmaps for example) and data is stored in
>>>> separate sequential raw file. I think you should start from it.
>>>
>>> I didn't know about that feature. I'll look at it.
>>>
>>> In case I use NBD to access the bitmap context and qcow2 as a solution
>>> for persistent layer. Would the patch be acceptable? This is
>>> significant
>>> change to my solution and I don't have enought time for it at the
>>> moment
>>> (mainly due to other parts of my bachelor's thesis). I just want to
>>> know
>>> if this kind of feature is interesting to you and its implementation is
>>> worth my time.
>>
>> Honestly, at this point I think it doesn't. If existing features
>> satisfy your use-case, no reason to increase complexity of file-posix
>> driver and QAPI.
>>
>
> It's unpleasant to say this, keeping in mind that that's your first
> submission :(
>
> I can still recommend in a connection with your bachelor's thesis to
> look at the videos at kvm-forum youtube channel, searching for backup:
>
>  
> https://www.youtube.com/channel/UCRCSQmAOh7yzgheq-emy1xA/search?query=backup
>
> You'll get a lot of information about current developments of external
> backup API.
>
> Also note, that there is (or there will be ?) libvirt Backup API,
> which includes an API for external backup. I don't know the current
> status of it, but if your project is based on libvirt, it's better to
> use libvirt backup API instead of using qemu directly. About Libvirt
> Backup API it's better to ask Eric Blake (adding him to CC).
Unfortunately, my solution is based on Proxmox so I can't use libvirt's
features. I know that a beta version of Proxmox Backup Server has been
released and it would be much better to improve their solution, but they
did it too late so I couldn't change assignment of my bachelor's thesis.