Re: Fwd: [PATCH 0/2] block/raw: implemented persistent dirty bitmap and ability to dump bitmap content via qapi

[Patrik Janoušek]

On 3/22/21 11:48 AM, Max Reitz wrote:
> Hi,
>
> On 20.03.21 11:01, Patrik Janoušek wrote:
>> I'm sorry, but I forgot to add you to the cc, so I'm forwarding the
>> patch to you additionally. I don't want to spam the mailing list
>> unnecessarily.
>
> I think it’s better to still CC the list.  It’s so full of mail, one
> more won’t hurt. :)
>
> (Re-adding qemu-block and qemu-devel, because the discussion belongs
> on the list(s).)
>
>> -------- Forwarded Message --------
>> Subject:     [PATCH 0/2] block/raw: implemented persistent dirty
>> bitmap and ability to dump bitmap content via qapi
>> Date:     Sat, 20 Mar 2021 10:32:33 +0100
>> From:     Patrik Janoušek <pj@patrikjanousek.cz>
>> To:     qemu-devel@nongnu.org
>> CC:     Patrik Janoušek <pj@patrikjanousek.cz>, lmatejka@kiv.zcu.cz
>>
>>
>>
>> Currently, QEMU doesn't support persistent dirty bitmaps for raw format
>> and also dirty bitmaps are for internal use only, and cannot be accessed
>> using third-party applications. These facts are very limiting
>> in case someone would like to develop their own backup tool becaouse
>> without access to the dirty bitmap it would be possible to implement
>> only full backups. And without persistent dirty bitmaps, it wouldn't
>> be possible to keep track of changed data after QEMU is restarted. And
>> this is exactly what I do as a part of my bachelor thesis. I've
>> developed a tool that is able to create incremental backups of drives
>> in raw format that are LVM volumes (ability to create snapshot is
>> required).
>
> Similarly to what Vladimir has said already, the thing is that
> conceptually I can see no difference between having a raw image with
> the bitmaps stored in some other file, i.e.:
>
>   { "driver": "raw",
>     "dirty-bitmaps": [ {
>       "filename": "sdc1.bitmap",
>       "persistent": true
>     } ],
>     "file": {
>       "driver": "file",
>       "filename": "/dev/sdc1"
>     } }
>
> And having a qcow2 image with the raw data stored in some other file,
> i.e.:
>
>   { "driver": "qcow2",
>     "file": {
>       "driver": "file",
>       "filename": "sdc1.metadata"
>     },
>     "data-file": {
>       "driver": "file",
>       "filename": "/dev/sdc1"
>     } }
>
> (Where sdc1.metadata is a qcow2 file created with
> “data-file=/dev/sdc1,data-file-raw=on”.)
>
> To use persistent bitmaps with raw images, you need to add metadata
> (namely, the bitmaps).  Why not store that metadata in a qcow2 file?
>
> Max

So if I understand it correctly. I can configure dirty bitmaps in the
latest version of QEMU to be persistently stored in some other file.
Because even Proxmox Backup Server can't perform an incremental backup
after restarting QEMU, and that means something to me. I think they
would implement it if it was that simple.

Could you please send me simple example on how to configure (via command
line args) one raw format drive that can store dirty bitmaps
persistently in other qcow2 file? I may be missing something, but I
thought QEMU couldn't do it, because Proxmox community wants this
feature for a long time.

Patrik