Re: Fwd: [PATCH 0/2] block/raw: implemented persistent dirty bitmap and ability to dump bitmap content via qapi

[Fabian Grünbichler]

On March 22, 2021 12:27 pm, Patrik Janoušek wrote:
> On 3/22/21 11:48 AM, Max Reitz wrote:
>> Hi,
>>
>> On 20.03.21 11:01, Patrik Janoušek wrote:
>>> I'm sorry, but I forgot to add you to the cc, so I'm forwarding the
>>> patch to you additionally. I don't want to spam the mailing list
>>> unnecessarily.
>>
>> I think it’s better to still CC the list.  It’s so full of mail, one
>> more won’t hurt. :)
>>
>> (Re-adding qemu-block and qemu-devel, because the discussion belongs
>> on the list(s).)
>>
>>> -------- Forwarded Message --------
>>> Subject:     [PATCH 0/2] block/raw: implemented persistent dirty
>>> bitmap and ability to dump bitmap content via qapi
>>> Date:     Sat, 20 Mar 2021 10:32:33 +0100
>>> From:     Patrik Janoušek <pj@patrikjanousek.cz>
>>> To:     qemu-devel@nongnu.org
>>> CC:     Patrik Janoušek <pj@patrikjanousek.cz>, lmatejka@kiv.zcu.cz
>>>
>>>
>>>
>>> Currently, QEMU doesn't support persistent dirty bitmaps for raw format
>>> and also dirty bitmaps are for internal use only, and cannot be accessed
>>> using third-party applications. These facts are very limiting
>>> in case someone would like to develop their own backup tool becaouse
>>> without access to the dirty bitmap it would be possible to implement
>>> only full backups. And without persistent dirty bitmaps, it wouldn't
>>> be possible to keep track of changed data after QEMU is restarted. And
>>> this is exactly what I do as a part of my bachelor thesis. I've
>>> developed a tool that is able to create incremental backups of drives
>>> in raw format that are LVM volumes (ability to create snapshot is
>>> required).
>>
>> Similarly to what Vladimir has said already, the thing is that
>> conceptually I can see no difference between having a raw image with
>> the bitmaps stored in some other file, i.e.:
>>
>>   { "driver": "raw",
>>     "dirty-bitmaps": [ {
>>       "filename": "sdc1.bitmap",
>>       "persistent": true
>>     } ],
>>     "file": {
>>       "driver": "file",
>>       "filename": "/dev/sdc1"
>>     } }
>>
>> And having a qcow2 image with the raw data stored in some other file,
>> i.e.:
>>
>>   { "driver": "qcow2",
>>     "file": {
>>       "driver": "file",
>>       "filename": "sdc1.metadata"
>>     },
>>     "data-file": {
>>       "driver": "file",
>>       "filename": "/dev/sdc1"
>>     } }
>>
>> (Where sdc1.metadata is a qcow2 file created with
>> “data-file=/dev/sdc1,data-file-raw=on”.)
>>
>> To use persistent bitmaps with raw images, you need to add metadata
>> (namely, the bitmaps).  Why not store that metadata in a qcow2 file?
>>
>> Max
> 
> So if I understand it correctly. I can configure dirty bitmaps in the
> latest version of QEMU to be persistently stored in some other file.
> Because even Proxmox Backup Server can't perform an incremental backup
> after restarting QEMU, and that means something to me. I think they
> would implement it if it was that simple.

the main reason we haven't implemented something like that (yet) is not 
that it is technically difficult, but that we have no way to safeguard 
against external manipulation of the (raw) image:

- VM is running, backup happens, bitmap represents the delta since this 
  backup
- VM is stopped
- user/admin does something to VM disk image, believing such an action 
  is safe because VM is not running
- VM is started again - bitmap does not contain the manual changes
- VM is running, backup happens, backup is inconsistent with actual data 
  stored in image

because of persistence, this error is now carried forward indefinitely 
(it might self-correct at some point if all the invalid stuff has been 
dirtied by the guest).

while it is of course possible to argue that this is entirely the user's 
fault, it looks like it's the backup software/hypervisor's fault (no 
indication anything is wrong, backup data is bogus).

it might happen at some point as opt-in feature with all the warnings 
associated with potentially dangerous features, but for now we are okay 
with just carrying the bitmap as long as the VM instance is running 
(including migrations), and having to re-read and chunk/hash the disks 
for fresh instances. the latter is expensive, but less expensive than 
having to explain to users why their backups are bogus.

> 
> Could you please send me simple example on how to configure (via command
> line args) one raw format drive that can store dirty bitmaps
> persistently in other qcow2 file? I may be missing something, but I
> thought QEMU couldn't do it, because Proxmox community wants this
> feature for a long time.
> 
> Patrik
> 
> 
> 
> 
> 
>