Re: [Qemu-block] dirty bitmap restore procedure

[John Snow]

On 11/29/2016 09:13 AM, Michael Ablassmeier wrote:
> hi there,
>
> im currently playing with incremental backups using the dirty-bitmap
> feature (2.7.91 (v2.8.0-rc1-dirty)
>
> My steps as follows (without making use of qemu agent or such):
>
>  1) create base image for live system
>    qemu-img create -f qcow2 disk1.imgc 5G
>  2) boot, create filesystem, mount, touch /mnt/FULL, sync
>  3) create snapshot via QMP:
>   blockdev-snapshot-sync device=ide0-hd0 snapshot-file=snapshot-full.img
>  4) backup original disk via cp
>   cp disk1.img full-backup_1.img
>  5) commit back:
>   block-commit device=ide0-hd0
>
> At this point for my understanding i should have
> created a sufficient full backup.
>
> Now im adding a dirty bitmap to the very same disk:

The problem here is that the bitmap you are creating is not synchronized 
to the backup you've just made. What if some writes have happened since 
you created your snapshot?

You need to create your snapshot and add (or reset) the bitmap using a 
transaction.

>  1) create bitmap
>     block-dirty-bitmap-add node=ide0-hd0 name=INC
>  2) copy some files to the filesystem, sync
>  3) issue backup to inc file from very same bitmap, wait for job to finish:
>     drive-backup device=ide0-hd0 target=inc-backup.1.img sync=incremental 
> bitmap=INC
>
> the inc-backup.1.img now contains my changed data blocks as
> far as i can see.

Only the changes made after creating the bitmap.

> after this i shutdown the virtual machine and remove my
> original disk. Now i wonder how to restore the system
> to its last state. Currently im using something along those
> lines:

As an exercise, I assume?

>  1) create new disk with backing device for full backup:
>     qemu-img create -f qcow2 -b backup-full.img disk1.img 5G
>  2) rebase disk to the inc backup:
>     qemu-img rebase backup-full.img -b inc-backup.1.img
>
> if booting the system i can now see my fileystem which contains
> all changes done prior to the snapshot and the full backup,
> but i dont see any changes applied from the INC backup, i tried
> commiting etc but it just does not work. Im sure i miss something
> here, can anyone give me a pointer?

Yeah, not quite how it works.

When you create an external snapshot, you get something like this:

[Base]<--[Top]

where the 'top' or 'active layer' has 'base' as a backing file. 
Typically, 'base' can be thought of as a snapshot, and 'top' contains 
the data that has changed since that snapshot instant.

Incremental backups work similarly:

[Full]<--[Inc0]<--[Inc1]<--[Inc2]

And so on and so forth. Each incremental backup adds only the data has 
changed in a cascading manner. It's like CSS, except less frustrating.

What you really want to do here is to take your full backup and have the 
incremental backup use the full backup as a backing file.

(1) You can create a qcow2 file with your full backup as a backing file 
prior to the incremental backup procedure and it will Do The Right Thing, or

(2) If the last incremental backup or the full backup is not available 
from QEMU's perspective, you can do an incremental backup to a file 
without a backing file and add the backing file later.

In the case of #2, you actually don't want to do a "real" rebase, which 
ensures that the data appears the same and after the rebase (we don't 
want this -- the incremental backup is only a partial image, we DO want 
to change the effective view of data.) What we want to do instead is an 
unsafe rebase that simply changes the backing file.

When you're done, no matter what route you take, you want this:

[Backup]<--[Incremental]


See the documentation for more information;
https://github.com/qemu/qemu/blob/master/docs/bitmaps.md

Thanks,
--John


> thanks,
>     - michael