Re: [Qemu-devel] [PATCH v5 00/12] Dirty bitmaps migration

[John Snow]

On 06/19/2015 12:57 PM, Denis V. Lunev wrote:
> On 13/06/15 01:16, John Snow wrote:
>> Ping for maintainer reviews -- I've given it a first pass and it should
>> hopefully be close.
>>
>> Thanks,
>> --js
> do we need to respin? I do not see much attention from maintainers here.
> 
> Den
> 

Not sure. Either everyone else wants to see the persistence series
refined before they merge/review this, or just haven't gotten to this
one yet.


> 
>> On 05/13/2015 11:29 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> These patches provide dirty bitmap migration feature. Only named dirty
>>> bitmaps are to be migrated. Migration may be enabled using migration
>>> capabilities.
>>>
>>> v5:
>>>      - rebase on master
>>>      - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
>>>      - remove rfc, as incremental backup series by John Snow are in
>>>        upstream
>>>     
>>> v4 significant changes:
>>>   0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
>>>   0007: set chunk size to 1kb, disable live iteration for
>>>         migrating data < 1mb size.
>>>
>>>   tests: only one with md5 sum is here. used function event_wait
>>>          by John Snow. (I hope, you don't mind me just adding this
>>>          function with your 'Signed-off-by')
>>>
>>>   rfc: This patch set is based on v13 of
>>>        "block: incremental backup series" by John Snow, which are
>>>        not pushed yet.
>>>
>>> v3:
>>>   based on v13 of "block: incremental backup series" by John Snow.
>>>
>>>   changes from v2:
>>>   removed patch for adding dirty parameter (migration capablities used
>>>   instead).
>>>     0001: printf's dropped, qapi used
>>>   0002: part0 -> zeroes
>>>   0003: part0 -> zeroes
>>>   0005: dirty_dirty -> meta
>>>         add comments about meta bitmap
>>>           0006: the format is changed, nodes used instead of devices.
>>>
>>>   other patches are new.
>>>
>>>   rfc: there are two tests. They are the same but using different
>>>   interfaces: md5 checksum of the bitmap last layer in query-block or
>>>   separate query-block-dirty-bitmap with dirty bitmap regions.
>>>   The second form is more appropriate for debugging, the first is more
>>>   appropriate for simple regression control. Which should go to
>>>   upstream?
>>>
>>> v2:
>>>   1. bug-fixes, that are already in upstream, and renaming of function
>>>   bdrv_reset_dirty_bitmap (which is already in Snow's series) are
>>>   dropped
>>>   2. bitmap store/restore: the concept renamed to serialization, added
>>>   function hbitmap_deserialize_part0, to not transfer zero blocks
>>>   3. migration dirty parameter: added description comment
>>>   4. Other patches are new.
>>>
>>> v2.rfc:
>>> Actually, in this version of the series I'm trying not use
>>> migration/block.c at all. Instead a separate migration unit is added
>>> in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
>>> like blocks in block migration, they have their "dirty-dirty" bitmaps,
>>> for tracking set/unset changes during migration.
>>>
>>> The advantages are:
>>>    - no complications of migration/block.c
>>>    - separate dirty-dirty bitmaps provide handling of "unset's"
>>>    - more effective meta-data/data ratio - no tiny bitmap-blocks.
>>>
>>>
>>>
>>> v1:
>>> These patches provide dirty bitmap migration feature. Only named dirty
>>> bitmaps are to be migrated. Migration is made as a part of block
>>> migration in block-migration.c.
>>>
>>> Dirty bitmap migration may be enabled by "dirty" parameter for qmp
>>> migrate
>>> command. If "blk" and "inc" parameters are false when "dirty" is true
>>> block migration is actually skipped: no allocatoions, no bdrv_read's,
>>> no bdrv_write's, only bitmaps are migrated.
>>>
>>> The patch set includes two my previous bug fixes, which are necessary
>>> for it. The patch set is based on Incremental backup series by John
>>> Snow.
>>> Vladimir Sementsov-Ogievskiy (12):
>>>    hbitmap: serialization
>>>    block: BdrvDirtyBitmap serialization interface
>>>    block: tiny refactoring: minimize hbitmap_(set/reset) usage
>>>    block: add meta bitmaps
>>>    block: add bdrv_next_dirty_bitmap()
>>>    qapi: add dirty-bitmaps migration capability
>>>    migration: add migration/block-dirty-bitmap.c
>>>    iotests: maintain several vms in test
>>>    iotests: add add_incoming_migration to VM class
>>>    qapi: add md5 checksum of last dirty bitmap level to query-block
>>>    iotests: add dirty bitmap migration test
>>>    migration/qemu-file: make functions qemu_(get/put)_string public
>>>
>>>   block.c                        |  98 +++++-
>>>   include/block/block.h          |  22 ++
>>>   include/migration/block.h      |   1 +
>>>   include/migration/migration.h  |   1 +
>>>   include/migration/qemu-file.h  |  17 +
>>>   include/qemu/hbitmap.h         |  67 ++++
>>>   migration/Makefile.objs        |   2 +-
>>>   migration/block-dirty-bitmap.c | 693
>>> +++++++++++++++++++++++++++++++++++++++++
>>>   migration/migration.c          |   9 +
>>>   migration/qemu-file.c          |  18 ++
>>>   qapi-schema.json               |   4 +-
>>>   qapi/block-core.json           |   4 +-
>>>   tests/qemu-iotests/117         |  84 +++++
>>>   tests/qemu-iotests/117.out     |   5 +
>>>   tests/qemu-iotests/group       |   1 +
>>>   tests/qemu-iotests/iotests.py  |  14 +-
>>>   util/hbitmap.c                 | 106 +++++++
>>>   vl.c                           |   1 +
>>>   18 files changed, 1137 insertions(+), 10 deletions(-)
>>>   create mode 100644 migration/block-dirty-bitmap.c
>>>   create mode 100755 tests/qemu-iotests/117
>>>   create mode 100644 tests/qemu-iotests/117.out
>>>
> 

-- 
—js