[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RFC PATCH COLO v2 01/13] docs: block replication's des
From: |
Wen Congyang |
Subject: |
Re: [Qemu-devel] [RFC PATCH COLO v2 01/13] docs: block replication's description |
Date: |
Fri, 3 Apr 2015 10:35:55 +0800 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 |
On 03/26/2015 02:31 PM, Fam Zheng wrote:
> On Wed, 03/25 17:36, Wen Congyang wrote:
>> Signed-off-by: Wen Congyang <address@hidden>
>> Signed-off-by: Paolo Bonzini <address@hidden>
>> Signed-off-by: Yang Hongyang <address@hidden>
>> Signed-off-by: zhanghailiang <address@hidden>
>> Signed-off-by: Gonglei <address@hidden>
>> ---
>> docs/block-replication.txt | 147
>> +++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 147 insertions(+)
>> create mode 100644 docs/block-replication.txt
>>
>> diff --git a/docs/block-replication.txt b/docs/block-replication.txt
>> new file mode 100644
>> index 0000000..874ed8e
>> --- /dev/null
>> +++ b/docs/block-replication.txt
>> @@ -0,0 +1,147 @@
>> +Block replication
>> +----------------------------------------
>> +Copyright Fujitsu, Corp. 2015
>> +Copyright (c) 2015 Intel Corporation
>> +Copyright (c) 2015 HUAWEI TECHNOLOGIES CO.,LTD.
>> +
>> +This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +See the COPYING file in the top-level directory.
>> +
>> +The block replication is used for continuous checkpoints. It is designed
>> +for COLO that Secondary VM is running. It can also be applied for FT/HA
>> +scene that Secondary VM is not running.
>> +
>> +This document gives an overview of block replication's design.
>> +
>> +== Background ==
>> +High availability solutions such as micro checkpoint and COLO will do
>> +consecutive checkpoint. The VM state of Primary VM and Secondary VM is
>> +identical right after a VM checkpoint, but becomes different as the VM
>> +executes till the next checkpoint. To support disk contents checkpoint,
>> +the modified disk contents in the Secondary VM must be buffered, and are
>> +only dropped at next checkpoint time. To reduce the network transportation
>> +effort at the time of checkpoint, the disk modification operations of
>> +Primary disk are asynchronously forwarded to the Secondary node.
>> +
>> +== Workflow ==
>> +The following is the image of block replication workflow:
>> +
>> + +----------------------+ +------------------------+
>> + |Primary Write Requests| |Secondary Write Requests|
>> + +----------------------+ +------------------------+
>> + | |
>> + | (4)
>> + | V
>> + | /-------------\
>> + | Copy and Forward | |
>> + |---------(1)----------+ | Disk Buffer |
>> + | | | |
>> + | (3) \-------------/
>> + | speculative ^
>> + | write through (2)
>> + | | |
>> + V V |
>> + +--------------+ +----------------+
>> + | Primary Disk | | Secondary Disk |
>> + +--------------+ +----------------+
>> +
>> + 1) Primary write requests will be copied and forwarded to Secondary
>> + QEMU.
>> + 2) Before Primary write requests are written to Secondary disk, the
>> + original sector content will be read from Secondary disk and
>> + buffered in the Disk buffer, but it will not overwrite the existing
>> + sector content in the Disk buffer.
>
> Could you elaborate a bit about the "existing sector content" here? IIUC, it
> could be from either "Secondary Write Requests" or previous COW of "Primary
> Write Requests". Is that right?
>
>> + 3) Primary write requests will be written to Secondary disk.
>> + 4) Secondary write requests will be buffered in the Disk buffer and it
>> + will overwrite the existing sector content in the buffer.
>> +
>> +== Architecture ==
>> +We are going to implement COLO block replication from many basic
>> +blocks that are already in QEMU.
>> +
>> + virtio-blk ||
>> + ^ || .----------
>> + | || | Secondary
>> + 1 Quorum || '----------
>> + / \ ||
>> + / \ ||
>> + Primary 2 NBD -------> 2 NBD
>> + disk client || server
>> virtio-blk
>> + || ^
>> ^
>> +--------. || |
>> |
>> +Primary | || Secondary disk <--------- hidden-disk 4
>> <--------- active-disk 3
>> +--------' || | backing ^
>> backing
>> + || | |
>> + || | |
>> + || '-------------------------'
>> + || drive-backup sync=none
>> +
>> +1) The disk on the primary is represented by a block device with two
>> +children, providing replication between a primary disk and the host that
>> +runs the secondary VM. The read pattern for quorum can be extended to
>> +make the primary always read from the local disk instead of going through
>> +NBD.
>> +
>> +2) The secondary disk receives writes from the primary VM through QEMU's
>> +embedded NBD server (speculative write-through).
>> +
>> +3) The disk on the secondary is represented by a custom block device
>> +(called active-disk). It should be an empty disk, and the format should
>> +be qcow2.
>> +
>> +4) The hidden-disk is created automatically. It buffers the original content
>> +that is modified by the primary VM. It should also be an empty disk, and
>> +the dirver supports bdrv_make_empty().
>> +
>> +== New block driver interface ==
>> +We add three block driver interfaces to control block replication:
>> +a. bdrv_start_replication()
>> + Start block replication, called in migration/checkpoint thread.
>> + We must call bdrv_start_replication() in secondary QEMU before
>> + calling bdrv_start_replication() in primary QEMU.
>> +b. bdrv_do_checkpoint()
>> + This interface is called after all VM state is transfered to
>> + Secondary QEMU. The Disk buffer will be dropped in this interface.
>> + The caller must hold the I/O mutex lock if it is in migration/checkpoint
>> + thread.
>> +c. bdrv_stop_replication()
>> + It is called when failover. We will flush the Disk buffer into
>> + Secondary Disk and stop block replication. The vm should be stopped
>> + before calling it. The caller must hold the I/O mutex lock if it is
>> + in migration/checkpoint thread.
>> +
>> +== Usage ==
>> +Primary:
>> + -drive if=xxx,driver=quorum,read-pattern=fifo,\
>> + children.0.file.filename=1.raw,\
>> + children.0.driver=raw,\
>> + children.1.file.driver=nbd+colo,\
>> + children.1.file.host=xxx,\
>> + children.1.file.port=xxx,\
>> + children.1.file.export=xxx,\
>> + children.1.driver=raw,\
>> + children.1.ignore-errors=on
> ^^^^^^^^^
> Won't the leading spaces cause trouble? :)
>
>> + Note:
>> + 1. NBD Client should not be the first child of quorum.
>> + 2. There should be only one NBD Client.
>> + 3. host is the secondary physical machine's hostname or IP
>> + 4. Each disk must have its own export name.
>> +
>> +Secondary:
>> + -drive if=none,driver=raw,file=1.raw,id=nbd_target1 \
>> + -drive if=xxx,driver=qcow2+colo,file=active_disk.qcow2,export=xxx,\
>> + backing_reference.drive_id=nbd_target1,\
>> + backing_reference.hidden-disk.file.filename=hidden_disk.qcow2,\
>> + backing_reference.hidden-disk.driver=qcow2,\
>> + backing_reference.hidden-disk.allow-write-backing-file=on
>> + Then run qmp command:
>> + nbd_server_start host:port
>
> s/nbd_server_start/nbd-server-start
The command name in qmp-commands.hx is nbd-server-start, but
(qemu) nbd-server-start 192.168.3.1:8889
unknown command: 'nbd-server-start'
What's wrong?
Thanks
Wen Congyang
>
> For a few more too.
>
>
>> + Note:
>> + 1. The export name for the same disk must be the same in primary
>> + and secondary QEMU command line
>> + 2. The qmp command nbd_server_start must be run before running the
>> + qmp command migrate on primary QEMU
>> + 3. Don't use nbd_server_start's other options
>> + 4. Active disk, hidden disk and nbd target's length should be the
>> + same.
>> + 5. It is better to put active disk and hidden disk in ramdisk.
>
> Fam
> .
>
- Re: [Qemu-devel] [RFC PATCH COLO v2 01/13] docs: block replication's description,
Wen Congyang <=