Re: [Qemu-devel] [RFC PATCH 01/14] docs: block replication's description

[Wen Congyang]

On 02/26/2015 06:02 PM, Fam Zheng wrote:
> On Thu, 02/26 17:07, Wen Congyang wrote:
>> On 02/26/2015 04:44 PM, Fam Zheng wrote:
>>> On Thu, 02/26 14:38, Wen Congyang wrote:
>>>> On 02/25/2015 10:46 AM, Fam Zheng wrote:
>>>>> On Tue, 02/24 15:50, Wen Congyang wrote:
>>>>>> On 02/12/2015 04:44 PM, Fam Zheng wrote:
>>>>>>> On Thu, 02/12 15:40, Wen Congyang wrote:
>>>>>>>> On 02/12/2015 03:21 PM, Fam Zheng wrote:
>>>>>>>>> Hi Congyang,
>>>>>>>>>
>>>>>>>>> On Thu, 02/12 11:07, Wen Congyang wrote:
>>>>>>>>>> +== Workflow ==
>>>>>>>>>> +The following is the image of block replication workflow:
>>>>>>>>>> +
>>>>>>>>>> +        +----------------------+            
>>>>>>>>>> +------------------------+
>>>>>>>>>> +        |Primary Write Requests|            |Secondary Write 
>>>>>>>>>> Requests|
>>>>>>>>>> +        +----------------------+            
>>>>>>>>>> +------------------------+
>>>>>>>>>> +                  |                                       |
>>>>>>>>>> +                  |                                      (4)
>>>>>>>>>> +                  |                                       V
>>>>>>>>>> +                  |                              /-------------\
>>>>>>>>>> +                  |      Copy and Forward        |             |
>>>>>>>>>> +                  |---------(1)----------+       | Disk Buffer |
>>>>>>>>>> +                  |                      |       |             |
>>>>>>>>>> +                  |                     (3)      \-------------/
>>>>>>>>>> +                  |                 speculative      ^
>>>>>>>>>> +                  |                write through    (2)
>>>>>>>>>> +                  |                      |           |
>>>>>>>>>> +                  V                      V           |
>>>>>>>>>> +           +--------------+           +----------------+
>>>>>>>>>> +           | Primary Disk |           | Secondary Disk |
>>>>>>>>>> +           +--------------+           +----------------+
>>>>>>>>>> +
>>>>>>>>>> +    1) Primary write requests will be copied and forwarded to 
>>>>>>>>>> Secondary
>>>>>>>>>> +       QEMU.
>>>>>>>>>> +    2) Before Primary write requests are written to Secondary disk, 
>>>>>>>>>> the
>>>>>>>>>> +       original sector content will be read from Secondary disk and
>>>>>>>>>> +       buffered in the Disk buffer, but it will not overwrite the 
>>>>>>>>>> existing
>>>>>>>>>> +       sector content in the Disk buffer.
>>>>>>>>>
>>>>>>>>> I'm a little confused by the tenses ("will be" versus "are") and 
>>>>>>>>> terms. I am
>>>>>>>>> reading them as "s/will be/are/g"
>>>>>>>>>
>>>>>>>>> Why do you need this buffer?
>>>>>>>>
>>>>>>>> We only sync the disk till next checkpoint. Before next checkpoint, 
>>>>>>>> secondary
>>>>>>>> vm write to the buffer.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> If both primary and secondary write to the same sector, what is saved 
>>>>>>>>> in the
>>>>>>>>> buffer?
>>>>>>>>
>>>>>>>> The primary content will be written to the secondary disk, and the 
>>>>>>>> secondary content
>>>>>>>> is saved in the buffer.
>>>>>>>
>>>>>>> I wonder if alternatively this is possible with an imaginary "writable 
>>>>>>> backing
>>>>>>> image" feature, as described below.
>>>>>>>
>>>>>>> When we have a normal backing chain,
>>>>>>>
>>>>>>>                {virtio-blk dev 'foo'}
>>>>>>>                          |
>>>>>>>                          |
>>>>>>>                          |
>>>>>>>     [base] <- [mid] <- (foo)
>>>>>>>
>>>>>>> Where [base] and [mid] are read only, (foo) is writable. When we add an 
>>>>>>> overlay
>>>>>>> to an existing image on top,
>>>>>>>
>>>>>>>                {virtio-blk dev 'foo'}        {virtio-blk dev 'bar'}
>>>>>>>                          |                              |
>>>>>>>                          |                              |
>>>>>>>                          |                              |
>>>>>>>     [base] <- [mid] <- (foo)  <---------------------- (bar)
>>>>>>>
>>>>>>> It's important to make sure that writes to 'foo' doesn't break data for 
>>>>>>> 'bar'.
>>>>>>> We can utilize an automatic hidden drive-backup target:
>>>>>>>
>>>>>>>                {virtio-blk dev 'foo'}                                   
>>>>>>>  {virtio-blk dev 'bar'}
>>>>>>>                          |                                              
>>>>>>>             |
>>>>>>>                          |                                              
>>>>>>>             |
>>>>>>>                          v                                              
>>>>>>>             v
>>>>>>>
>>>>>>>     [base] <- [mid] <- (foo)  <----------------- (hidden target) 
>>>>>>> <--------------- (bar)
>>>>>>>
>>>>>>>                          v                              ^
>>>>>>>                          v                              ^
>>>>>>>                          v                              ^
>>>>>>>                          v                              ^
>>>>>>>                          >>>> drive-backup sync=none >>>>
>>>>>>>
>>>>>>> So when guest writes to 'foo', the old data is moved to (hidden 
>>>>>>> target), which
>>>>>>> remains unchanged from (bar)'s PoV.
>>>>>>>
>>>>>>> The drive in the middle is called hidden because QEMU creates it 
>>>>>>> automatically,
>>>>>>> the naming is arbitrary.
>>>>>>
>>>>>> I don't understand this. In which function, the hidden target is created 
>>>>>> automatically?
>>>>>>
>>>>>
>>>>> It's to be determined. This part is only in my mind :)
>>>>
>>>> What about this:
>>>> -drive file=nbd-target,if=none,id=nbd-target0 \
>>>> -drive 
>>>> file=active-disk,if=virtio,driver=qcow2,backing.file.filename=hidden-disk,backing.driver=qcow2,backing.backing=nbd-target0
>>>>
>>>
>>> It's close. I suppose backing.backing is referencing another drive as its
>>> backing_hd, then you cannot have the other backing.file.* option - they
>>> conflict. It would be something along:
>>>
>>> -drive file=nbd-target,if=none,id=nbd-target0 \
>>> -drive file=hidden-disk,if=none,id=hidden0,backing.backing=nbd-target0 \
>>> -drive file=active-disk,if=virtio,driver=qcow2,backing.backing=hidden0
>>>
>>> Or for simplicity, s/backing.backing=/backing=/g
>>
>> If using backing=drive_id, backing.backing and backing.file.* are not 
>> conflict.
>> backing.backing=$drive_id means that: backing file's backing file's id is 
>> $drive_id.
> 
> I see.
> 
>>
>>>
>>> Yes, adding these "backing=$drive_id" option is also exactly what we expect
>>> in order to support image-fleecing, but we haven't figured how to allow that
>>> without breaking other qmp operations like block jobs, etc.
>>
>> I don't understand this. In which case, qmp operations will be broken? Can 
>> you give
>> me some examples?
>>
> 
> I don't mean there is a fundamental stopper for this, but in order to relax 
> the
> assumption that "only top BDS can have a BlockBackend", we need to think
> through the whole block layer, and add new finer checks/restrictions where 
> it's
> necessary, otherwise it will be a mess to allow arbitrary backing reference.
> 
> Some random questions I'm now aware of:
> 
> 1. nbd-target0 is writable here, without the drive-backup, hidden0 could be
> corrupted by writings to it. So there need to be a new convention and
> invariance to follow.

Hmm, I understand while the hidden-disk should be opened automatically now.
If we use backing reference, I think we should open a hindden-disk, and set
drive backup automatically. Block any conflict operations(commit, change, 
eject?)

> 
> 2. in qmp, block-commit hidden0 to nbd-target0 or it's backing file, will
> corrupt data (from nbd-target0's perspective).
> 
> 3. unclear implications of "change" and "eject" when there is backing
> reference.
> 
> 4. can a drive be backing referenced by more than one other drives?

We can forbid it first.

Thanks
Wen Congyang

> 
> Just two cents, and I still need to think about it systematically.
> 
> Fam
> .
>