[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v2 0/6] external backup api
From: |
Markus Armbruster |
Subject: |
Re: [Qemu-devel] [PATCH v2 0/6] external backup api |
Date: |
Fri, 19 Feb 2016 09:51:26 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) |
Fam Zheng <address@hidden> writes:
> On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
>> On Thu, Feb 18, 2016 at 12:11:14PM +0000, Daniel P. Berrange wrote:
>> > On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy
>> > wrote:
>> > > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
>> > > >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> > > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>> > > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>> > > >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> > > >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
>> > > >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>> > > >>>>>>>Hi all.
>> > > >>>>>>>
>> > > >>>>>>>These series which aims to add external backup api. This is
>> > > >>>>>>>needed to allow
>> > > >>>>>>>backup software use our dirty bitmaps.
>> > > >>>>>>>
>> > > >>>>>>>Vmware and Parallels Cloud Server have this feature.
>> > > >>>>>>What is the advantage of this appraoch over "drive-backup
>> > > >>>>>>sync=incremental
>> > > >>>>>>..."?
>> > > >>>>>This will allow third-party vendors to backup QEMU VMs into
>> > > >>>>>their own formats or to the cloud etc.
>> > > >>>>As an example, there is a third-party backup format called VMA from
>> > > >>>>Proxmox. A few years ago I posted a proof-of-concept external backup
>> > > >>>>tool in Python:
>> > > >>>>
>> > > >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>> > > >>>>
>> > > >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
>> > > >>>>but the same can be done with incremental backups.
>> > > >>>>
>> > > >>>>Does this NBD approach meet your requirements?
>> > > >>>>
>> > > >>>>Stefan
>> > > >>>for us we should somehow provide implementation of
>> > > >>>calls posted by Vladimir. They are available in Parallels Server
>> > > >>>version 6 and should be available in the next QEMU based
>> > > >>>release using "Parallels SDK to libvirt" convertor. The problem
>> > > >>>for us is that this old approach is used in the other side
>> > > >>>of the product - in containers implementation while this
>> > > >>>SDK is a universal access tool to both things.
>> > > >>Point taken. I think many other backup applications will expect a
>> > > >>similar API, so it's pragmatic to provide something compatible.
>> > > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
>> > > >concerns about the QMP monitor:
>> > > >
>> > > >Previously I described incremental backup in "push" mode (already
>> > > >supported today with drive-backup). QEMU connects to a remote NBD
>> > > >server and writes out the contents of all dirty blocks:
>> > > >
>> > > > QEMU ---Write dirty blocks--> Backup appliance (server)
>> > > >
>> > > >This doesn't lend itself well to existing backup applications that
>> > > >expect to iterate the dirty bitmap manually.
>> > > >
>> > > >Let's add a "pull" mode where the connection of the NBD connection is
>> > > >reversed. The backup application connects to QEMU's NBD server (image
>> > > >fleecing). The NBD protocol is extended to support the SCSI Get LBA
>> > > >Status command for querying block provisioning information. Now the
>> > > >backup application can use Get LBA Status to fetch the dirty block
>> > > >information from QEMU.
>> > > >
>> > > > QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup
>> > > > appliance
>> > > >
>> > > >The dirty block information goes over the same NBD connection used to
>> > > >read the contents of the dirty blocks. The QMP monitor is not used to
>> > > >transfer dirty block information.
>> > > >
>> > > >It may be necessary to extend the nbd-server-add command so that a
>> > > >bitmap name can be passed. This bitmap will be used to answer Get LBA
>> > > >Status queries instead of using on bdrv_co_get_block_status(). This
>> > > >would be necessary if image fleecing (point in time snapshot) is used.
>> > > >
>> > > >Stefan
>> > >
>> > > There are no such commands in nbd spec here:
>> > >
>> > > https://github.com/yoe/nbd/blob/master/doc/proto.md
>> > >
>> > >
>> > > So, I'm not sure, that adding something qemu-specific to this external
>> > > protocol will be simple or even true way. Is Qemu already extending
>> > > original
>> > > nbd?
>> >
>> > No, we don't do any QEMU specific extensions. The point of the approach
>> > Stefan suggests here though, is that it is *not* an inherantly
>> > QEMU-specific
>> > concept, it is relevant to any NBD server implementation.
>> >
>> > For example, consider you were using a regular NBD server to export a
>> > sparse LVM volume. This proposed extension would be relevant to such
>> > a use case. As such this proposed extension is something that is likely
>> > to be acceptable for the generic NBD specification.
>>
>> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
>>
>> NBD already supports a TRIM command so the ability to query the
>> allocation status is a natural feature to add.
>
> Is it an abuse to "Get LBA Status" to return dirty information? Because in
> SCSI
> the command reports "mapped", "allocated" and "anchored" statuses. Does that
> mean NBD will use a different status set?
Perhaps some conceptual gymnastics can get us to standard semantics.
Incremental backup wants to copy out an image's "dirty" blocks.
We can view this as a bitmap telling us which of the image's blocks are
dirty.
An alternative view would be base image + dirty delta image. In the the
dirty delta, exactly the dirty blocks are allocated. The delta image
may be conceptual.
Now, consider the dirty delta *without* its backing image. You can find
its allocated blocks with Get LBA Status. As usual, you can read even
unallocated blocks, see SBC3 table 9.
If we NBD-export exactly that, we can use standard semantics, can't we?
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, (continued)
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Stefan Hajnoczi, 2016/02/16
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Vladimir Sementsov-Ogievskiy, 2016/02/16
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Denis V. Lunev, 2016/02/16
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Stefan Hajnoczi, 2016/02/18
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Markus Armbruster, 2016/02/18
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Vladimir Sementsov-Ogievskiy, 2016/02/17
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Fam Zheng, 2016/02/17
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Daniel P. Berrange, 2016/02/18
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Stefan Hajnoczi, 2016/02/18
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Fam Zheng, 2016/02/18
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api,
Markus Armbruster <=
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, John Snow, 2016/02/24
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Paolo Bonzini, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Paolo Bonzini, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Denis V. Lunev, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, John Snow, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Denis V. Lunev, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Fam Zheng, 2016/02/26
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Markus Armbruster, 2016/02/29
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Paolo Bonzini, 2016/02/29
- Re: [Qemu-devel] [PATCH v2 0/6] external backup api, Paolo Bonzini, 2016/02/29