qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Domain backup file explodes on s3fs


From: Eric Blake
Subject: Re: Domain backup file explodes on s3fs
Date: Tue, 7 Apr 2020 14:37:26 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0

[adding libvirt list]

On 4/7/20 2:13 PM, Tim Haley wrote:
Hi all,

Have been playing with `virsh backup-begin` of late and think it's an excellent feature. I've noticed one behavior I'm not sure I understand.

It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1814664 is a similar description of the same problem: namely, if qemu is not able to determine that the destination already reads as zero, then it forcefully zeroes the destination of a backup job. We may want to copy the fact that qemu 5.0 is adding 'qemu-img convert --target-is-zero' to add a similar knob to the QMP commands that trigger disk copying (blockdev-backup, blockdev-mirror, possibly others) as well as logic to avoid writing zeroes when the destination is already treated as zero (whether by a probe, or by the knob being set).

...

If my /backups directory is just XFS, I get a backup file that looks like it is just the size of data blocks in use

-rw------- 1 root  root  2769551360 Mar 19 16:56 vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a

For a local file, qemu is easily able to probe whether the destination starts as all zeroes (thanks to lseek(SEEK_DATA));


but if I write to an s3fs (object storage backend) the file blows up to the whole size of the disk

-rw------- 1 root  root  8591507456 Mar 18 19:03 vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a

whereas for s3fs, it looks like qemu does not have access to a quick test to learn if the image starts all zero (POSIX does not provide a quick way for doing this on a generic block device, but if you are aware of an ioctl or otherwise that qemu could use, that might be helpful). Or maybe the s3fs really is random contents rather than all zero, in which case forcefully writing zeroes is the only correct behavior.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org




reply via email to

[Prev in Thread] Current Thread [Next in Thread]