Re: Bug? qemu-img convert to preallocated image makes it sparse

[Max Reitz]

On 16.01.20 15:13, Richard W.M. Jones wrote:
> I'm not necessarily saying this is a bug, but a change in behaviour in
> qemu has caused virt-v2v to fail.  The reproducer is quite simple.
> 
> Create sparse and preallocated qcow2 files of the same size:
> 
>   $ qemu-img create -f qcow2 sparse.qcow2 50M
>   Formatting 'sparse.qcow2', fmt=qcow2 size=52428800 cluster_size=65536 
> lazy_refcounts=off refcount_bits=16
> 
>   $ qemu-img create -f qcow2 prealloc.qcow2 50M -o 
> preallocation=falloc,compat=1.1
>   Formatting 'prealloc.qcow2', fmt=qcow2 size=52428800 compat=1.1 
> cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
> 
>   $ du -m sparse.qcow2 prealloc.qcow2 
>   1 sparse.qcow2
>   51  prealloc.qcow2
> 
> Now copy the sparse file into the preallocated file using the -n
> option so qemu-img doesn't create the target:
> 
>   $ qemu-img convert -p -n -f qcow2 -O qcow2 sparse.qcow2 prealloc.qcow2
>       (100.00/100%)
> 
> In new qemu that makes the target file sparse:
> 
>   $ du -m sparse.qcow2 prealloc.qcow2 
>   1 sparse.qcow2
>   1 prealloc.qcow2         <-- should still be 51
> 
> In old qemu the target file remained preallocated, which is what
> I and virt-v2v are expecting.
> 
> I bisected this to the following commit:
> 
> 4d7c487eac1652dfe4498fe84f32900ad461d61b is the first bad commit
> commit 4d7c487eac1652dfe4498fe84f32900ad461d61b
> Author: Max Reitz <address@hidden>
> Date:   Wed Jul 24 19:12:29 2019 +0200
> 
>     qemu-img: Fix bdrv_has_zero_init() use in convert
>     
>     bdrv_has_zero_init() only has meaning for newly created images or image
>     areas.  If qemu-img convert did not create the image itself, it cannot
>     rely on bdrv_has_zero_init()'s result to carry any meaning.
>     
>     Signed-off-by: Max Reitz <address@hidden>
>     Message-id: address@hidden
>     Reviewed-by: Maxim Levitsky <address@hidden>
>     Reviewed-by: Stefano Garzarella <address@hidden>
>     Signed-off-by: Max Reitz <address@hidden>
> 
>  qemu-img.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> Reverting this commit on the current master branch restores the
> expected behaviour.

The commit changed the behavior because now qemu-img realizes that it
cannot skip writing to areas that are supposed to be zero when it
converts to an existing image (because we have no idea what data that
existing image contains).  So that’s a bug fix, and I don’t think we can
undo it without being wrong.

The problem is that qemu-img will try to be quickthat about making these
areas zero, and so it does zero writes (actually, it even zeroes the
whole image) and in the process it will of course discard all preallocation.

Now, about fixing the problem I’m not so sure.

The problem is that it isn’t easy for qemu-img or the qcow2 driver to
determine whether a preallocated area is zero.  It needs to read the
data from disk and compare it to zero.  That would be slow and not trivial.

So off top of my head, the only thing that comes to my mind would be a
new flag for convert that lets you guarantee the image is zero and qemu
doesn’t need to zero it.

The problem with this is that I don’t think we ever guaranteed that
preallocated images stay preallocated when written to, and so even if we
assume the image is fully zero and thus restore the old behavior for
your case, we might break it in the future again.


So are there any ways to safely convert an image to an existing one and
keeping the destinations preallocation intact?  Sadly, I don’t think
there is.  Well, you can always use -S 0, but that will change your
falloc preallocation into a full one.

So I suppose the best idea I can come up with is indeed a
--target-is-zero flag for qemu-img convert -n.  Would that work for you?

Max

signature.asc
Description: OpenPGP digital signature