Re: Clarification regarding new qemu-img convert --target-is-zero flag

[Vladimir Sementsov-Ogievskiy]

10.06.2020 15:19, Sam Eiderman wrote:
> Thanks David,
>
> Yes, I imaging the following use case:
>
> disk.vmdk is a 50 GB disk that contains 12 MB binary of zeroes in its beginning.
> /dev/sda is a raw disk containing garbage
>
> I invoke:
> qemu-img convert disk.vmdk -O raw /dev/sda
>
> Required output:
> The first 12 MB of /dev/sda contain zeros, the rest garbage, qemu-img
> finishes fast.
>
> Kevin, from what I understood from you, this is the default behavior.
>
> So if my VMDK is causing trouble (all virtual size is being written)
> this is probably since all the grains in the VMDK are zero allocated
> right?
>
> Thanks!

I'm not sure that skipping unallocated clusters in qcow2/vmdk is default. As I 
see,
briefly looking at the code, unallocated clusters are skipped with -B option. 
But
it assuming using some backing file, which is not your case.

Let's check:
]# ./qemu-img create -f raw b 1M
Formatting 'b', fmt=raw size=1048576
]# ./qemu-img create -f qcow2 a 1M
Formatting 'a', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off 
refcount_bits=16 compression_type=zlib
]# ./qemu-io -c 'write -P 0xff 0 1M' -f raw b
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 00.05 sec (21.646 MiB/sec and 21.6457 ops/sec)
]# xxd b | head
00000000: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000010: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000030: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000040: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000050: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000060: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff  ................
]# ./qemu-img convert -f qcow2 -O raw a b
]# xxd b | head
00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
]# ./qemu-io -c 'write -P 0xff 0 1M' -f raw b
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 00.05 sec (20.648 MiB/sec and 20.6478 ops/sec)
]# ./qemu-img create -f qcow2 base 1M
Formatting 'base', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off 
refcount_bits=16 compression_type=zlib
]# ./qemu-img convert -f qcow2 -O raw -B base a b
qemu-img: Backing file not supported for file format 'raw'


So you see, in a newly created qcow2 file all cllusters are unallocated. Still 
by default qemu-img convert writes all zeroes. And we can't use -B with raw 
tartget.

> On Wed, Jun 10, 2020 at 2:56 PM David Edmondson <dme@dme.org> wrote:
> > On Wednesday, 2020-06-10 at 08:28:29 +03, Sam Eiderman wrote:
> >
> > > Hi,
> > >
> > > 168468fe19c8 ("qemu-img: Add --target-is-zero to convert") has added a
> > > nice functionality for cloud scenarios:
> > >
> > > * Create a virtual disk
> > > * Convert a sparse image (qcow2, vmdk) to the virtual disk using
> > > --target-is-zero
> > > * Use the virtual disk
> > >
> > > This saves many unnecessary writes - a qcow2 with 1MB of allocated
> > > data but with 100GB virtual size will be converted efficiently.
> > >
> > > However, does this pose a problem if the virtual disk is not zero initialized?
> >
> > As Vladimir indicated, the intent of the flag is supposed to be clear
> > from the name :-) If your storage doesn't read zeroes absent any earlier
> > writes, you probably don't want to be using it.
> >
> > > Theoretically - if all unallocated blocks contain garbage - this
> > > shouldn't matter, however what about allocated blocks of zero? Will
> > > convert skip copying allocated zero blocks in the source image to the
> > > target since it assumes that the target is zeroed out first thing?
> >
> > So something like a "--no-need-to-zero" flag would do what you want,
> > presuming that it would write known zeroes but no longer clean the
> > device before use?
> >
> > dme.
> > --
> > You can't hide from the flipside.


--
Best regards,
Vladimir