Re: [Qemu-block] [Libguestfs] [PATCH] v2v: Add --print-estimate option to print source size estimate.

[Kevin Wolf]

[ Cc: qemu-block ]

Am 15.08.2018 um 12:55 hat Richard W.M. Jones geschrieben:
> (Adding Stefan who implemented the subcommand)
> 
> On Wed, Aug 15, 2018 at 12:44:44PM +0200, Kevin Wolf wrote:
> > Am 15.08.2018 um 12:26 hat Richard W.M. Jones geschrieben:
> > > On Tue, Aug 14, 2018 at 09:31:06PM +0300, Nir Soffer wrote:
> > > > On Tue, Aug 14, 2018 at 8:29 PM Richard W.M. Jones <address@hidden>
> > > > wrote:
> > > > 
> > > > > This option prints the estimated size of the data that will be copied
> > > > > from the source disk.
> > > > >
> > > > > For interest, the test prints:
> > > > >
> > > > > 3747840 ../test-data/phony-guests/windows.img
> > > > > Estimate: 3710976
> > > > >
> > > > 
> > > > Why not use qemu-img measure on the overlay?
> > > 
> > > Yes this would be better, but oddly it doesn't work properly when
> > > the output format is set to 'raw':
> > > 
> > >   /usr/bin/qemu-img 'measure' '-f' 'qcow2' '-O' 'raw' '--output=human' 
> > > '/home/rjones/d/libguestfs/tmp/v2vovl62b7c2.qcow2'
> > >   required size: 6442450944
> > >   fully allocated size: 6442450944
> > > 
> > > However it's OK if the output format is set to 'qcow2':
> > > 
> > >   /usr/bin/qemu-img 'measure' '-f' 'qcow2' '-O' 'qcow2' '--output=human' 
> > > '/home/rjones/d/libguestfs/tmp/v2vovla53d7c.qcow2'
> > >   required size: 1047986176
> > >   fully allocated size: 6443696128
> > > 
> > > I guess it ignores sparseness of raw images, but I can't see a way to
> > > specify that on the command line.
> > > 
> > > OTOH the qcow2 figure is probably a good enough guess for our purposes
> > > (ie. estimating how much data will be copied).
> > 
> > 'qemu-img measure' calculates the resulting file size, not the number of
> > used blocks. I think this is because its main purpose is to create block
> > devices (like LVs) of the right size, so sparseness on a filesystem
> > level doesn't buy you anything.
> 
> But if I run ‘qemu-img convert ... -O raw output.raw’ then output.raw
> will be a sparse file, and that's the file size I'd expect measure to
> give us for "required size" (of course "fully allocated size" would be
> the virtual size, and that is correct).
> 
> It does look as if the block/raw-format.c:raw_measure function is wrong.

No, it is correct. Its output is what the file size will be. For raw
images, that is the same as the virtual disk size.

Not all blocks in the file will be actually allocated, but the file size
is what 'ls -l' prints, not what 'du' prints (for regular files).

It becomes even clearer when you create LVs as the target. If you have a
10 GB image in which only the last 1 MB actually contains data, you
still need a 10 GB volume. You can't create a 1 MB volume and then store
data at an offset 10 GB - 1 MB, that would be way after the end of the
block device.

> In any case we can use the qcow2 estimate for our purposes as it will
> be near enough what we need (a rough estimate of the size of the copy).

I don't know what the exact purpose is, and it feels a bit hacky, but it
might be good enough.

I suppose what you really want is that 'qemu-img measure' provides
another number for the space taken by allocated blocks. (Probably
excluding metadata for non-raw formats? Might depend on the actual
purpose.)

Kevin