Re: [Qemu-devel] [Bug 1399191] [NEW] Large VHDX image size

[AMULYA L]

-----Original Message-----
From: Jeff Cody [mailto:address@hidden
Sent: Wednesday, January 21, 2015 2:41 AM
To: Lokesha, Amulya
Cc: Bug 1399191; Geoffroy, Daniel
Subject: Re: [Qemu-devel] [Bug 1399191] [NEW] Large VHDX image size

On Mon, Jan 19, 2015 at 03:19:05PM +0000, Lokesha, Amulya wrote:
>
>
> -----Original Message-----
> From: Jeff Cody [mailto:address@hidden
> Sent: Tuesday, January 13, 2015 9:18 AM
> To: Lokesha, Amulya
> Cc: Bug 1399191; Geoffroy, Daniel
> Subject: Re: [Qemu-devel] [Bug 1399191] [NEW] Large VHDX image size
>
> On Sun, Jan 11, 2015 at 11:33:31PM -0500, Lokesha, Amulya wrote:
> > Please find comments inline
> >
> > -----Original Message-----
> > From: address@hidden<mailto:address@hidden> [mailto:address@hidden On Behalf
> > Of Jeff Cody
> > Sent: Wednesday, January 07, 2015 9:11 PM
> > To: Lokesha, Amulya
> > Subject: Re: [Qemu-devel] [Bug 1399191] [NEW] Large VHDX image size
> >
> > On Wed, Jan 07, 2015 at 03:30:28PM +0000, Stefan Hajnoczi wrote:
> > > On Thu, Dec 04, 2014 at 01:31:38PM -0000, AMULYA L wrote:
> > > > Public bug reported:
> > > >
> > > > We are trying to convert a VMDK image to VHDX image for deploy
> > > > to HyperV Server ( SCVMM 2012 SP1) using qemu-img.  We tried
> > > > converting the image using both 'fixed' as well as 'dynamic'
> > > > format. We found that both the disks occupy the same size of 50GB.
> > > > When the same is done with VHD image, we found that the dynamic
> > > > disks are much lesser in size (5 GB) than the fixed disk (50GB).
> > > >
> > > > Why is that the VHDX generates large sized images for both the
> > > > formats?
> > > >
> > > > The following commands were used to convert the vmdk image to
> > > > VHDX format
> > >
> > > Jeff, did you fix this recently in commit
> > > 85b712c9d5b873562c864e72f69cbf0d87d2fe40 ("block: vhdx - set
> > > .bdrv_has_zero_init to bdrv_has_zero_init_1")?
> > >
> > > Stefan
> >
> >
> > Yes, although there has been a report that there are issues in the
> > resulting vhdx image in a MS server, after this patch.  I am going
> > to test and fix (if confirmed) once my MSDN subscription is renewed
> > (in process now, I expect it anytime).
> >
> > Jeff
> >
> > --
> >
> > Hi Jeff,
> >
> > Could you please give us a possible timeline of when fix will be
> > available. Our customers are waiting on this as we are blocked with
> > the image creation.
> >
> > Thanks, Amulya
>
> Hi Amulya,
>
> I have my MSDN stuff sorted out now, so give me a day or two get my test 
> machine up and running & testing.
>
> -Jeff
>
>
> Hi Jeff,
>
> Any updates on this? Where you able to get it tested
>
>  Thanks,
> -Amulya
>

Yes, I have sent a patch that I believe fixes the issue (I cc'ed you on
the patch).  If you wouldn't mind testing, and verifying that it fixes
your particular issue, that would be great.  I tested on Windows Server
2012 w/Hyper-V, and I was able to verify the original problem and the
fix.

On what the issue was:

The v1.0.0 spec for VHDX denotes that the FileOffsetMB portion of the
block state field is 'reserved' for blocks in the state
PAYLOAD_BLOCK_ZERO.  Before, we just went ahead and wrote the file
offset value for that block into the field, and just ignored it during
reads.

If we force the FileOffsetMB field to be zero, then Hyper-V is able to
read the images.  Furthermore, inspecting images converted by Hyper-V
shows that Hyper-V writes '0' for the FileOffsetMB field in BAT entries
that are in the PAYLOAD_BLOCK_ZERO state.

The patch I sent mimics that behavior, and forces any BAT writes of
PAYLOAD_BLOCK_ZERO state to have a FileOffsetMB value of 0.


Hi Jeff,

Thanks a lot for the fix. We tested the VHDX creation with the new patch
fix and were able to get it deployed successfully into our Windows
HyperV Server 2012.

However we have one more issue and were hoping if we could get any help.
We have a vmdk image (size 13MB) with a 500 GB data disk size. After
conversion of vmdk image to vhdx format, the image size is 126 GB.

# qemu-img info Test.vmdk
image: Test.vmdk
file format: vmdk
virtual size: 500G (536870912000 bytes)
disk size: 13M
cluster_size: 65536
Format specific information:
    cid: 4124953209
    parent cid: 4294967295
    create type: streamOptimized
    extents:
        [0]:
            compressed: true
            virtual size: 536870912000
            filename: Test.vmdk
            cluster size: 65536
            format:

qemu conversion to vhdx done as below
# qemu-img convert -p -o subformat=dynamic -f vmdk -O vhdx Test.vmdk Test.vhdx
    (100.00/100%)

# ls -ltrh
total 69M
-rw-r--r-- 1 root root  13M Jan 21 21:42 Test.vmdk
-rw-r--r-- 1 root root 126G Jan 22 05:03 Test.vhdx

# qemu-img info Test.vhdx
image: Test.vhdx
file format: vhdx
virtual size: 500G (536870912000 bytes)
disk size: 56M
cluster_size: 33554432


When we tried to copy this to the SCVMM manager server (Windows OS) it failed 
due to disk limitation. Windows does think it is a 126 GB file. Can anything be 
done to fix this issue?

Thanks,
Amulya
> >
> >
> > You received this bug notification because you are subscribed to the bug 
> > report.
> > https://bugs.launchpad.net/bugs/1399191
> >
> > Title:
> >   Large VHDX image size
> >
> > Status in QEMU:
> >   New
> >
> > Bug description:
> >   We are trying to convert a VMDK image to VHDX image for deploy to HyperV 
> > Server ( SCVMM 2012 SP1) using qemu-img.
> >   We tried converting the image using both 'fixed' as well as 'dynamic' 
> > format. We found that both the disks occupy the same size of 50GB. When the 
> > same is done with VHD image, we found that the dynamic disks are much 
> > lesser in size (5 GB) than the fixed disk (50GB).
> >
> >   Why is that the VHDX generates large sized images for both the
> >   formats?
> >
> >   The following commands were used to convert the vmdk image to VHDX
> >   format
> >
> >   1. qemu-img convert -p -o subformat=fixed  -f vmdk -O vhdx Test.vmdk
> >   Test-fixed.vhdx
> >
> >   qemu-img info Test-fixed.vhdx
> >   image: Test-fixed.vhdx
> >   file format: vhdx
> >   virtual size: 50G (53687091200 bytes)
> >   disk size: 50G
> >   cluster_size: 16777216
> >
> >
> >
> >   2. qemu-img convert -p -o subformat=dynamic  -f vmdk -O vhdx
> > Test.vmdk Test-dynamic.vhdx
> >
> >   qemu-img info Test-dynamic.vhdx
> >   image: Test-dynamic.vhdx
> >   file format: vhdx
> >   virtual size: 50G (53687091200 bytes)
> >   disk size: 50G
> >   cluster_size: 16777216
> >
> >
> >   We tried this with the following version of qemu
> >   1. qemu-2.0.0
> >   2. qemu-2.1.2
> >   3. qemu-2.2.0-rc4
> >
> >
> >   Please let us know how to create compact VHDX images using qemu-img.
> >   Thank you
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/qemu/+bug/1399191/+subscriptions

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1399191

Title:
  Large VHDX image size

Status in QEMU:
  New

Bug description:
  We are trying to convert a VMDK image to VHDX image for deploy to HyperV 
Server ( SCVMM 2012 SP1) using qemu-img.
  We tried converting the image using both 'fixed' as well as 'dynamic' format. 
We found that both the disks occupy the same size of 50GB. When the same is 
done with VHD image, we found that the dynamic disks are much lesser in size (5 
GB) than the fixed disk (50GB). 

  Why is that the VHDX generates large sized images for both the
  formats?

  The following commands were used to convert the vmdk image to VHDX
  format

  1. qemu-img convert -p -o subformat=fixed  -f vmdk -O vhdx Test.vmdk
  Test-fixed.vhdx

  qemu-img info Test-fixed.vhdx
  image: Test-fixed.vhdx
  file format: vhdx
  virtual size: 50G (53687091200 bytes)
  disk size: 50G
  cluster_size: 16777216


  
  2. qemu-img convert -p -o subformat=dynamic  -f vmdk -O vhdx Test.vmdk 
Test-dynamic.vhdx

  qemu-img info Test-dynamic.vhdx
  image: Test-dynamic.vhdx
  file format: vhdx
  virtual size: 50G (53687091200 bytes)
  disk size: 50G
  cluster_size: 16777216

  
  We tried this with the following version of qemu
  1. qemu-2.0.0
  2. qemu-2.1.2
  3. qemu-2.2.0-rc4

  
  Please let us know how to create compact VHDX images using qemu-img.
  Thank you

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1399191/+subscriptions