[Qemu-devel] Re: Virtual PC images still corrupt - but better than before

[Kevin Wolf]

Jamie Lokier schrieb:
> There were changes to VPC support merged from qemu-svn into kvm-84.
> I used "qemu-img convert" from kvm-88 for testing, except that I
> confirmed the size issue occurs with kvm-84 too:
> 
>   1. Strikingly, it produces the wrong raw image size.
> 
>      Firstly, the new code interprets the disk image with the wrong size:
> 
>      $ /usr/local/kvm-83/bin/kvm-img info original.vpc
>      image: original.vpc
>      file format: vpc
>      virtual size: 78G (83886080000 bytes)
>      disk size: 6.0G
> 
>      $ /usr/local/kvm-84/bin/kvm-img info original.vpc
>      image: original.vpc
>      file format: vpc
>      virtual size: 78G (83884277760 bytes)
>      disk size: 6.0G
> 
>      $ ls -l expanded_by_VirtualPC.vhd 
>      -rw-r--r-- 1 jamie jamie 83886080512 2008-05-26 07:47 
> expanded_by_VirtualPC.vhd
> 
>      The correct image size is probably 83886080000 bytes, which means
>      the older code calculated it correctly and the new code does not.

I think this depends on the definition of "correct". VPC unfortunately
seems to like contradicting values in its image - CHS geometry and image
size rarely match. The guest seemed to see the CHS geometry, so I
changed the code to take the geometry instead of the size header field
to calculate the image size.

This change resulted in md5sum /dev/hda giving the same result in qemu
and VPC, which I consider more correct than it was before. I haven't
tested with a wide range of guests, though.

>      Microsoft's conversion tool adds an extra 512 bytes on the end,
>      which are all zeros and probably aren't required for qemu/kvm.
> 
>      Unsurprisingly given the above output, "qemu-img convert"
>      produces raw image of the wrong size too - the size reported as
>      "virtual size".
> 
>      I checked the partition size on this image, and found the
>      partition used for real data to be smaller than the smaller size
>      produced by the new VPC format code.  So I looked at the extra
>      1802240 bytes, and found they were all zeros.

This is probably just the MS conversion tool taking the image size
header which is usually larger than the CHS geometry, but remains empty
as the guest cannot see it in VPC.

>   2. The amount of corrupt sectors is very much improved.
> 
>      Whereas kvm-72 produced a huge number of corrupt sectors - more
>      than I counted, kvm-88 produced just 3 corrupt sectors of 512
>      bytes in the entire 80GB file.  In two of them, the correct file
>      has data where the kvm-88 output has zeros; in the other corrupt
>      sector, kvm-88 has non-zeros where the correct file has zeros.
> 
>      (I'm assuming the output from Microsoft's tool is correct).
> 
>      That's close to perfect, but obviously not perfect enough.
> 
>      I have the offsets of the corrupt sectors, so I'm hoping it won't
>      be hard to reproduce by attempting to read just those sectors
>      through the block-vpc.c code and deducing where it goes wrong.

You could have a try with qemu-io if you know the offsets. Should be as
easy as "read -v $OFFSET 512". The VHD format actually is pretty easy,
too, so it shouldn't be too hard to calculate the sector's position in
the image file manually for comparison.

I suspect that it's something with a sector set to "(not) allocated" in
the block bitmap. The qemu driver currently ignores this for reading as
it should only be an optimization. So if the bitmap says the block is
empty and in fact it isn't, you'd have a corrupted image.

Kevin