[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-discuss] Discard disk image holes for sparse shrink

From: Friedrich Oslage
Subject: Re: [Qemu-discuss] Discard disk image holes for sparse shrink
Date: Tue, 9 Apr 2019 19:19:25 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.3

1. You need to set the discard option for the -drive parameter, eg -drive if=none,...,format=raw,discard=unmap since the default is still to simply ignore discards.

You may also want to set the detect-zeroes option to unmap, to discard all-zero blocks instead of actually writing them.

2. The disk image you created is thick-provisioned. How is qemu supposed to discard anything in that file? It could turn it into a sparse file or overwrite it with zeroes I suppose, but both options are undesirable.

To make use of the discard operation your backing storage has to support it. Either by beeing thin-provisioned, like qcow2 or raw sparse files, or by actually beeing a ssd/nvme disk.

3. Your testing method is flawed. You are using a journaling filesystem to write the data but to read it you access the block device directly . Even if the discard operation was working strings(1) would still show the hello-world string in the journal.

You could try something like this:

$ echo hello, world > /dev/sda
$ strings /dev/sda
$ blkdiscard /dev/sda
$ strings /dev/sda


On 4/9/19 4:38 PM, Pascal wrote:
the blocks do not seem to be discarded on a raw format disk: the data
remains on the disk...

on host :

dd if=/dev/zero of=/tmp/ssd.disk bs=1M count=128
qemu -drive media=disk,file=linux.disk -device virtio-scsi-pci,id=scsi
-device scsi-hd,drive=hd -drive if=none,id=hd,file=/tmp/ssd.disk,format=raw

on Linux VM :

fdisk /dev/sda
     # create one partition on gpt type...
fdisk -l /dev/sda
     Disk /dev/sda: 128 MiB, 134217728 bytes, 262144 sectors
     Units: sectors of 1 * 512 = 512 bytes
     Sector size (logical/physical): 512 bytes / 512 bytes
     I/O size (minimum/optimal): 512 bytes / 512 bytes
     Disklabel type: gpt
     Disk identifier: B8D98B8E-7790-4043-8F37-D4D8CA218884
     Device         Start    End Sectors  Size Type
     /dev/sda1  2048 262110  260063  127M Linux filesystem
mkfs.ext4 -L ssd /dev/sda1
mount /dev/sda1 /ssd
     /dev/sda1 on /ssd type ext4 (rw,relatime,data=ordered)
echo hello world > /ssd/test
cat /ssd/test
     hello world
sync && sysctl -q vm.drop_caches=3
rm -f /ssd/test
sync && sysctl -q vm.drop_caches=3
fstrim /ssd
umount /ssd
strings /dev/sda1 | grep 'hello world'
     hello world

Le mar. 9 avr. 2019 à 08:36, Narcis Garcia <address@hidden> a écrit :

Thank you.

* I use RAW images because of it's easier to offline mount/chroot when
some issue happens.
* I call directly qemu instead of libvirt

I'll try to parse these parameters to a qemu-system call.

El 8/4/19 a les 22:03, Friedrich Oslage ha escrit:

You'll have to use the virtio-scsi driver, to my knowledge it's the only
driver that supports block discards.

Quick example:

# create a new empty disk
$ qemu-img create -f qcow2 disk.qcow2 10G

# check size, should only be a few kilobytes
$ du -sh disk.qcow2

# add it to your vm
$ virtsh edit your-vm
   <disk type='block' device='disk'>
     <driver name='qemu' type='qcow2' discard='unmap'
     <source dev='/tmp/disk.qcow2'/>
     <target dev='sda' bus='scsi'/>
     <address type='drive' controller='0' bus='0' target='0' unit='0'/>
   <controller type='scsi' index='0' model='virtio-scsi'>
     <driver queues='4'/>

# in your vm, format and mount it
$ mkfs.ext4 /dev/sda
$ mount /dev/sda /mnt/discardtest

# put some data on it
$ dd if=/dev/urandom of=/mnt/discardtest/dummy.bin bs=1M count=1000
$ sync

# on host, check size...it should be about 1GB
$ du -sh disk.qcow2

# in vm, delete file and trim it
$ rm /mnt/discardtest/dummy.bin
$ sync
$ fstrim /mnt/discardtest

# on host, check size...should only be a few megabytes
$ du -sh disk.qcow2

It works with other guest OSes as well, for Windows you just use the
Optimize-Volume cmdlets instead of fstrim.

You also don't have to use qcow2 for backing. Anything that can handle
discards will do, including sparse files.

My advice, however, would be to use qcow2, since sparse files don't work
particularly well. Executing the example above would leave you with
about 300MB at the end instead of less than 10MB.


On 4/5/19 4:04 PM, Narcis Garcia wrote:
No; I want to study the possibility of recovering allocated blocs of
host filesystem when guest filesystem removes its files (unallocates its
Host -> HD or SSD (independent) with sparse-mode image
Guest -> Virtual SSD (to signal discards)

El 5/4/19 a les 16:01, Pascal ha escrit:

if I understand correctly, you want to study the possibility of
recovering deleted files from an SSD disk: is that right ?

regards, lacsaP.

Le jeu. 4 avr. 2019 à 08:24, Narcis Garcia <address@hidden
<mailto:address@hidden>> a écrit :

      Is there a way to specify a disk to be detected as an SSD drive?

      Once reached this, I want to look for the possibility to recover
      space when a guest discards disk image blocks, and this image is
      format and sparse allocated file.

      Thank you.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]