Re: [Qemu-devel] Using cache=writeback safely on qemu 1.4.0 and later

[Stefan Hajnoczi]

On Fri, Aug 15, 2014 at 03:18:15PM -0500, Andrew Martin wrote:
> Hello,
> 
> I am running several qemu-kvm VM servers on Ubuntu 12.04 with qemu-kvm 1.4.0.
> Most of the guests are also running Ubuntu 12.04. I am using qcow2 disk images
> with the virtio driver in almost all cases, and am storing the disks on two
> types of storage devices: 
> * a shared NFS server mounted on each host 
> * local ext4 filesystems on each host (on top of an md RAID1 of HDDs)
> 
> It seems like the consensus for NFS is to use cache=none in order to support
> live migration between hosts.
> 
> For local ext4 filesystems, I see from this message and elsewhere that
> cache=writeback is now the default disk cache mode, and is safe on ext4:
> http://lists.gnu.org/archive/html/qemu-devel/2012-02/msg02689.html
> 
> All of the hosts and guests are using ext4 (with the default
> data=ordered,barrier). Thus it seems probable that the kernel (3.2.x or newer)
> should support flushing correctly? Is there a way to check in the guest is
> utilizing WCE (or any other required components) correctly?

If you strace -f the QEMU process on the host, you will see fdatasync(2)
system calls when the guest flushes the disk.

You can find the file descriptor number by checking ls -l
/proc/$PID_OF_QEMU/fd and looking for the disk image file.

> I recently experienced UPS failure on several hosts which caused a hard
> shutdown. After restarting, 3 of the guests had corruption on their disks and
> required a fairly long fsck to fix. Afterwards, data that had been written to
> the disks several hours before the crash was corrupted, which makes me think
> that it was never fsync()-ed to the non-volatile storage.

What exactly was the "corruption" you encountered?  Which application,
error message, etc.

> Is it safe in this setup to use cache=writeback? Or, should I use
> cache=writethrough instead?

Ubuntu 12.04 is recent and sends write cache flushes.

Are you sure the file system and/or application workload are flushing
the disk cache?  Please check the mount options and application-specific
configuration.

Stefan

pgpgCbHxPGcHW.pgp
Description: PGP signature