[Qemu-discuss] Understanding QEMU Cache Modes and Emulated Disk Write Cache

[Andrew Martin]

Hello,

I am running QEMU 2.5 on Ubuntu 16.04. My guests are a mixture of Linux (Ubuntu
12.04 and newer with ext4) and Windows. The guest qcow2 images are stored in a 
ZFS filesystem with sync=standard (on top of a mirrored vdev of SSDs) on the 
host
Ubuntu 16.04 OS. After having read through documentation and some previous 
threads on this list, I am still somewhat confused about caching.

First, my understanding is that if I strace the qemu process and see it
regularly issuing fdatasync(2), this means WCE is enabled and it is safe to use
cache=writeback, correct?

I see fdatasync(2) coming through about every 6 seconds, so I believe the guest
is properly flushing to disk. Now in this setup, my understanding is there are 4
caches and permanent storage:
* guest page cache
* guest disk write cache (virtio)
* host page cache
* host disk write cache (SSD)
* permanent storage

I understand that dirty pages get queued up in the guest page cache, and that
when an ext4 barrier is hit an fdatasync(2) call causes them to get flushed to
disk. At this point, does fdatasync(2) force the data into the guest disk write
cache, the host page cache, the host disk write cache, or permanent storage?
Does this vary depending on if the virtio disk is set to cache=writeback or
cache=writethrough? Can you explain how this works for both cases?

I am confused about the difference between the guest disk write cache and the
host page cache. Aren't these effectively the same (in both cases storing the
dirty data in the host's RAM)? Is there a difference between how one gets 
flushed and how the other gets flushed? Or do writes get written to the guest 
disk write cache first, then the host page cache?

Thanks for the help!

Andrew