Re: [PATCH] make vfio and DAX cache work together

[Dev Audsin]

Hi Alex and David

@Alex:

Justification on why this region cannot be a DMA target for the device,

virtio-fs with DAX is currently not compatible with NIC Pass through.
When a SR-IOV VF attaches to a qemu process, vfio will try to pin the
entire DAX Window but it is empty when the guest boots and will fail.
A method to make VFIO and DAX to work together is to make vfio skip
DAX cache.

Currently DAX cache need to be set to 0, for the SR-IOV VF to be
attached to Kata containers. Enabling both SR-IOV VF and DAX work
together will potentially improve performance for workloads which are
I/O and network intensive.

@David

1. If DAX mode of virtiofs isn't yet in qemu, what is the best project
that this could be discussed?
2a. Regarding your comment on hard coding the name, I am referring to
the device by its name, which has been initialised in
hw/virtio/vhost-user-fs.c. I downloaded the source code of qemu with
virtiofs support which I obtained in reference to the Kata container
project and analysed it.  I see the following  code snippet in
hw/virtio/vhost-user-fs
  if (fs->conf.cache_size) {
        /* Anonymous, private memory is not counted as overcommit */
        cache_ptr = mmap(NULL, fs->conf.cache_size, DAX_WINDOW_PROT,
                         MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
        if (cache_ptr == MAP_FAILED) {
            error_setg(errp, "Unable to mmap blank cache");
            return;
        }

        memory_region_init_ram_ptr(&fs->cache, OBJECT(vdev),
                                   "virtio-fs-cache",
                                   fs->conf.cache_size, cache_ptr);
    }

In the above code snippet, the memory region is initialised with
device name  "virtio-fs-cache",which I am referring to in my source
code.

2b. Regarding, needing a way for the cache to declare it wants to be
omitted, I am not sure thats what is needed. Currently virtio-fs with
DAX is currently not compatible with vfio. I want to overcome this
problem by making vfio not using the cache.
What I want is cache to be used for purposes other than the VFIO
device. For example, in my deployment scenario, I want DAX cache to be
not used by SR-IOV VF (which is a VFIO device) but used by all other
system.

3. Moved to  vfio_listener_skip_section () and patch resubmitted.

Best
Dev