Re: ioregionfd with io_uring IORING_OP_URING_CMD

[Elena]

On Mon, Jun 06, 2022 at 03:57:47PM +0100, Stefan Hajnoczi wrote:
> Hi,
> During Elena Afanasova's Outreachy project we discussed whether
> ioregionfd should be a custom struct file_operations (anon inode) or a
> userspace-provided file (socketpair, UNIX domain socket, etc).
>

Hello Stefan,

> Back then it seemed more flexible and simpler to let userspace provide
> the file. It may be worth revisiting this decision in light of the
> recent io_uring IORING_OP_URING_CMD feature, which fits for this
> performance-critical interface.
>
And Paolo was asking about io_uring in the review of the initial
patches.

> IORING_OP_URING_CMD involves a new struct file_operations->uring_cmd()
> callback. It's a flexible userspace interface like ioctl(2) but designed
> to be asynchronous. ioregionfd can provide a uring_cmd() that reads a
> ioregionfd response from userspace and then writes the next ioregionfd
> request to userspace.
> 
> This single operation merges the request/response so only 1 syscall is
> necessary per KVM MMIO/PIO exit instead of a read() + write(). Bypassing
> the net/socket infrastructure is likely to help too.
> 
> It would be interesting to benchmark this and compare it against the
> existing userspace-provided file approach. Although it's not the same
> scenario, results for the Linux NVMe driver using ->uring_cmd() are
> promising:
> https://www.snia.org/educational-library/enabling-asynchronous-i-o-passthru-nvme-native-applications-2021
> 
Yes, looks interesting, we were thinking about adding this.

> The downside is it requires more code than general purpose I/O. In
> addition to ->uring_cmd(), it's also worth implementing struct
> file_operations read/write/poll so traditional file I/O syscalls work
> for simple applications that don't want to use io_uring.
> 
> It's possible to add ->uring_cmd() later but as a userspace developer I
> would prefer the ->uring_cmd() approach, so I'm not sure it's worth
> committing to the existing userspace-provided file approach?

Makes total sense. I am going to start working on this and will
come back with more questions.

Thank you!
Elena
> 
> Stefan