Re: [ANNOUNCE] libblkio v0.1.0 preview release

[Richard W.M. Jones]

On Thu, Apr 29, 2021 at 03:05:50PM +0100, Stefan Hajnoczi wrote:
> The purpose of this preview release is to discuss both the API design
> and general direction of the project. API documentation is available
> here:
> 
>   https://gitlab.com/libblkio/libblkio/-/blob/v0.1.0/docs/blkio.rst

libvirt originally, and now libnbd, keep a per-thread error message
(stored in thread-local storage).  It's a lot nicer than having to
pass &errmsg to every function.  You can just write:

 if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
   fprintf (stderr,
            "failed to connect to remote server: %s (errno = %d)\n",
            nbd_get_error (), nbd_get_errno ());
   exit (EXIT_FAILURE);
 }

(https://libguestfs.org/libnbd.3.html#ERROR-HANDLING)

It means you can extend the range of error information available in
future.  Also you can return a 'const char *' and the application
doesn't have to worry about lifetimes, at least in the common case.

> Examples are available here:
> 
>   https://gitlab.com/libblkio/libblkio/-/tree/v0.1.0/examples
> 
> The goal is to eventually include the following drivers:
> - Linux io_uring
> - NVMe (VFIO and vfio-user)
> - virtio-blk (VFIO, vfio-user, vhost-user-blk, and vhost-vdpa-blk)
>
> There are a few reasons why libblkio is needed:
> 
> 1. QEMU, Ceph, GlusterFS, MariaDB, and other programs have been adding
>    more low-level block I/O code, most of it duplicated. Usually only
>    one or two of Linux AIO, io_uring, userspace drivers, vhost-user
>    drivers, etc are implemented. This makes it difficult to benefit from
>    the latest advances in high-performance block I/O.
> 
> 2. Coding to a standard API makes it possible to introduce new
>    optimizations or hardware interfaces without costly changes to the
>    software stack.
> 
> 3. A client library is needed so applications can take advantage of
>    qemu-storage-daemon's vhost-user-blk exports.
> 
> 4. Implementing block I/O as a library allows QEMU to use Rust for new
>    code without messy QEMU internal API bindings. Note that libblkio
>    currently does not provide a Rust crate, it only offers a C API.

This is where I get confused about what this library actually does.
It's not just a nicer wrapper around io_uring, but what is it actually
doing?

Rich.

> For QEMU integration the next step is a libblkio BlockDriver. This new
> BlockDriver will provide libblkio functionality to QEMU. Eventually
> libblkio will provide block/file-posix.c, block/nvme.c, and the upcoming
> vhost-vdpa-blk functionality and the QEMU code can be simplified or
> removed.
> 
> libblkio does not contain an event loop implementation or disk image
> format drivers and that functionality will not be part of libblkio.
> 
> How to participate:
> 1. Share your thoughts on the direction and your requirements.
> 2. Review the API docs and give feedback.
> 3. Write a libblkio driver for NVMe VFIO/vfio-user, virtio-blk
>    VFIO/vfio-user, vhost-user-blk. Or just a null driver for
>    benchmarking/testing :).
> 4. Integrate libblkio into Ceph, GlusterFS, MariaDB, etc.
> 
> I am now beginning QEMU and fio integration. Stefano Garzarella is
> looking at adding a vhost-vdpa-blk driver to libblkio.
> 
> Stefan



-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html