[RFC PATCH V2 00/11] hw/block/nvme: support multi-path for ctrl/ns

[Minwoo Im]

Hello,

This patch series introduces NVMe subsystem device to support multi-path
I/O in NVMe device model.  Two use-cases are supported along with this
patch: Multi-controller, Namespace Sharing.

V1 RFC has been discussed with Klaus and Keith, I really appreciate them
for this patch series to have proper direction [1].

This patch series contains few start-up refactoring pathces from the
first to fifth patches to make nvme-ns device not to rely on the nvme
controller always.  Because nvme-ns shall be able to be mapped to the
subsystem level, not a single controller level so that it should provide
generic initialization code: nvme_ns_setup() with NvmeCtrl.  To do that,
the first five patches are to remove the NvmeCtrl * instance argument
from the nvme_ns_setup().  I'd be happy if they are picked!

For controller and namespace devices, 'subsys' property has been
introduced to map them to a subsystem.  If multi-controller needed, we
can specify 'subsys' to controllers the same.

For namespace deivice, if 'subsys' is not given just like it was, it
will have to be provided with 'bus' parameter to specify a nvme
controller device to attach, it means, they are mutual-exlusive.  To
share a namespace between or among controllers, then nvme-ns should have
'subsys' property to a single nvme subsystem instance.  To make a
namespace private one, then we need to specify 'bus' property rather
than the 'subsys'.

Of course, this series does not require any updates for the run command
for the previos users.

Plase refer the following example with nvme-cli output:

QEMU Run:
  -device nvme-subsys,id=subsys0 \
  -device nvme,serial=foo,id=nvme0,subsys=subsys0 \
  -device nvme,serial=bar,id=nvme1,subsys=subsys0 \
  -device nvme,serial=baz,id=nvme2,subsys=subsys0 \
  -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0 \
  -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 \
  \
  -device nvme,serial=qux,id=nvme3 \
  -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

nvme-cli:
  root@vm:~/work# nvme list -v
  NVM Express Subsystems

  Subsystem        Subsystem-NQN                                                
                                    Controllers
  ---------------- 
------------------------------------------------------------------------------------------------
 ----------------
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                 
                                    nvme0, nvme1, nvme2
  nvme-subsys3     nqn.2019-08.org.qemu:qux                                     
                                    nvme3

  NVM Express Controllers

  Device   SN                   MN                                       FR     
  TxPort Address        Subsystem    Namespaces
  -------- -------------------- ---------------------------------------- 
-------- ------ -------------- ------------ ----------------
  nvme0    foo                  QEMU NVMe Ctrl                           1.0    
  pcie   0000:00:06.0   nvme-subsys1 nvme1n1
  nvme1    bar                  QEMU NVMe Ctrl                           1.0    
  pcie   0000:00:07.0   nvme-subsys1 nvme1n1
  nvme2    baz                  QEMU NVMe Ctrl                           1.0    
  pcie   0000:00:08.0   nvme-subsys1 nvme1n1, nvme1n2
  nvme3    qux                  QEMU NVMe Ctrl                           1.0    
  pcie   0000:00:09.0   nvme-subsys3

  NVM Express Namespaces

  Device       NSID     Usage                      Format           Controllers
  ------------ -------- -------------------------- ---------------- 
----------------
  nvme1n1      1        134.22  MB / 134.22  MB    512   B +  0 B   nvme0, 
nvme1, nvme2
  nvme1n2      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme2
  nvme3n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme3

Summary:
  - Refactored nvme-ns device not to rely on controller during the
    setup.  [1/11 - 5/11]
  - Introduced a nvme-subsys device model. [6/11]
  - Create subsystem NQN based on subsystem. [7/11]
  - Introduced multi-controller model. [8/11 - 9/11]
  - Updated namespace sharing scheme to be based on nvme-subsys
    hierarchy. [10/11 - 11/11]

Since RFC V1:
  - Updated namespace sharing scheme to be based on nvme-subsys
    hierarchy.

Thanks,

[1] https://lists.gnu.org/archive/html/qemu-block/2021-01/msg00425.html

Minwoo Im (11):
  hw/block/nvme: remove unused argument in nvme_ns_init_zoned
  hw/block/nvme: open code for volatile write cache
  hw/block/nvme: remove unused argument in nvme_ns_init_blk
  hw/block/nvme: split setup and register for namespace
  hw/block/nvme: remove unused argument in nvme_ns_setup
  hw/block/nvme: introduce nvme-subsys device
  hw/block/nvme: support to map controller to a subsystem
  hw/block/nvme: add CMIC enum value for Identify Controller
  hw/block/nvme: support for multi-controller in subsystem
  hw/block/nvme: add NMIC enum value for Identify Namespace
  hw/block/nvme: support for shared namespace in subsystem

 hw/block/meson.build   |   2 +-
 hw/block/nvme-ns.c     |  40 ++++++++++------
 hw/block/nvme-ns.h     |   9 +++-
 hw/block/nvme-subsys.c | 101 +++++++++++++++++++++++++++++++++++++++++
 hw/block/nvme-subsys.h |  30 ++++++++++++
 hw/block/nvme.c        |  92 ++++++++++++++++++++++++++++++++-----
 hw/block/nvme.h        |   5 +-
 include/block/nvme.h   |   8 ++++
 8 files changed, 259 insertions(+), 28 deletions(-)
 create mode 100644 hw/block/nvme-subsys.c
 create mode 100644 hw/block/nvme-subsys.h

-- 
2.17.1