[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH V6 4/6] hw/block/nvme: support for multi-controller in subsys
From: |
Keith Busch |
Subject: |
Re: [PATCH V6 4/6] hw/block/nvme: support for multi-controller in subsystem |
Date: |
Tue, 26 Jan 2021 09:57:23 -0800 |
On Tue, Jan 26, 2021 at 09:52:48AM +0900, Minwoo Im wrote:
> On 21-01-25 10:11:43, Keith Busch wrote:
> > On Mon, Jan 25, 2021 at 07:03:32PM +0100, Klaus Jensen wrote:
> > > On Jan 24 11:54, Minwoo Im wrote:
> > > > We have nvme-subsys and nvme devices mapped together. To support
> > > > multi-controller scheme to this setup, controller identifier(id) has to
> > > > be managed. Earlier, cntlid(controller id) used to be always 0 because
> > > > we didn't have any subsystem scheme that controller id matters.
> > > >
> > > > This patch introduced 'cntlid' attribute to the nvme controller
> > > > instance(NvmeCtrl) and make it allocated by the nvme-subsys device
> > > > mapped to the controller. If nvme-subsys is not given to the
> > > > controller, then it will always be 0 as it was.
> > > >
> > > > Added 'ctrls' array in the nvme-subsys instance to manage attached
> > > > controllers to the subsystem with a limit(32). This patch didn't take
> > > > list for the controllers to make it seamless with nvme-ns device.
> > > >
> > > > Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com>
> > > > ---
> > > > hw/block/nvme-subsys.c | 21 +++++++++++++++++++++
> > > > hw/block/nvme-subsys.h | 4 ++++
> > > > hw/block/nvme.c | 29 +++++++++++++++++++++++++++++
> > > > hw/block/nvme.h | 1 +
> > > > 4 files changed, 55 insertions(+)
> > > >
> > > > diff --git a/hw/block/nvme.c b/hw/block/nvme.c
> > > > index b525fca14103..7138389be4bd 100644
> > > > --- a/hw/block/nvme.c
> > > > +++ b/hw/block/nvme.c
> > > > @@ -4481,6 +4484,10 @@ static void nvme_init_ctrl(NvmeCtrl *n,
> > > > PCIDevice *pci_dev)
> > > > id->psd[0].enlat = cpu_to_le32(0x10);
> > > > id->psd[0].exlat = cpu_to_le32(0x4);
> > > >
> > > > + if (n->subsys) {
> > > > + id->cmic |= NVME_CMIC_MULTI_CTRL;
> > > > + }
> > >
> > > Since multiple controllers show up with a PCIe port of their own, do we
> > > need to set bit 0 (NVME_CMIC_MULTI_PORT?) as well? Or am I
> > > misunderstanding that bit?
> >
> > AIUI, if you report this MULTI_PORT bit, then each PCI device in the
> > subsystem needs to report a different "Port Number" in their PCIe Link
> > Capabilities register. I don't think we can manipulate that value from
> > the nvme "device", but I also don't know what a host should do with this
> > information even if we could. So, I think it's safe to leave it at 0.
>
> AFAIK, If we leave it to 0, kernel will not allocate disk for multi-path
> case (e.g., nvmeXcYnZ).
Kernel only checks for MULTI_CTRL. It doesn't do anything with MULTI_PORT.
[PATCH V6 5/6] hw/block/nvme: add NMIC enum value for Identify Namespace, Minwoo Im, 2021/01/23
[PATCH V6 6/6] hw/block/nvme: support for shared namespace in subsystem, Minwoo Im, 2021/01/23
Re: [PATCH V6 0/6] hw/block/nvme: support multi-path for ctrl/ns, Klaus Jensen, 2021/01/25
Re: [PATCH V6 0/6] hw/block/nvme: support multi-path for ctrl/ns, Keith Busch, 2021/01/26