Re: [RFC 0/3] Introduce a new Qemu machine for RISC-V

[Atish Patra]

On Tue, Apr 19, 2022 at 9:51 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Mon, Apr 11, 2022 at 07:10:06PM -0700, Atish Patra wrote:
> >
> > The RISC-V virt machine has helped RISC-V software eco system to evolve at a
> > rapid pace even in absense of the real hardware. It is definitely 
> > commendable.
> > However, the number of devices & commandline options keeps growing as a 
> > result
> > of that as well. That adds flexibility but will also become bit difficult
> > to manage in the future as more extension support will be added. As it is 
> > the
> > most commonly used qemu machine, it needs to support all kinds of device and
> > interrupts as well. Moreover, virt machine has limitations on the maximum
> > number of harts it can support because of all the MMIO devices it has to 
> > support.
> >
> > The RISC-V IMSIC specification allows to develop machines completely relying
> > on MSI and don't care about the wired interrupts at all. It just requires
> > all the devices to be present behind a PCI bus or present themselves as 
> > platform
> > MSI device. The former is a more common scenario in x86 world where most
> > of the devices are behind PCI bus. As there is very limited MMIO device
> > support, it can also scale to very large number of harts.
> >
> > That's why, this patch series introduces a minimalistic yet very extensible
> > forward looking machine called as "RISC-V Mini Computer" or "minic". The
> > idea is to build PC or server like systems with this machine. The machine 
> > can
> > work with or without virtio framework. The current implementation only
> > supports RV64. I am not sure if building a RV32 machine would be of interest
> > for such machines. The only mmio device it requires is clint to emulate
> > the mtimecmp.
>
> I would ask what you see as the long term future usage for 'virt' vs
> 'minic' machine types ? Would you expect all existing users of 'virt'
> to ultimately switch to 'minic', or are there distinct non-overlapping
> use cases for 'virt' vs 'minic' such that both end up widely used ?
>

Nope. I don't expect existing 'virt' users to switch to 'minic' as
they aim to cater to different users.

Here are the major differences
1. virt machine supports MMIO devices & wired interrupts. Minic doesn't
2. virt machine doesn't support the MSI only option yet (can be added
though[1]). Minic does.
3. Number of cpu supported by virt machine are limited because of the
MMIO devices. Minic can scale to very
large numbers of cpu.
4. 'Minic' only supports PCI based MSI capable devices. Thus, MSI is a
mandatory requirement for 'minic' while
it is optional for 'virt'.

'Minic' aims towards the users who want to create virtual machines
that are MSI based and don't care about
a million options that virt machines provide.  Virt machine is more
complex so that it can be flexible in terms of
what it supports. Minic is a minimalistic machine which doesn't need
to be expanded a lot in the future given that
most of the devices can be behind PCI.

[1] https://github.com/atishp04/qemu/tree/virt_imsic_only

> Is 'minic' intended to be able to mimic real physical hardware at all,
> or is it still intended as a purely virtual machine, like a 'virt-ng' ?
>

Any future hardware that relies only on PCI-MSI based devices, they
can be created on top of minic.
At that point, minic will provide a useful abstract for all those
machines as well. minic doesn't need a virtio framework.
Thus, it can closely emulate such hardware as well.

> Essentially 'virt' was positioned as the standard machine to use if
> you want to run a virtual machine, without any particular match to
> physical hardware. It feels like 'minic' is creating a second machine
> type to fill the same purpose, so how do users decide which to use ?
>

I envision 'minic' to be a standard machine for a specific set of user
requirements (x86 style PCI based
machines). Virt machine will continue to be a standard machine for
more generic use cases with MMIO devices.

> > "Naming is hard". I am not too attached with the name "minic".
> > I just chose least bad one out of the few on my mind :). I am definitely
> > open to any other name as well.
> >
> > The other alternative to provide MSI only option to aia in the
> > existing virt machine to build MSI only machines. This is certainly doable
> > and here is the patch that supports that kind of setup.
> >
> > https://github.com/atishp04/qemu/tree/virt_imsic_only
> >
> > However, it even complicates the virt machine even further with additional
> > command line option, branches in the code. I believe virt machine will 
> > become
> > very complex if we continue this path. I am interested to learn what 
> > everyone
> > else think.
> >
> > It is needless to say that the current version of minic machine
> > is inspired from virt machine and tries to reuse as much as code possible.
> > The first patch in this series adds MSI support for serial-pci device so
> > console can work on such a machine. The 2nd patch moves some common 
> > functions
> > between minic and the virt machine to a helper file. The PATCH3 actually
> > implements the new minic machine.
> >
> > I have not added the fw-cfg/flash support. We probably should add those
> > but I just wanted to start small and get the feedback first.
> > This is a work in progress and have few more TODO items before becoming the
> > new world order :)
> >
> > 1. OpenSBI doesn't have PCI support. Thus, no console support for OpenSBI
> > for now.
> > 2. The ns16550 driver in OpenSBI also need to support MSI/MSI-X.
> > 3. Add MSI-X support for serial-pci device.
> >
> > This series can boot Linux distros with the minic machine with or without 
> > virtio
> > devices with out-of-tree Linux kernel patches[1]. Here is an example 
> > commandline
> >
> > Without virtio devices (nvme, serial-pci & e1000e):
> > =====================================================
> > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m 1G 
> > -smp 4 -nographic -nodefaults \
> > -display none -bios 
> > /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf \
> > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \
> > -chardev stdio,mux=on,signal=off,id=charconsole0 \
> > -mon chardev=charconsole0,mode=readline \
> > -device pci-serial,msi=true,chardev=charconsole0 \
> > -drive 
> > id=disk3,file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none,format=raw
> >  \
> > -device nvme,serial=deadbeef,drive=disk3 \
> > -netdev user,id=usernet,hostfwd=tcp::10000-:22 -device 
> > e1000e,netdev=usernet,bus=pcie.0 \
> > -append 'root=/dev/nvme0n1p2 rw loglevel=8 memblock=debug console=ttyS0 
> > earlycon' -d in_asm -D log.txt -s
> >
> > With virtio devices (virtio-scsi-pci, serial-pci & virtio-net-pci)
> > ==================================================================
> > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m 1G 
> > -smp 4 -nographic -nodefaults \
> > -display none -bios 
> > /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf \
> > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \
> > -chardev stdio,mux=on,signal=off,id=charconsole0 \
> > -mon chardev=charconsole0,mode=readline \
> > -device pci-serial,msi=true,chardev=charconsole0 \
> > -drive 
> > file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none
> >  \
> > -device virtio-scsi-pci,id=scsi0 -device 
> > scsi-hd,bus=scsi0.0,drive=drive-system-disk,id=system-disk,bootindex=1 \
> > -netdev user,id=n1,hostfwd=tcp::10000-:22 -device virtio-net-pci,netdev=n1 \
> > -append 'root=/dev/sda2 rw loglevel=8 memblock=debug console=ttyS0 earlycon'
> >
> > The objective of this series is to engage the community to solve this 
> > problem.
> > Please suggest if you have another alternatve solution.
> >
> > [1] https://github.com/atishp04/linux/tree/msi_only_console
> >
> > Atish Patra (3):
> > serial: Enable MSI capablity and option
> > hw/riscv: virt: Move common functions to a separate helper file
> > hw/riscv: Create a new qemu machine for RISC-V
> >
> > configs/devices/riscv64-softmmu/default.mak |   1 +
> > hw/char/serial-pci.c                        |  36 +-
> > hw/riscv/Kconfig                            |  11 +
> > hw/riscv/machine_helper.c                   | 417 +++++++++++++++++++
> > hw/riscv/meson.build                        |   2 +
> > hw/riscv/minic.c                            | 438 ++++++++++++++++++++
> > hw/riscv/virt.c                             | 403 ++----------------
> > include/hw/riscv/machine_helper.h           |  87 ++++
> > include/hw/riscv/minic.h                    |  65 +++
> > include/hw/riscv/virt.h                     |  13 -
> > 10 files changed, 1090 insertions(+), 383 deletions(-)
> > create mode 100644 hw/riscv/machine_helper.c
> > create mode 100644 hw/riscv/minic.c
> > create mode 100644 include/hw/riscv/machine_helper.h
> > create mode 100644 include/hw/riscv/minic.h
> >
> > --
> > 2.25.1
> >
> >
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>
>


-- 
Regards,
Atish