Re: About creating machines on the command line

[Luc Michel]

On 2/3/21 6:09 PM, Alexander Graf wrote:
> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
> > On 1/11/21 3:50 PM, Luc Michel wrote:
> > > Hi,
> > >
> > > We would like to work on improving QEMU to be able to create custom
> > > machines from the command line. The goal here is to get feedback from
> > > the community and shape the future developments.
> >
> > Cc'ing John who is working on command line, and some developers from
> > the "inter-VM device emulation interface" call.
> > https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
> >
> > > The use case mainly comes from people working with tools to customize
> > > their designs, such as SiFive Core Designer
> > > (https://scs.sifive.com/core-designer). This kind of tools may allow
> > > creation or customization of a whole SoC, from the number of cores, to
> > > the memory and IRQ mapping of peripherals etc.
> > >
> > > The ultimate goal would be to be able to create any kind of machine on
> > > the command line. However we are aware that this is a substantial amount
> > > of changes in QEMU.
>
> Is the command line really the right abstraction level here? Wouldn't it 
> make more sense to have a QOM / <scripting language> bridge that allows 
> you to create and connect QOM objects using for example Python?
Yes, after some discussions with the community, we are now working on 
improving QMP to achieve this. We first started with the idea of the 
command line because it seems to be the place where we had "almost" 
everything we needed already. In either cases we are planning to use a 
front-end script to go from e.g. a DTB to whatever QEMU interface we 
will end up using.

> You could then have machine descriptions in a script, which could be 
> generated by the SoC customization tools.
Yes, most likely a DTB in our case.

> In combination with plugin support for platform devices, that should 
> allow pretty much any customization you would need to happen, no?
I'm not sure what you mean by "plugin support for platform devices". If 
you refer to the current QEMU plugin API, it's pretty much limited to 
TCG. If you suggest this API could be enhanced to allow for custom 
devices to be added by plugins, for now we are planning to use only 
existing device models. It's more a matter of having flexibility on 
connections between the devices than on the devices themselves (but yes 
that would be a nice next step).

Thanks,

--
Luc

> Alex
>
> > > In its current state, QEMU allows for very limited customization of
> > > existing machines on the command line. We identified the following
> > > limitations (feel free to add to the list):
> > >
> > >    - Most devices are not user creatable. Moreover, sysbus devices must
> > > be explicitly allowed by a machine to be creatable through `-device`,
> > >
> > >    - Memory regions cannot be created on the command line,
> > >
> > >    - Device MMIO regions cannot be mapped on a bus from the command 
> > > line,
> > >
> > >    - GPIOs and clocks cannot be wired from the command line,
> > >
> > >    - CPUs are not sysbus devices (and not user-creatable). They need
> > > special care when creating them regarding system reset. Not being on a
> > > bus means that they must be reset manually on system reset. This is done
> > > in machines by registering a QEMU reset handler.
> > >
> > >    - Machine specific boot code is usually hard-coded into the machine
> > > itself.  Some architectures (e.g. ARM) do factorize bootloader related
> > > code, but there is no standard way of doing that in QEMU.
> > >
> > > We don't want to address all those limitations at once. We plan to start
> > > with the following scenario:
> > >
> > >    - Start with a base machine that would handle CPU creation and
> > > bootloader stuff. Note that the "none" machine is probably not
> > > sufficient in its current shape. It does allow only one CPU and
> > > obviously does not handle the boot process.
> > >
> > >    - Allow for this machine every sysbus devices we want to be user
> > > command-line creatable (and mark them user_creatable if needed)
> > >
> > >    - Add command line options to create memory regions (probably ram 
> > > ones
> > > at first)
> > >
> > >    - Add command line options to map a memory region (including sysbus
> > > device MMIO regions) onto another (memory_region_add_subregion)
> > >
> > >    - Add command line options to connect GPIOs and clocks.
> > >
> > > This would hopefully allow for simple machines creation. We would then
> > > be able to use either the command line or the `-readconfig` option to
> > > create the machine.
> > >
> > > Note that we are not planning to use QMP/HMP for now. From our
> > > understanding, a `device_add` request is always considered as hot-plug,
> > > which is not what we want here.
> > >
> > > Please tell us what do you think about this plan. Any feedback is
> > > appreciated.  Then we can discuss the details of how to do this 
> > > properly.
> > > Thanks!
> > >
> > > --
> > > Luc
>
>
>
> Amazon Development Center Germany GmbH
> Krausenstr. 38
> 10117 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
> Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
> Sitz: Berlin
> Ust-ID: DE 289 237 879