qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC PATCH] docs/specs: QMP configuration design specification


From: Mirela Grujic
Subject: Re: [RFC PATCH] docs/specs: QMP configuration design specification
Date: Mon, 20 Sep 2021 13:39:34 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0

Hi,


We haven't got any feedback on this, anyone, please?


Thanks,

Mirela


On 6/28/21 11:55 AM, Mirela Grujic wrote:
Hi,

While implementing a prototype we run into a couple of challenges. We're not sure whether our current implementation would be acceptable and whether there is a better solution, so we would highly appreciate your feedback.

The device_add command, which was originally used for hot-plugging devices, initializes and realizes the device in a single step. In contrast, existing models in C code heavily rely on disjoint initialize and realize steps. Due to this discrepancy, it's challenging to replace hard-coded configurations with the QMP equivalent.

For example, we need to create a CPU cluster with two CPUs. In C code, this is done as follows:
1) Initialize the CPU cluster
2) Create CPUs as children of the cluster
3) Realize cluster

Note that the cluster cannot be realized empty (without the CPU children). We need to establish parent-child relationships between the cluster and its CPU children before realizing the cluster. Consequently, we cannot simply add the cluster and CPUs using the existing device_add command. A similar issue exists if there are bidirectional links between the devices.

An approach to solve the problem above is to allow the device_add command to skip realizing the device and to use the qom-set command to realize the device later. To support this, we would add a 'realized' option to the device_add command with the default true value. Setting 'realized' to false would be only allowed during the machine initialization phase, thus the existing behavior of the device_add command for hot-plugging devices wouldn't change.

To initialize and realize the device in two steps we would run:

    device_add driver=cpu-cluster id=cluster0 cluster-id=0 realized=false
    qom-set path=cluster0 property=realized value=true

Additionally, to set the parent we would add the 'parent' option to the
device_add command. The cluster and its CPUs would be added as follows:

    device_add driver=cpu-cluster cluster-id=0 id=cluster0 realized=false
    device_add driver=cortex-a72-arm-cpu id=apu-cpu0 parent=cluster0 ...
    device_add driver=cortex-a72-arm-cpu id=apu-cpu1 parent=cluster0 ...
    qom-set path=cluster0 property=realized value=true

The implementation is quite simple. If anyone would like to take a look we can submit an RFC or point to a repository.

An alternative to the proposal above would be to create device wrappers that would allow creating multiple devices that mutually depend, using a single device_add command.

While the first approach is generic, it requires users to have some knowledge of QEMU internals (isn't this expected from someone who is configuring the machine at this level?). The second approach appears to be more user-friendly, but it would require additional device models just to enable the use of existing ones with the QMP configuration.

Could you please let us know what do you think? Any other proposals?

Thanks,
Mirela



On 6/3/21 6:02 PM, Paolo Bonzini wrote:
On 01/06/21 12:07, Mirela Grujic wrote:
diff --git a/docs/specs/qmp-configuration.txt b/docs/specs/qmp-configuration.txt
new file mode 100644
index 0000000000..69ff49cae6
--- /dev/null
+++ b/docs/specs/qmp-configuration.txt

docs/specs is not the right place, as it is for guest devices. But this is completely irrelevant to the proposal, which doesn't need to live in docs/ at all.  So forget about it. :)

In general this is a good starting point.  We can either deprecate or stabilize x-machine-init once there is enough code written to have a working qemu-qmp-* binary.  Then we will know whether an accel-created->machine-init transition is needed or can be left implicit.

For now, what you propose is a very nice compromise that allows parallel work on machine/accel configuration on one hand, and device configuration on the other hand.  One other reason why that works well is that we have more or less opposite interests (I care mostly about machine/accel for example) so that's a natural way to split the work.

Thanks!

Paolo

@@ -0,0 +1,112 @@
+
+Overview
+--------
+
+The QMP configuration in the context of this design stands for customizing an +existing machine using the QEMU Monitor Protocol (QMP). The requirement for such +a configuration comes from the use cases where a complete system-on-chip can be +customized: from the number of cores, available peripherals, memories, IRQ +mapping, etc. Our goal is to enable the emulation of customized platforms +without requiring modifications in QEMU, and thus the QEMU recompilation.
+
+We will perform the QMP configuration from a QMP client that will communicate +with QEMU via a socket. As an example of a QMP client, we will include a script, +namely the QMP configurator, that will read QMP commands from a configuration +file and send them to QEMU, one by one. The configuration file will be a text
+file that includes only a list of QMP commands to be executed.
+
+We will start QEMU with the -preconfig command-line option, thus QEMU will wait +for the QMP configuration at an early initialization phase, before the machine +initialization. The following configuration flow will rely on the machine +initialization phases. In each initialization phase, a set of QMP commands will
+be available for configuring the machine and advancing it to the next
+initialization phase. Upon reaching the final initialization phase, the machine
+shall be ready to run. We specify detailed configuration flow in
+"QMP configuration flow in QEMU."
+
+
+QMP configurator
+----------------
+
+We decided to include the QMP configurator, a QMP client script that will +communicate with QEMU, to automate the configuration. The QMP configurator will +read the QMP commands from the configuration file, send each QMP command to +QEMU, and quit at the end or exit earlier in the case of an error. We have not +envisioned the QMP configurator to be interactive. For debugging purposes, one
+could use the QMP shell instead (scripts/qmp/qmp-shell).
+
+
+QMP configuration file
+----------------------
+
+The QMP configuration file will be a text file that includes only a list of +QMP commands which configure the machine. QMP commands in the configuration file +shall be in the same format and order as if they were issued using the QMP +shell. The QMP configurator will convert each command into JSON format before
+sending it to QEMU, similarly as the QMP shell does.
+
+There are several ways to create a configuration file. One approach is to +manually create a list of QMP commands which specify how to configure the +machine. Another convenient method is to generate QMP commands from existing +descriptions, such as the device tree or a proprietary format. Either way, the
+creation of the configuration file is out of the scope of this work.
+
+However, along with the QMP configurator, we will include an example of the +machine and its configuration file to demonstrate the QMP configuration
+approach.
+
+
+QMP configuration flow in QEMU
+------------------------------
+
+We will build the QMP configuration flow on top of the machine initialization
+phases, that are:
+1) no-machine: machine does not exist yet (current_machine == NULL)
+2) machine-created: machine exists, but its accelerator does not
+   (current_machine->accelerator == NULL)
+3) accel-created: machine's accelerator is configured
+   (current_machine->accelerator != NULL), but machine class's init() has not +   been called (no properties validated, machine_init_done notifiers not
+   registered, no sysbus, etc.)
+4) initialized: machine class's init() has been called, thus machine properties +   are validated, machine_init_done notifiers registered, sysbus realized, etc.
+   Devices added at this phase are considered to be cold-plugged.
+5) ready: machine_init_done notifiers are called, then QEMU is ready to start
+   CPUs. Devices added at this phase are considered to be hot-plugged.
+
+We can stop QEMU today using the -preconfig command-line option at phase 3 +('accel-created'). This option was introduced to enable the QMP configuration of +parameters that affect the machine initialization. We cannot add devices at +this point because the machine class's init() has not been called, thus sysbus +does not exist yet (a device cannot be added because there is no bus to attach
+it to).
+
+QEMU can be also stopped using the -S command-line option at the machine 'ready' +phase. However, it is too late to add devices at this phase because the machine +is already configured, and any devices added at this point are considered to be
+hot-plugged.
+
+Since the existing -preconfig command-line option stops QEMU too early, and the +-S option stops too late, we need a way to stop QEMU in between (after the
+machine is initialized and before it becomes ready).
+
+We will reuse the existing -preconfig command-line option to stop QEMU at the +'accel-created' phase. Then, we will add a QMP command, namely 'x-machine-init',
+to advance and stop the machine in the next initialization phase
+('initialized' phase). We will configure the machine during this phase using the +existing 'device_add' QMP command. Note that the use of 'device_add' QMP command +is currently not allowed before the machine is ready. We will enable the use of
+'device_add' during the 'initialized' phase.
+
+Once we complete the configuration, we will advance the machine to the 'ready' +phase using the existing 'x-exit-preconfig' command. Upon the execution of +'x-exit-preconfig' command, the machine will immediately start running the guest
+unless the -S option is provided as the command-line argument.
+
+We will also implement a QMP command to query the current machine initialization
+phase, namely the 'query-machine-phase' command.
+
+--------------------------------------------------------------------------------
+
+This work is supported by Xilinx, SiFive, and Greensocs.
+







reply via email to

[Prev in Thread] Current Thread [Next in Thread]