[RFC DOCUMENT 11/12] kubevirt-and-kvm: Add Backpropagation page

[Andrea Bolognani]

https://gitlab.com/abologna/kubevirt-and-kvm/-/blob/master/Backpropagation.md

# Backpropagation

Whenever a partial VM configuration is submitted to libvirt, any missing
information is automatically filled in to obtain a configuration that's
complete enough to guarantee long-term guest ABI stability.

PCI addresses are perhaps the most prominent example of this: most management
applications don't include this information at all in the XML they submit to
libvirt, and rely on libvirt building a reasonable PCI topology to support the
requested devices.

For example, using a made-up YAML syntax for brevity, the input could look like

```yaml
devices:
  disks:
  - image: /path/to/image.qcow2
```

and the output could be augmented by libvirt to look like

```yaml
devices:
  controllers:
  - model: pcie-root-port
    address:
      type: pci
      domain: 0x0000
      bus: 0x00
      slot: 0x01
      function: 0x0
  disks:
  - image: /path/to/image.qcow2
    model: virtio-blk
    address:
      type: pci
      domain: 0x0000
      bus: 0x01
      slot: 0x00
      function: 0x0
```

This is where backpropagation comes in: the only version of the VM
configuration that is complete enough to guarantee a stable guest ABI is the
one that includes all information added by libvirt, so if the management
application wants to be able to make further changes to the VM it needs to
reflect the additional information back into its understanding of the VM
configuration somehow.

For applications like virsh and virt-manager, this is easy: they don't have
their own configuration format or even store the VM configuration, and
simply fetch it from libvirt and operate on it directly every single time.

oVirt, to the best of my knowledge, generates an initial VM configuration based
on the settings provided by the user, submits it to libvirt and then parses
back the augmented version, figuring out what information was added and
updating its database to match: if the VM configuration needs to be generated
again later, it will include all information present in the database, including
those that originated from libvirt rather than the user.

KubeVirt does not currently perform any backpropagation. There are two ways a
user can influence PCI address allocation:

* explicitly add a `pciAddress` attribute for the device, which will cause
  KubeVirt to pass the corresponding address to libvirt, which in turn will
  attempt to comply with the user's request;
* add the `kubevirt.io/placePCIDevicesOnRootComplex` annotation to the VM
  configuration, which will cause KubeVirt to provide libvirt with a
  fully-specified PCI topology where all devices live on the PCIe Root Bus.

In all cases but the one where KubeVirt defines the full PCI topology itself,
it's implicitly relying on libvirt always building the PCI topology in the
exact same way every single time in order to have a stable guest ABI. While
this works in practice, it's not something that libvirt actually guarantees:
once a VM has been defined, libvirt will never change its PCI topology, but
submitting the same partial VM configuration to different libvirt versions can
result in different PCI topologies.