Re: [Qemu-devel] [PATCH v3 00/34] pc: ACPI memory hotplug

[Peter Crosthwaite]

On Tue, May 27, 2014 at 11:00 PM, Igor Mammedov <address@hidden> wrote:
> NOTE to commiter:
>  * update precompiled ACPI hex files for iasl-less hosts
>    and ACPI tables test to match new ACPI tables.
>
> What's new since v9:
>  * drop usage of error_is_set()
>  * exit QEMU if it's started with memory hotplug but
>    compat machine doesn't support it
>  * rename cpu_hotplug_defs.h to pc-hotplug.h
>  * use subsection when migrating acpi memory hotplug state
>  * add property to PC_MACHINE for getting hotplug memory region size
>
> What's new since v8:
>  * rebased on top of Marcel's QOMyfing machine work
>     depends on patch from qom-next:
>      "machine: Conversion of QEMUMachineInitArgs to MachineState"
>  * fixed QEMU abort if it's running in daemonized mode
>  * fixed leak in memdev backend
>  * introduced custom PCMachine
>  * DIMM devices are now bus-less and use bus-less hotplug method
>  * DIMMDevice: renamed property/field 'start' to 'addr'
>  * ACPI tables:
>    * avoid punching hotples in PCI CRS by placing
>      MEMORY_HOPTLUG_DEVICE on PCI0 bus
>  * incorporated most of comments/fixes from reviewers
>
> What's new since v7:
> * Per Andreas' suggestion dropped DIMMBus concept.
> * Added hotplug binding for bus-less devices
> * DIMM device is split to backend and frontend. Therefore following
>   command/options were added for supporting it:
>
>   For memory-ram backend:
>       CLI: -object-add memory-ram,
>           with options: 'id' and 'size'
>   For dimm frontend:
>       option "size" became readonly, pulling it's size from attached backend
>       added option "memdev" for specifying backend by 'id'
>
> * dropped support for 32 bit guests
> * failed hotplug action doesn't consume 1 slot anymore
> * vaious fixes adressing reviewer's comments most of them in ACPI part
> ---
>
> This series allows to hotplug 'arbitrary' DIMM devices specifying size,
> NUMA node mapping (guest side), slot and address where to map it, at runtime.
>
> Due to ACPI limitation there is need to specify a number of possible
> DIMM devices. For this task -m option was extended to support
> following format:
>
>   -m [mem=]RamSize[,slots=N,maxmem=M]
>
> To allow memory hotplug user must specify a pair of additional parameters:
>     'slots' - number of possible increments
>     'maxmem' - max possible total memory size QEMU is allowed to use,
>                including RamSize.
>
> minimal monitor command syntax to hotplug DIMM device:
>
>   object_add memory-ram,id=memX,size=1G
>   device_add dimm,id=dimmX,memdev=memX
>
> DIMM device provides following properties that could be used with
> device_add / -device to alter default behavior:
>
>   id    - unique string identifying device [mandatory]
>   slot  - number in range [0-slots) [optional], if not specified
>           the first free slot is used
>   node  - NUMA node id [optional] (default: 0)
>   size  - amount of memory to add, readonly derived from backing memdev
>   addr - guest's physical address where to plug DIMM [optional],
>           if not specified the first gap in hotplug memory region
>           that fits DIMM is used
>
>  -device option could be used for adding potentially hotunplugable DIMMs
> and also for specifying hotplugged DIMMs in migration case.
>

So this has some themes in common with my MemoryRegion QOMification
series. This series does a lot to facilitate linking and exposing
memory regions between QOM objects which was one of my goals. Series
here:

https://lists.nongnu.org/archive/html/qemu-devel/2014-05/msg05374.html
http://lists.gnu.org/archive/html/qemu-devel/2014-05/msg03265.html

That said, your on V10 and your patches are pretty close IMO. So i'd
rather see it go through and tweak the DIMM/Membackend stuff to use
QOMified MRs as follow up. I've made a few comments on the Memoryish
bits of the series but Ive glossed over the PC/ACPI specific stuffs.
General approach WRT QOM and the linkages and user interface looks
good to me. The busless machine level hotplug idea looks good.

So for all uncommented patches,

Acked-by: Peter Crosthwaite <address@hidden>

> Tested guests:
>  - RHEL 6x64
>  - Windows 2012DCx64
>  - Windows 2008DCx64
>
> Known limitations/bugs/TODOs:
>  - hot-remove is not supported, yet
>  - max number of supported DIMM devices 256 (due to ACPI object name
>    limit), could be increased creating several containers and putting
>    DIMMs there. (exercise for future)
>  - e820 table doesn't include DIMM devices added with -device /
>    (or after reboot devices added with device_add)
>  - Windows 2008 remembers DIMM configuration, so if DIMM with other
>    addr/size is added into the same slot, it refuses to use it insisting
>    on old mapping.
>
> QEMU git tree for testing is available at:
>   https://github.com/imammedo/qemu/commits/memory-hotplug-v10
>
> Example QEMU cmd line:
>   qemu-system-x86_64 -enable-kvm -monitor unix:/tmp/mon,server,nowait \
>      -m 4096,slots=4,maxmem=8G guest.img
>
> PS:
>   Windows guest requires SRAT table for hotplug to work so add an extra 
> option:
>    -numa node
>   to QEMU command line.
>
>
> Igor Mammedov (32):
>   pc: ACPI BIOS: use enum for defining memory affinity flags
>   object_add: allow completion handler to get canonical path
>   vl.c: daemonize before guest memory allocation
>   add memdev backend infrastructure
>   vl.c: extend -m option to support options for memory hotplug
>   pc: create custom generic PC machine type
>   qdev: hotplug for buss-less devices
>   qdev: expose DeviceState.hotplugged field as a property
>   memory: add memory_region_is_mapped() API
>   dimm: do not allow to set already used memdev
>   pc: initialize memory hotplug address space
>   pc: exit QEMU if number of slots more than supported 256
>   pc: add 'etc/reserved-memory-end' fw_cfg interface for SeaBIOS
>   pc: exit QEMU if compat machine doesn't support memory hotlpug
>   pc: add memory hotplug handler to PC_MACHINE
>   dimm: add busy address check and address auto-allocation
>   dimm: add busy slot check and slot auto-allocation
>   acpi: rename cpu_hotplug_defs.h to pc-hotplug.h
>   acpi: memory hotplug ACPI hardware implementation
>   trace: add acpi memory hotplug IO region events
>   trace: pc: add DIMM slot & address allocation
>   acpi:piix4: allow plug/unlug callbacks handle not only PCI devices
>   acpi:piix4: add memory hotplug handling
>   pc: ich9 lpc: make it work with global/compat properties
>   acpi:ich9: add memory hotplug handling
>   pc: migrate piix4 & ich9 MemHotplugState
>   pc: add acpi-device link to PCMachineState
>   pc: propagate memory hotplug event to ACPI device
>   pc: ACPI BIOS: implement memory hotplug interface
>   pc: add "hotplug-memory-region-size" property to PC_MACHINE
>   pc: ACPI BIOS: reserve SRAT entry for hotplug mem hole
>   pc: ACPI BIOS: make GPE.3 handle memory hotplug event on PIIX and Q35
>     machines
>
> Marcel Apfelbaum (1):
>   machine: Conversion of QEMUMachineInitArgs to MachineState
>
> Vasilis Liaskovitis (1):
>   dimm: implement dimm device abstraction
>
>  backends/Makefile.objs                   |    2 +
>  backends/hostmem-ram.c                   |   54 +++++++
>  backends/hostmem.c                       |  113 ++++++++++++++
>  default-configs/i386-softmmu.mak         |    1 +
>  default-configs/x86_64-softmmu.mak       |    1 +
>  docs/specs/acpi_mem_hotplug.txt          |   44 ++++++
>  hw/Makefile.objs                         |    1 +
>  hw/acpi/Makefile.objs                    |    1 +
>  hw/acpi/ich9.c                           |   62 ++++++++
>  hw/acpi/memory_hotplug.c                 |  187 +++++++++++++++++++++++
>  hw/acpi/piix4.c                          |   74 ++++++++--
>  hw/alpha/dp264.c                         |   12 +-
>  hw/arm/collie.c                          |   10 +-
>  hw/arm/cubieboard.c                      |   11 +-
>  hw/arm/digic_boards.c                    |    2 +-
>  hw/arm/exynos4_boards.c                  |   22 ++--
>  hw/arm/gumstix.c                         |    6 +-
>  hw/arm/highbank.c                        |   24 ++--
>  hw/arm/integratorcp.c                    |   12 +-
>  hw/arm/kzm.c                             |   12 +-
>  hw/arm/mainstone.c                       |   14 +-
>  hw/arm/musicpal.c                        |   10 +-
>  hw/arm/nseries.c                         |   22 ++--
>  hw/arm/omap_sx1.c                        |   21 ++--
>  hw/arm/palm.c                            |   10 +-
>  hw/arm/realview.c                        |   44 +++---
>  hw/arm/spitz.c                           |   26 ++--
>  hw/arm/stellaris.c                       |   12 +-
>  hw/arm/tosa.c                            |   10 +-
>  hw/arm/versatilepb.c                     |   26 ++--
>  hw/arm/vexpress.c                        |   21 ++--
>  hw/arm/virt.c                            |   16 +-
>  hw/arm/xilinx_zynq.c                     |   12 +-
>  hw/arm/z2.c                              |   10 +-
>  hw/core/null-machine.c                   |    2 +-
>  hw/core/qdev.c                           |   30 ++++
>  hw/cris/axis_dev88.c                     |   10 +-
>  hw/i386/Makefile.objs                    |    3 +-
>  hw/i386/acpi-build.c                     |   75 +++++++++-
>  hw/i386/acpi-dsdt.dsl                    |    7 +-
>  hw/i386/pc.c                             |  227 ++++++++++++++++++++++++++++-
>  hw/i386/pc_piix.c                        |  160 +++++++++++---------
>  hw/i386/pc_q35.c                         |   92 +++++++-----
>  hw/i386/q35-acpi-dsdt.dsl                |    7 +-
>  hw/i386/ssdt-mem.dsl                     |   77 ++++++++++
>  hw/i386/ssdt-misc.dsl                    |  164 ++++++++++++++++++++
>  hw/isa/lpc_ich9.c                        |   33 ++++-
>  hw/lm32/lm32_boards.c                    |   16 +-
>  hw/lm32/milkymist.c                      |   10 +-
>  hw/m68k/an5206.c                         |    8 +-
>  hw/m68k/dummy_m68k.c                     |    8 +-
>  hw/m68k/mcf5208.c                        |    8 +-
>  hw/mem/Makefile.objs                     |    1 +
>  hw/mem/dimm.c                            |  240 
> ++++++++++++++++++++++++++++++
>  hw/microblaze/petalogix_ml605_mmu.c      |    6 +-
>  hw/microblaze/petalogix_s3adsp1800_mmu.c |    8 +-
>  hw/mips/mips_fulong2e.c                  |   12 +-
>  hw/mips/mips_jazz.c                      |   12 +-
>  hw/mips/mips_malta.c                     |   14 +-
>  hw/mips/mips_mipssim.c                   |   12 +-
>  hw/mips/mips_r4k.c                       |   12 +-
>  hw/moxie/moxiesim.c                      |   12 +-
>  hw/openrisc/openrisc_sim.c               |    8 +-
>  hw/ppc/e500.c                            |   44 +++---
>  hw/ppc/e500.h                            |    2 +-
>  hw/ppc/e500plat.c                        |    4 +-
>  hw/ppc/mac_newworld.c                    |   14 +-
>  hw/ppc/mac_oldworld.c                    |   14 +-
>  hw/ppc/mpc8544ds.c                       |    4 +-
>  hw/ppc/ppc405_boards.c                   |   18 +-
>  hw/ppc/ppc440_bamboo.c                   |   12 +-
>  hw/ppc/prep.c                            |   14 +-
>  hw/ppc/spapr.c                           |   14 +-
>  hw/ppc/virtex_ml507.c                    |   16 +-
>  hw/s390x/s390-virtio-ccw.c               |   10 +-
>  hw/s390x/s390-virtio.c                   |   10 +-
>  hw/sh4/r2d.c                             |   10 +-
>  hw/sh4/shix.c                            |    4 +-
>  hw/sparc/leon3.c                         |    8 +-
>  hw/sparc/sun4m.c                         |   68 +++++-----
>  hw/sparc64/sun4u.c                       |   36 +++---
>  hw/unicore32/puv3.c                      |   10 +-
>  hw/xenpv/xen_machine_pv.c                |    8 +-
>  hw/xtensa/xtensa_lx60.c                  |   26 ++--
>  hw/xtensa/xtensa_sim.c                   |    8 +-
>  include/exec/memory.h                    |    8 +
>  include/hw/acpi/acpi.h                   |    6 +
>  include/hw/acpi/cpu_hotplug.h            |    2 +-
>  include/hw/acpi/cpu_hotplug_defs.h       |   33 ----
>  include/hw/acpi/ich9.h                   |    4 +
>  include/hw/acpi/memory_hotplug.h         |   37 +++++
>  include/hw/acpi/pc-hotplug.h             |   57 +++++++
>  include/hw/boards.h                      |   36 +++--
>  include/hw/i386/pc.h                     |   64 ++++++++-
>  include/hw/mem/dimm.h                    |   80 ++++++++++
>  include/sysemu/hostmem.h                 |   60 ++++++++
>  memory.c                                 |   15 ++-
>  qemu-options.hx                          |    9 +-
>  qmp.c                                    |   11 +-
>  trace-events                             |   17 ++
>  vl.c                                     |   80 ++++++++--
>  101 files changed, 2400 insertions(+), 652 deletions(-)
>  create mode 100644 backends/hostmem-ram.c
>  create mode 100644 backends/hostmem.c
>  create mode 100644 docs/specs/acpi_mem_hotplug.txt
>  create mode 100644 hw/acpi/memory_hotplug.c
>  create mode 100644 hw/i386/ssdt-mem.dsl
>  create mode 100644 hw/mem/Makefile.objs
>  create mode 100644 hw/mem/dimm.c
>  delete mode 100644 include/hw/acpi/cpu_hotplug_defs.h
>  create mode 100644 include/hw/acpi/memory_hotplug.h
>  create mode 100644 include/hw/acpi/pc-hotplug.h
>  create mode 100644 include/hw/mem/dimm.h
>  create mode 100644 include/sysemu/hostmem.h
>
>