Re: [Qemu-devel] [PATCH v4 01/20] ppc/pnv: add skeleton PowerNV platform

[Cédric Le Goater]

On 10/07/2016 06:14 AM, David Gibson wrote:
> On Mon, Oct 03, 2016 at 09:24:37AM +0200, Cédric Le Goater wrote:
>> From: Benjamin Herrenschmidt <address@hidden>
>>
>> The goal is to emulate a PowerNV system at the level of the skiboot
>> firmware, which loads the OS and provides some runtime services. Power
>> Systems have a lower firmware (HostBoot) that does low level system
>> initialization, like DRAM training. This is beyond the scope of what
>> qemu will address in a PowerNV guest.
>>
>> No devices yet, not even an interrupt controller. Just to get started,
>> some RAM to load the skiboot firmware, the kernel and initrd. The
>> device tree is fully created in the machine reset op.
>>
>> Signed-off-by: Benjamin Herrenschmidt <address@hidden>
>> [clg: - updated for qemu-2.7
>>       - replaced fprintf by error_report
>>       - used a common definition of _FDT macro
>>       - removed VMStateDescription as migration is not yet supported
>>       - added IBM Copyright statements
>>       - reworked kernel_filename handling
>>       - merged PnvSystem and sPowerNVMachineState
>>       - removed PHANDLE_XICP
>>       - added ppc_create_page_sizes_prop helper
>>       - removed nmi support
>>       - removed kvm support
>>       - updated powernv machine to version 2.8
>>       - removed chips and cpus, They will be provided in another patches
>>       - added a machine reset routine to initialize the device tree (also)
>>       - french has a squelette and english a skeleton.
>>       - improved commit log.
>>       - reworked prototypes parameters
>>       - added a check on the ram size (thanks to Michael Ellerman)
>>       - fixed chip-id cell
>>       - changed MAX_CPUS to 2048
>>       - simplified memory node creation to one node only
>>       - removed machine version
>>       - rewrote the device tree creation with the fdt "rw" routines
>>       - s/sPowerNVMachineState/PnvMachineState/
>>       - etc.]
>> Signed-off-by: Cédric Le Goater <address@hidden>
>> ---
>>  Changes since v3:
>>
>>  - fixed printf format for hwaddr
>>  - used fdt_pack() before writing the tree in memory 
>>  - removed the requirement on having a kernel loaded as running with
>>    just a firmware is fine. We will need to discuss the inclusion of
>>    the file skiboot.lid under qemu.
> 
> Yes, this isn't terribly useful without it.  The normal procedure for
> new roms is this:
> 
>    1. Get the upstream git tree for the ROM mirrored to qemu.org
>    2. Add a git submodule under roms/ referencing the git mirror on
>       qemu.org
>    3. Add a pre-built ROM binary to pc-bios/
>    4. Add a brief description of the ROM, including upstream git URL
>       to pc-bios/README
> 
> Steps 2, 3 & 4 can (and usually should) be a single commit.
> 
> This code is looking close enough, that having a usable rom image is
> probably the last thing stopping merge, at least of these initial
> patches.
> 
> Probably best to get underway with the rom inclusion ASAP.
> 
>>  Changes since v2:
>>
>>  - some more copyright header cleanups
>>  - remove fdt_addr field from PnvMachineState
>>
>>  Changes since v1:
>>
>>  - changed MAX_CPUS to 2048
>>  - simplified memory node creation to one node only
>>  - removed machine version 
>>  - rewrote the device tree creation with the fdt "rw" routines
>>  - s/sPowerNVMachineState/PnvMachineState/
>>  - block_default_type is back to IF_IDE because of the AHCI device
>>
>>  default-configs/ppc64-softmmu.mak |   1 +
>>  hw/ppc/Makefile.objs              |   2 +
>>  hw/ppc/pnv.c                      | 223 
>> ++++++++++++++++++++++++++++++++++++++
>>  include/hw/ppc/pnv.h              |  38 +++++++
>>  4 files changed, 264 insertions(+)
>>  create mode 100644 hw/ppc/pnv.c
>>  create mode 100644 include/hw/ppc/pnv.h
>>
>> diff --git a/default-configs/ppc64-softmmu.mak 
>> b/default-configs/ppc64-softmmu.mak
>> index db5a4d6f5eea..67a9bcaa67fa 100644
>> --- a/default-configs/ppc64-softmmu.mak
>> +++ b/default-configs/ppc64-softmmu.mak
>> @@ -39,6 +39,7 @@ CONFIG_I8259=y
>>  CONFIG_XILINX=y
>>  CONFIG_XILINX_ETHLITE=y
>>  CONFIG_PSERIES=y
>> +CONFIG_POWERNV=y
>>  CONFIG_PREP=y
>>  CONFIG_MAC=y
>>  CONFIG_E500=y
>> diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
>> index 99a0d4e581bf..8105db7d5600 100644
>> --- a/hw/ppc/Makefile.objs
>> +++ b/hw/ppc/Makefile.objs
>> @@ -5,6 +5,8 @@ obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
>>  obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
>>  obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
>>  obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
>> +# IBM PowerNV
>> +obj-$(CONFIG_POWERNV) += pnv.o
>>  ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
>>  obj-y += spapr_pci_vfio.o
>>  endif
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> new file mode 100644
>> index 000000000000..02fc4826baa4
>> --- /dev/null
>> +++ b/hw/ppc/pnv.c
>> @@ -0,0 +1,223 @@
>> +/*
>> + * QEMU PowerPC PowerNV machine model
>> + *
>> + * Copyright (c) 2016, IBM Corporation.
>> + *
>> + * This library is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2 of the License, or (at your option) any later version.
>> + *
>> + * This library is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with this library; if not, see 
>> <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/error.h"
>> +#include "sysemu/sysemu.h"
>> +#include "sysemu/numa.h"
>> +#include "hw/hw.h"
>> +#include "target-ppc/cpu.h"
>> +#include "qemu/log.h"
>> +#include "hw/ppc/fdt.h"
>> +#include "hw/ppc/ppc.h"
>> +#include "hw/ppc/pnv.h"
>> +#include "hw/loader.h"
>> +#include "exec/address-spaces.h"
>> +#include "qemu/cutils.h"
>> +
>> +#include <libfdt.h>
>> +
>> +#define FDT_MAX_SIZE            0x00100000
>> +
>> +#define FW_FILE_NAME            "skiboot.lid"
>> +#define FW_LOAD_ADDR            0x0
>> +#define FW_MAX_SIZE             0x00400000
>> +
>> +#define KERNEL_LOAD_ADDR        0x20000000
>> +#define INITRD_LOAD_ADDR        0x40000000
>> +
>> +/*
>> + * On Power Systems E880, the max cpus (threads) should be :
>> + *     4 * 4 sockets * 12 cores * 8 threads = 1536
>> + * Let's make it 2^11
>> + */
>> +#define MAX_CPUS                2048
>> +
>> +/*
>> + * Memory nodes are created by hostboot, one for each range of memory
>> + * that has a different "affinity". In practice, it means one range
>> + * per chip.
>> + */
>> +static void powernv_populate_memory_node(void *fdt, int chip_id, hwaddr 
>> start,
>> +                                         hwaddr size)
>> +{
>> +    char *mem_name;
>> +    uint64_t mem_reg_property[2];
>> +    int off;
>> +
>> +    mem_reg_property[0] = cpu_to_be64(start);
>> +    mem_reg_property[1] = cpu_to_be64(size);
>> +
>> +    mem_name = g_strdup_printf("address@hidden"HWADDR_PRIx, start);
>> +    off = fdt_add_subnode(fdt, 0, mem_name);
>> +    g_free(mem_name);
>> +
>> +    _FDT((fdt_setprop_string(fdt, off, "device_type", "memory")));
>> +    _FDT((fdt_setprop(fdt, off, "reg", mem_reg_property,
>> +                       sizeof(mem_reg_property))));
>> +    _FDT((fdt_setprop_cell(fdt, off, "ibm,chip-id", chip_id)));
>> +}
>> +
>> +static void *powernv_create_fdt(PnvMachineState *pnv,
>> +                                const char *kernel_cmdline)
>> +{
>> +    void *fdt;
>> +    char *buf;
>> +    const char plat_compat[] = "qemu,powernv\0ibm,powernv";
>> +    int off;
>> +
>> +    fdt = g_malloc0(FDT_MAX_SIZE);
>> +    _FDT((fdt_create_empty_tree(fdt, FDT_MAX_SIZE)));
>> +
>> +    /* Root node */
>> +    _FDT((fdt_setprop_cell(fdt, 0, "#address-cells", 0x2)));
>> +    _FDT((fdt_setprop_cell(fdt, 0, "#size-cells", 0x2)));
>> +    _FDT((fdt_setprop_string(fdt, 0, "model",
>> +                             "IBM PowerNV (emulated by qemu)")));
>> +    _FDT((fdt_setprop(fdt, 0, "compatible", plat_compat,
>> +                      sizeof(plat_compat))));
>> +
>> +    buf =  qemu_uuid_unparse_strdup(&qemu_uuid);
>> +    _FDT((fdt_setprop_string(fdt, 0, "vm,uuid", buf)));
>> +    if (qemu_uuid_set) {
>> +        _FDT((fdt_property_string(fdt, "system-id", buf)));
>> +    }
>> +    g_free(buf);
>> +
>> +    off = fdt_add_subnode(fdt, 0, "chosen");
>> +    if (kernel_cmdline) {
>> +        _FDT((fdt_setprop_string(fdt, off, "bootargs", kernel_cmdline)));
>> +    }
>> +
>> +    if (pnv->initrd_size) {
>> +        uint32_t start_prop = cpu_to_be32(pnv->initrd_base);
>> +        uint32_t end_prop = cpu_to_be32(pnv->initrd_base + 
>> pnv->initrd_size);
>> +
>> +        _FDT((fdt_setprop(fdt, off, "linux,initrd-start",
>> +                               &start_prop, sizeof(start_prop))));
>> +        _FDT((fdt_setprop(fdt, off, "linux,initrd-end",
>> +                               &end_prop, sizeof(end_prop))));
>> +    }
>> +
>> +    /* Put all the memory in one node on chip 0 until we find a way to
>> +     * specify different ranges for each chip
>> +     */
>> +    powernv_populate_memory_node(fdt, 0, 0, ram_size);
> 
> machine->ram_size is preferred over the ram_size global these days.

OK. np. I will change that. 

> Doesn't matter that much, since this will be reworked for multiple
> chips at some point anyway.

We are not very far from multichip I think. XICS will need a little
more work. This mostly because cpu_index will exceed nr_servers. 
We can work around that with a get_icp() handler. LPC and PSIHB will
need some minor tweaks.


Thanks,

C.

>> +
>> +    return fdt;
>> +}
>> +
>> +static void ppc_powernv_reset(void)
>> +{
>> +    MachineState *machine = MACHINE(qdev_get_machine());
>> +    PnvMachineState *pnv = POWERNV_MACHINE(machine);
>> +    void *fdt;
>> +
>> +    qemu_devices_reset();
>> +
>> +    fdt = powernv_create_fdt(pnv, machine->kernel_cmdline);
>> +
>> +    /* Pack resulting tree */
>> +    _FDT((fdt_pack(fdt)));
>> +
>> +    cpu_physical_memory_write(POWERNV_FDT_ADDR, fdt, fdt_totalsize(fdt));
>> +}
>> +
>> +static void ppc_powernv_init(MachineState *machine)
>> +{
>> +    PnvMachineState *pnv = POWERNV_MACHINE(machine);
>> +    ram_addr_t ram_size = machine->ram_size;
>> +    MemoryRegion *ram;
>> +    char *fw_filename;
>> +    long fw_size;
>> +    long kernel_size;
>> +
>> +    /* allocate RAM */
>> +    if (ram_size < (1 * G_BYTE)) {
>> +        error_report("Warning: skiboot may not work with < 1GB of RAM");
>> +    }
>> +
>> +    ram = g_new(MemoryRegion, 1);
>> +    memory_region_allocate_system_memory(ram, NULL, "ppc_powernv.ram",
>> +                                         ram_size);
>> +    memory_region_add_subregion(get_system_memory(), 0, ram);
>> +
>> +    /* load skiboot firmware  */
>> +    if (bios_name == NULL) {
>> +        bios_name = FW_FILE_NAME;
>> +    }
>> +
>> +    fw_filename = qemu_find_file(QEMU_FILE_TYPE_BIOS, bios_name);
>> +
>> +    fw_size = load_image_targphys(fw_filename, FW_LOAD_ADDR, FW_MAX_SIZE);
>> +    if (fw_size < 0) {
>> +        hw_error("qemu: could not load OPAL '%s'\n", fw_filename);
>> +        exit(1);
>> +    }
>> +    g_free(fw_filename);
>> +
>> +    /* load kernel */
>> +    kernel_size = load_image_targphys(machine->kernel_filename,
>> +                                      KERNEL_LOAD_ADDR, 0x2000000);
>> +    if (kernel_size < 0) {
>> +        hw_error("qemu: could not load kernel'%s'\n", 
>> machine->kernel_filename);
>> +        exit(1);
>> +    }
>> +
>> +    /* load initrd */
>> +    if (machine->initrd_filename) {
>> +        pnv->initrd_base = INITRD_LOAD_ADDR;
>> +        pnv->initrd_size = load_image_targphys(machine->initrd_filename,
>> +                                  pnv->initrd_base, 0x10000000); /* 128MB 
>> max */
>> +        if (pnv->initrd_size < 0) {
>> +            error_report("qemu: could not load initial ram disk '%s'",
>> +                         machine->initrd_filename);
>> +            exit(1);
>> +        }
>> +    }
>> +}
>> +
>> +static void powernv_machine_class_init(ObjectClass *oc, void *data)
>> +{
>> +    MachineClass *mc = MACHINE_CLASS(oc);
>> +
>> +    mc->desc = "IBM PowerNV (Non-Virtualized)";
>> +    mc->init = ppc_powernv_init;
>> +    mc->reset = ppc_powernv_reset;
>> +    mc->max_cpus = MAX_CPUS;
>> +    mc->block_default_type = IF_IDE; /* Pnv provides a AHCI device for
>> +                                      * storage */
>> +    mc->no_parallel = 1;
>> +    mc->default_boot_order = NULL;
>> +    mc->default_ram_size = 1 * G_BYTE;
>> +}
>> +
>> +static const TypeInfo powernv_machine_info = {
>> +    .name          = TYPE_POWERNV_MACHINE,
>> +    .parent        = TYPE_MACHINE,
>> +    .instance_size = sizeof(PnvMachineState),
>> +    .class_init    = powernv_machine_class_init,
>> +};
>> +
>> +static void powernv_machine_register_types(void)
>> +{
>> +    type_register_static(&powernv_machine_info);
>> +}
>> +
>> +type_init(powernv_machine_register_types)
>> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
>> new file mode 100644
>> index 000000000000..c8a73bc74267
>> --- /dev/null
>> +++ b/include/hw/ppc/pnv.h
>> @@ -0,0 +1,38 @@
>> +/*
>> + * QEMU PowerPC PowerNV various definitions
>> + *
>> + * Copyright (c) 2014-2016 BenH, IBM Corporation.
>> + *
>> + * This library is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2 of the License, or (at your option) any later version.
>> + *
>> + * This library is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with this library; if not, see 
>> <http://www.gnu.org/licenses/>.
>> + */
>> +#ifndef _PPC_PNV_H
>> +#define _PPC_PNV_H
>> +
>> +#include "hw/boards.h"
>> +
>> +#define TYPE_POWERNV_MACHINE       MACHINE_TYPE_NAME("powernv")
>> +#define POWERNV_MACHINE(obj) \
>> +    OBJECT_CHECK(PnvMachineState, (obj), TYPE_POWERNV_MACHINE)
>> +
>> +typedef struct PnvMachineState {
>> +    /*< private >*/
>> +    MachineState parent_obj;
>> +
>> +    uint32_t initrd_base;
>> +    long initrd_size;
>> +} PnvMachineState;
>> +
>> +#define POWERNV_FDT_ADDR                0x01000000
>> +
>> +#endif /* _PPC_PNV_H */
>