Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sat, Oct 3, 2015 at 2:51 PM, Peter Maydell <address@hidden> wrote:
> On 3 October 2015 at 22:31, Peter Crosthwaite
> <address@hidden> wrote:
>> Hi,
>>
>> I have done an audit of the ARMv7 boards to see what can boot a
>> vanilla linux kernel. The basic approach is to build ARM
>> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
>> the kernel. The intersection of what mainline Linux has a DTB for and
>> what QEMU models is tested.
>
> Thanks for doing this; this is an interesting survey of what
> we implement.
>
>> The boards that do/should work include:
>>
>>     vexpress-a9
>>     vexpress-a15
>>     smdkc210 (exynos)
>>     xilinx-zynq-a9
>>     cubieboard (allwinner)
>>     highbank
>>     midway
>>     virt
>
> FWIW I have test images for vexpress-a9, vexpress-a15,
> cubieboard and virt, though of course these are often
> older kernels and probably tweaked configs.
>

Yes. I am avoiding tweaked configs, as I am trying to be
representative of someone trying to RYO Linux without QEMU
implementation knowledge.

>> virt:
>>
>> Virt largely works, but there are no immediately obvious
>> storage/network options that are supported by the kernel. So I have to
>> make some exceptions to the kernel config rule, that is, I add:
>>
>> CONFIG_VIRTIO_PCI=y"
>> CONFIG_VIRTIO_BLK=y"
>> CONFIG_VIRTIO_NET=y"
>>
>> To get disks and network for the virt board.
>>
>> Could we add these configs to mainline Linux's defconfig? Note, we
>> don't need to add a DTS for this board, as QEMU's bootloader will
>> generate and provide it for us.
>>
>> There is the known highmem PCI issue that is currently under discussion:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
>>
>> So aside from the missing VIRTIO kernel configs and highmem bug, virt
>> is in good shape.
>
> Did you build your kernel with LPAE or not? I think an LPAE
> config ought to avoid the PCI highmem bug (and it's definitely
> what you want for anything that's Cortex-A15 based).
>

No. Although any missing configs I consider to be our problem, or a
bug in the multi_v7_defconfig in upstream Linux. That defconfig really
should work for us.

>> vexpress:
>>
>> vexpress boots up to rootfs probing, however the only storage media
>> that seems to be supported is flash, which doesn't have the needed
>> kernel configs to boot from (not 100% sure on that but I gave up
>> quickly). What are people who use this board in real hardware doing?
>> It may just work best with initrd style boots.
>
> For real hardware I expect people will be using USB disks.
> We don't currently model the USB controller in QEMU, though.

Looks like a custom job too. Not much chance here unless this is a
rebadging of one of the existing HCIs?

> My test images use emulated SD card storage (ie -drive if=sd,file=...).
> (Emulated SD card will have lousy performance.)
>

Ok that's a good plan A. Emulated SDHCI on Zynq does just fine for
this level of testing. I missed it as my options search was for "SD"
rather than "MMC".

>> realview:
>>
>> Realview is not tested, but I discuss it here as it is a bit of a red
>> herring. The kernel has dtb support for the realview-1176 board,
>
> Actually, "realview-pb1176". The "PB" part is important!
>
>> however this is not modelled by QEMU. I creating a Realview variant
>> with 1176 in QEMU, however the memory maps of all the peripherals were
>> mismatched.
>>
>> Turns, out, QEMU is is modelling the realview FPGA emulation platform
>> which has a different peripheral map to the 1176 realview board
>> proper. So QEMU doesn't support the realview boards, just the FPGA
>> emulation platform, for which I can't find a kernel dtb. Should we
>> upstream one with some dtsi's for the CPU variation supported in QEMU?
>
> "realview" is not really a very helpful term to use here, because
> it's a generic label applied to a whole slew of ARM devboards.
> What QEMU models is:
>  "realview-eb" -- the RealView Emulation Baseboard with an ARM926
>  "realview-eb-mpcore" -- the RealView Emulation Baseboard with an 11MPCore
>  "realview-pb-a8" -- the RealView Platform Baseboard for Cortex A8
>  "realview-pbx-a9" -- the RealView Platform Baseboard Explore for Cortex A9
>
> The DT in the kernel tree is for the Realview Platform Baseboard
>   for 1176, ie PB1176. That's a different board from EB1176, as
> you've found. This is why "realview" on its own (or with a CPU
> name) is not sufficient to identify a board and why we have those
> -eb- and -pb(x)- infixes in our board names :-)
>

So does this mean that the EB memory maps are consistent but perhaps
not the PB? Can we reliably gen all combinations with EB vs PB with
just two memory maps, then add 1176 both ways?

Regards,
Peter

>> So Realview does not work within my testing parameters with no
>> immediate prospects.
>
> The kernel does support EB, PBA8 and PBX; presumably they haven't
> been converted to use DT (yet?).
>
>> how I am testing:
>>
>> So this is all powered by the Yocto project (Poky). I got some good
>> help from Nathan Rossi. I have a poky fork which changes the qemuarm
>> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
>> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
>> type of machine (from the list above) to yocto's runqemu command. The
>> command sets up boot media and network automatically.
>>
>> To use it:
>>
>> git clone https://github.com/pcrost/poky.git
>> cd poky
>> source ./oe-init-build-env
>> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
>> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
>>
>> The QEMUBIN is optional as Yocto does build QEMU for you, but this
>> lets you BYO QEMU if you are doing qemu development outside of the
>> Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to
>> see results. I suggest starting with virt. Skip cubieboard, that
>> assumes my SATA patches. Everything else you will see varying degrees
>> of success. Pass qemuparams="-m 256" to runqemu for the smdkc200
>> (Exynos) board. Highbank and Midway will blank screen as the
>> outstanding issues happen pre-UART. A meaningful logbuf can be pulled
>> from RAM over the monitor. Useful instructions here:
>>
>> http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction
>>
>> Some QEMU patches to follow to fix a few of the things mentioned.
>>
>> We should get this into regression testing. Yocto has some automated
>> testing features in it's own right that we should be able to leverage.
>> Yet to investigate (Richard pasted some stuff on an earlier thread).
>
> Nice work.
>
> thanks
> -- PMM