[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos
From: |
Philippe Mathieu-Daudé |
Subject: |
Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210 |
Date: |
Thu, 10 Oct 2019 15:43:35 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 |
On 10/9/19 9:07 PM, Cleber Rosa wrote:
On Wed, Oct 09, 2019 at 02:38:02PM +0100, Peter Maydell wrote:
On Tue, 8 Oct 2019 at 22:49, Cleber Rosa <address@hidden> wrote:
On Mon, Oct 07, 2019 at 05:28:49PM +0100, Peter Maydell wrote:
>>>
>>>> Do our other acceptance tests download random third-party
>>>> (ie "not a well-known distro") binaries for the tests ?
>>>> It seems a bit hazardous for reproducability and trustability
>>>> reasons...
[...]
I find it hard to judge precisely how much of a third-party some of
these are. I remember Philippe mentioning that one of them, I guess
the images used on linux_ssh_mips_malta.py, were "as official as it
gets" (my words, from my often misleading memory).
Reproducibility is definitely an issue, in the sense given that some
of these can indeed go away, but as long as they're available the hash
recorded in the test should guarantee that we're running the same
images.
So this thread is a follow up on:
https://www.mail-archive.com/address@hidden/msg546430.html
For Open Source software I can understand we want to be able to rebuild
them, and should provide a link about how to rebuild.
I don't want to rebuild images myself, I want to focus on testing QEMU.
I tried once to build a MIPSsim kernel and it was a total nightmare:
https://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg04071.html
(Thomas Huth eventually succeeded using buildroot).
Do you think we should do something different here?
I'm not sure, which is why I asked whether this new test
was in line with what we've done previously. Since these
are just test cases and we don't redistribute them to
other people there's less of a traceability/reproducibility
worry, and if we check hashes on download that cuts off
a lot of "fail to notice if the image changes for some
reason" possible problems.
Yep, because I have no clue how to do improve on this (redistributing
the binaries is definitely not on the improvement side, and neither
is not testing some machine types), the current approach seems good.
QEMU machines are not restricted to running Linux or other Open Source
software. It seems important to be able to test for regressions with
closed-source code too, because it usually has been well tested on real
hardware.
I just posted a firmware test:
https://www.mail-archive.com/address@hidden/msg651012.html
I could find it stored compressed and uuencoded on the Wayback Machine.
Since I don't want to abuse from this amazing service, and other script
to decode/uncompress it, I stored it on a new repository with its SHA-1
(the default hash used by Avocado):
https://github.com/philmd/qemu-testing-blob/tree/master/hppa/hp9000/712
Is this acceptable? Incorrect?
Regarding Avocado tests, maybe we can simply add a "untrusted" or
"closedsource" tag, so people willing to test untrusted binaries could
still run the tests using 'avocado --tag untrusted_software', and we
could skip these tests by default.
I am very interested because I already experimented with:
- AIX on PReP/40p
- some RTOS on Canon PowerShot A1100
- VxWorks on MIPS and SPARC
- closed-source bootloader for Raspi3/4
Regards,
Phil.
- [PATCH 0/5] hw/arm/exynos4210: Add acceptance tests to the SMDKC210 board, Philippe Mathieu-Daudé, 2019/10/05
- [PATCH 2/5] hw/sd/sdhci: Add a comment to distinct the i.MX eSDHC functions, Philippe Mathieu-Daudé, 2019/10/05
- [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Philippe Mathieu-Daudé, 2019/10/05
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Peter Maydell, 2019/10/07
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Cleber Rosa, 2019/10/08
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Guenter Roeck, 2019/10/08
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Peter Maydell, 2019/10/09
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Cleber Rosa, 2019/10/09
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210,
Philippe Mathieu-Daudé <=
- Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Philippe Mathieu-Daudé, 2019/10/21
Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210, Cleber Rosa, 2019/10/08
[PATCH 3/5] hw/sd/sdhci: Add dummy Samsung SDHCI controller, Philippe Mathieu-Daudé, 2019/10/05
[PATCH 4/5] hw/arm/exynos4210: Use the Samsung s3c SDHCI controller, Philippe Mathieu-Daudé, 2019/10/05
[PATCH 5/5] tests/boot_linux_console: Add sdcard test for the Exynos4210, Philippe Mathieu-Daudé, 2019/10/05
Re: [PATCH 0/5] hw/arm/exynos4210: Add acceptance tests to the SMDKC210 board, Krzysztof Kozlowski, 2019/10/07