[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 26/26] tests/qtest/npcm7xx_rng-test: Disable randomness tests
From: |
Peter Maydell |
Subject: |
[PULL 26/26] tests/qtest/npcm7xx_rng-test: Disable randomness tests |
Date: |
Mon, 2 Nov 2020 17:10:05 +0000 |
The randomness tests in the NPCM7xx RNG test fail intermittently
but fairly frequently. On my machine running the test in a loop:
while QTEST_QEMU_BINARY=./qemu-system-aarch64 ./tests/qtest/npcm7xx_rng-test;
do true; done
will fail in less than a minute with an error like:
ERROR:../../tests/qtest/npcm7xx_rng-test.c:256:test_first_byte_runs:
assertion failed (calc_runs_p(buf.l, sizeof(buf) * BITS_PER_BYTE) > 0.01):
(0.00286205989 > 0.01)
(Failures have been observed on all 4 of the randomness tests,
not just first_byte_runs.)
It's not clear why these tests are failing like this, but intermittent
failures make CI and merge testing awkward, so disable running them
unless a developer specifically sets QEMU_TEST_FLAKY_RNG_TESTS when
running the test suite, until we work out the cause.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20201102152454.8287-1-peter.maydell@linaro.org
Reviewed-by: Havard Skinnemoen <hskinnemoen@google.com>
---
tests/qtest/npcm7xx_rng-test.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/tests/qtest/npcm7xx_rng-test.c b/tests/qtest/npcm7xx_rng-test.c
index da6e639bf6f..e7cde85fbbc 100644
--- a/tests/qtest/npcm7xx_rng-test.c
+++ b/tests/qtest/npcm7xx_rng-test.c
@@ -265,10 +265,16 @@ int main(int argc, char **argv)
qtest_add_func("npcm7xx_rng/enable_disable", test_enable_disable);
qtest_add_func("npcm7xx_rng/rosel", test_rosel);
- qtest_add_func("npcm7xx_rng/continuous/monobit", test_continuous_monobit);
- qtest_add_func("npcm7xx_rng/continuous/runs", test_continuous_runs);
- qtest_add_func("npcm7xx_rng/first_byte/monobit", test_first_byte_monobit);
- qtest_add_func("npcm7xx_rng/first_byte/runs", test_first_byte_runs);
+ /*
+ * These tests fail intermittently; only run them on explicit
+ * request until we figure out why.
+ */
+ if (getenv("QEMU_TEST_FLAKY_RNG_TESTS")) {
+ qtest_add_func("npcm7xx_rng/continuous/monobit",
test_continuous_monobit);
+ qtest_add_func("npcm7xx_rng/continuous/runs", test_continuous_runs);
+ qtest_add_func("npcm7xx_rng/first_byte/monobit",
test_first_byte_monobit);
+ qtest_add_func("npcm7xx_rng/first_byte/runs", test_first_byte_runs);
+ }
qtest_start("-machine npcm750-evb");
ret = g_test_run();
--
2.20.1
- [PULL 19/26] hw/display/omap_lcdc: Fix potential NULL pointer dereference, (continued)
- [PULL 19/26] hw/display/omap_lcdc: Fix potential NULL pointer dereference, Peter Maydell, 2020/11/02
- [PULL 15/26] target/arm: fix LORID_EL1 access check, Peter Maydell, 2020/11/02
- [PULL 18/26] hw/arm/boot: fix SVE for EL3 direct kernel boot, Peter Maydell, 2020/11/02
- [PULL 14/26] target/arm: fix handling of HCR.FB, Peter Maydell, 2020/11/02
- [PULL 20/26] hw/display/exynos4210_fimd: Fix potential NULL pointer dereference, Peter Maydell, 2020/11/02
- [PULL 21/26] target/arm: Get correct MMU index for other-security-state, Peter Maydell, 2020/11/02
- [PULL 23/26] hw/intc/arm_gicv3_cpuif: Make GIC maintenance interrupts work, Peter Maydell, 2020/11/02
- [PULL 22/26] configure: Test that gio libs from pkg-config work, Peter Maydell, 2020/11/02
- [PULL 25/26] qemu-option-trace.rst.inc: Don't use option:: markup, Peter Maydell, 2020/11/02
- [PULL 24/26] scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments, Peter Maydell, 2020/11/02
- [PULL 26/26] tests/qtest/npcm7xx_rng-test: Disable randomness tests,
Peter Maydell <=
- Re: [PULL 00/26] target-arm queue, Peter Maydell, 2020/11/03