[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH 20/40] mtest2make: stop disabling meson test timeouts
From: |
Alex Bennée |
Subject: |
[PATCH 20/40] mtest2make: stop disabling meson test timeouts |
Date: |
Thu, 21 Dec 2023 10:37:58 +0000 |
From: Daniel P. Berrangé <berrange@redhat.com>
The mtest2make.py script passes the arg '-t 0' to 'meson test' which
disables all test timeouts. This is a major source of pain when running
in GitLab CI and a test gets stuck. It will stall until GitLab kills the
CI job. This leaves us with little easily consumable information about
the stalled test. The TAP format doesn't show the test name until it is
completed, and TAP output from multiple tests it interleaved. So we
have to analyse the log to figure out what tests had un-finished TAP
output present and thus infer which test case caused the hang. This is
very time consuming and error prone.
By allowing meson to kill stalled tests, we get a direct display of what
test program got stuck, which lets us more directly focus in on what
specific test case within the test program hung.
The other issue with disabling meson test timeouts by default is that it
makes it more likely that maintainers inadvertantly introduce slowdowns.
For example the recent-ish change that accidentally made migrate-test
take 15-20 minutes instead of around 1 minute.
The main risk of this change is that the individual test timeouts might
be too short to allow completion in high load scenarios. Thus, there is
likely to be some short term pain where we have to bump the timeouts for
certain tests to make them reliable enough. The preceeding few patches
raised the timeouts for all failures that were immediately apparent
in GitLab CI.
Even with the possible short term instability, this should still be a
net win for debuggability of failed CI pipelines over the long term.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20230717182859.707658-13-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231215070357.10888-17-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
scripts/mtest2make.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 179dd548718..eb01a05ddbd 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -27,7 +27,8 @@ def names(self, base):
.speed.slow = $(foreach s,$(sort $(filter-out %-thorough, $1)), --suite $s)
.speed.thorough = $(foreach s,$(sort $1), --suite $s)
-.mtestargs = --no-rebuild -t 0
+TIMEOUT_MULTIPLIER = 1
+.mtestargs = --no-rebuild -t $(TIMEOUT_MULTIPLIER)
ifneq ($(SPEED), quick)
.mtestargs += --setup $(SPEED)
endif
--
2.39.2
- [PATCH 25/40] target/arm: Use GDBFeature for dynamic XML, (continued)
- [PATCH 25/40] target/arm: Use GDBFeature for dynamic XML, Alex Bennée, 2023/12/21
- [PATCH 27/40] target/riscv: Use GDBFeature for dynamic XML, Alex Bennée, 2023/12/21
- [PATCH 33/40] hw/core/cpu: Remove gdb_get_dynamic_xml member, Alex Bennée, 2023/12/21
- [PATCH 32/40] gdbstub: Infer number of core registers from XML, Alex Bennée, 2023/12/21
- [PATCH 35/40] plugins: Use different helpers when reading registers, Alex Bennée, 2023/12/21
- [PATCH 24/40] target/riscv: Validate misa_mxl_max only once, Alex Bennée, 2023/12/21
- [PATCH 31/40] gdbstub: Simplify XML lookup, Alex Bennée, 2023/12/21
- [PATCH 26/40] target/ppc: Use GDBFeature for dynamic XML, Alex Bennée, 2023/12/21
- [PATCH 16/40] tests/qtest: Bump the device-introspect-test timeout to 12 minutes, Alex Bennée, 2023/12/21
- [PATCH 34/40] gdbstub: Add members to identify registers to GDBFeature, Alex Bennée, 2023/12/21
- [PATCH 20/40] mtest2make: stop disabling meson test timeouts,
Alex Bennée <=
- [PATCH 17/40] tests/unit: Bump test-aio-multithread test timeout to 2 minutes, Alex Bennée, 2023/12/21
- [PATCH 28/40] gdbstub: Use GDBFeature for gdb_register_coprocessor, Alex Bennée, 2023/12/21
- [PATCH 21/40] hw/riscv: Use misa_mxl instead of misa_mxl_max, Alex Bennée, 2023/12/21
- [PATCH 29/40] gdbstub: Use GDBFeature for GDBRegisterState, Alex Bennée, 2023/12/21
- [PATCH 18/40] tests/unit: Bump test-crypto-block test timeout to 5 minutes, Alex Bennée, 2023/12/21
- [PATCH 30/40] gdbstub: Change gdb_get_reg_cb and gdb_set_reg_cb, Alex Bennée, 2023/12/21