[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v2 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update
From: |
Daniel P . Berrangé |
Subject: |
[PATCH v2 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file |
Date: |
Fri, 4 Mar 2022 19:36:04 +0000 |
When developing an I/O test it is typical to add some logic to the
test script, run it to view the output diff, and then apply the
output diff to the reference file. This can be drastically simplified
by letting the test runner update the reference file in place.
By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the
failure and show the diff, but at the same time update the reference
file. So next time the I/O test is run it will succeed.
Continuing to display the diff when updating the reference gives the
developer a chance to review what was changed.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/qemu-iotests/testrunner.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index 9a94273975..8a82696a6b 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -25,6 +25,7 @@
import contextlib
import json
import termios
+import shutil
import sys
from multiprocessing import Pool
from contextlib import contextmanager
@@ -320,6 +321,11 @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
diff = file_diff(str(f_reference), str(f_bad))
if diff:
+ if os.environ.get("QEMU_IOTESTS_REGEN", None) is not None:
+ shutil.copyfile(str(f_bad), str(f_reference))
+ print("########################################")
+ print("##### REFERENCE FILE UPDATED #####")
+ print("########################################")
return TestResult(status='fail', elapsed=elapsed,
description=f'output mismatch (see {f_bad})',
diff=diff, casenotrun=casenotrun)
--
2.34.1
- [PATCH v2 01/12] crypto: mandate a hostname when checking x509 creds on a client, (continued)
- [PATCH v2 01/12] crypto: mandate a hostname when checking x509 creds on a client, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 03/12] block/nbd: support override of hostname for TLS certificate validation, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 05/12] block/nbd: don't restrict TLS usage to IP sockets, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 02/12] block: pass desired TLS hostname through from block driver client, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 04/12] qemu-nbd: add --tls-hostname option for TLS certificate validation, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 08/12] tests/qemu-iotests: introduce filter for qemu-nbd export list, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file,
Daniel P . Berrangé <=
- [PATCH v2 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 07/12] tests/qemu-iotests: expand _filter_nbd rules, Daniel P . Berrangé, 2022/03/04
- [PATCH v2 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK, Daniel P . Berrangé, 2022/03/04