qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v7 09/11] iotests: add testrunner.py


From: Vladimir Sementsov-Ogievskiy
Subject: Re: [PATCH v7 09/11] iotests: add testrunner.py
Date: Fri, 22 Jan 2021 18:01:24 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1

22.01.2021 17:51, Kevin Wolf wrote:
Am 16.01.2021 um 14:44 hat Vladimir Sementsov-Ogievskiy geschrieben:
Add TestRunner class, which will run tests in a new python iotests
running framework.

There are some differences with current ./check behavior, most
significant are:
- Consider all tests self-executable, just run them, don't run python
   by hand.
- Elapsed time is cached in json file
- Elapsed time precision increased a bit
- use python difflib instead of "diff -w", to ignore spaces at line
   ends strip lines by hand. Do not ignore other spaces.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Sorry for sending so many individual mails,

Not a problem

but I'm running into things
as I actually run the code during the review of the final 'check'
rewrite.

I get this output for a failing test case:

     001   fail       [15:44:41] [15:44:42]   0.5s   (last: 13.2s) output 
mismatch (see {f_bad})
     --- /home/kwolf/source/qemu/tests/qemu-iotests/001.out

     +++ 001.out.bad

     @@ -1,4 +1,4 @@

     -QA output created by 001
     +Broken QA output created by 001
      Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

      == reading whole image ==
     Failures: 001

Somehow we get additional empty lines in the diff header that shouldn't
be there. Not sure where it comes from. Especially the empty line after
@@ even corrupts the patch if you try to apply it.

Looking at it.

Seems the problem is that difflib adds '\n' for lines that it generates, but 
not for lines passed to it as lines list. And then I work with final lines list 
like there no embedded '\n' at all.

this helps:

--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -50,7 +50,9 @@ def file_diff(file1: str, file2: str) -> List[str]:
         # and use strict diff here!
         seq1 = [line.rstrip() for line in f1]
         seq2 = [line.rstrip() for line in f2]
-        return list(difflib.unified_diff(seq1, seq2, file1, file2))
+        res = [line.rstrip()
+               for line in difflib.unified_diff(seq1, seq2, file1, file2)]
+        return res



The other problem is obvious enough:

+        diff = file_diff(str(f_reference), str(f_bad))
+        if diff:
+            return TestResult(status='fail', elapsed=elapsed,
+                              description='output mismatch (see {f_bad})',

This needs to be an f-string.

Kevin



--
Best regards,
Vladimir



reply via email to

[Prev in Thread] Current Thread [Next in Thread]