Re: [PATCH v8 0/5] Rework iotests/check

[Kevin Wolf]

Am 25.01.2021 um 17:36 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 25.01.2021 19:08, Kevin Wolf wrote:
> > Am 23.01.2021 um 22:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > v8:
> > > 
> > > about linters:
> > > 
> > > I didn't modify 297, as Max already staged 297 modifications to test all 
> > > files.
> > > 
> > > Also, now I have two complains:
> > > +************* Module testenv
> > > +testenv.py:158:4: R0915: Too many statements (53/50) 
> > > (too-many-statements)
> > > +************* Module testrunner
> > > +testrunner.py:222:4: R0911: Too many return statements (7/6) 
> > > (too-many-return-statements)
> > >   Success: no issues found in 5 source files
> > > 
> > > And I feel, I'm tired to refactor it now.. Probably we can ignore them in 
> > > 297. Probably I can
> > > do some refactoring as a follow-up.
> > 
> > I don't think these warning are very helpful, I would agree with
> > disabling them (even globally).
> > 
> > When testing this with the other image formats, I found some problems.
> > 
> > 1. The first one probably means that we have changed the order of some
> >     checks: 150 and 178 have reference outputs for raw and qcow2, but no
> >     other formats.
> > 
> >     Previously, the _supported_fmt line in the test would just skip the 
> > test:
> > 
> >     $ build/check -vhdx 150 178
> >     150      not run    [16:45:46] [16:45:46]                    not 
> > suitable for this image format: vhdx
> >     178      not run    [16:45:46] [16:45:46]                    not 
> > suitable for this image format: vhdx
> > 
> >     Now we seem to test first if a reference output exists and fail:
> > 
> >     150   fail       [16:49:18] [16:49:18]   ...                  No 
> > qualified output (expected 
> > /home/kwolf/source/qemu/tests/qemu-iotests/150.out)
> >     178   fail       [16:49:18] [16:49:18]   ...                  No 
> > qualified output (expected 
> > /home/kwolf/source/qemu/tests/qemu-iotests/178.out)
> 
> 
> Hmm. Still, I do think that new order is better: no reason to run the
> test, when we don't have corresponding .out file. So, may be just
> change it into "not run", with same "No qualified output (expected
> ..)" message, what do you think?

Works for me.

(There would actually be a reason to run the test, namely for creating
the reference output when you add the test. But this didn't leave a .bad
file behind before either, and just doing 'touch 123.out' first is easy
enough anyway.)

Kevin