Re: [Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

[Vladimir Sementsov-Ogievskiy]

12.01.2018 19:54, Eric Blake wrote:
> On 01/12/2018 05:43 AM, Vladimir Sementsov-Ogievskiy wrote:
>
> > > > +++ b/tests/qemu-iotests/201.out
> > > > @@ -0,0 +1,5 @@
> > > > +.......
> > > > +----------------------------------------------------------------------
> > > > +Ran 7 tests
> > > I'm not a fan of python tests that are difficult to debug.  Your
> > > additions to 147 in patch 4/6 are okay (hard to debug, but an
> > > incremental addition); but is it possible to rewrite this test in a bit
> > > more verbose manner?  See test 194 and this message for more details:
> > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg00234.html
> > hmm, what do you mean by "difficult to debug"? This is a usual python
> > unittest based test.
> And the list archives show several threads of people complaining that
> ./check failing with a diff that merely shows:
>
> -.....
> +..E..

didn't see that. usually, for failed iotests I see

-.....
+..E..

+ some kind of assert-fail in one of testcases

so we know in which testcase and in which line it was failed.

> makes it rather hard to see WHAT test 2 was doing that caused an error
> instead of a pass, let alone set up a reproduction scenario on JUST the
> failing test.  Yes, a lot of existing iotests use this unittest layout,
> and on that grounds, I'm not opposed to adding another one; but test 194
> really IS easier to debug when something goes wrong.
>
> > And there 3 test cases, sharing same setUp. Do not you say that unittest
> > becomes
> > deprecated in qemu? I think, if we have only one testcase, we may use
> > 194-like approach,
> > but if we have more, it's better to use unittest.
> Yes, I think a nice goal for improved testing is to write more
> python-based iotests in the style that uses actual output, and not just
> the unittest framework, in the test log.  It's not a hard requirement as
> long as no one has converted existing tests, but is food for thought.

I think, it doesn't mean that we should not use unittest at all, we just 
need more output with
it.

--
Best regards,
Vladimir