Re: qemu iotest 161 and make check

[Christian Borntraeger]

Am 10.02.22 um 18:44 schrieb Vladimir Sementsov-Ogievskiy:
> 10.02.2022 20:13, Thomas Huth wrote:
> > On 10/02/2022 15.51, Christian Borntraeger wrote:
> > > Am 10.02.22 um 15:47 schrieb Vladimir Sementsov-Ogievskiy:
> > > > 10.02.2022 10:57, Christian Borntraeger wrote:
> > > > > Hello,
> > > > >
> > > > > I do see spurious failures of 161 in our CI, but only when I use
> > > > > make check with parallelism (-j).
> > > > > I have not yet figured out which other testcase could interfere
> > > > >
> > > > > @@ -34,6 +34,8 @@
> > > > >   *** Commit and then change an option on the backing file
> > > > >
> > > > >   Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=1048576
> > > > > +qemu-img: TEST_DIR/t.IMGFMT.base: Failed to get "write" lock
> > > > > +Is another process using the image [TEST_DIR/t.IMGFMT.base]?
> > > > >   Formatting 'TEST_DIR/t.IMGFMT.int', fmt=IMGFMT size=1048576 
> > > > > backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> > > > >   Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 
> > > > > backing_file=TEST_DIR/t.IMGFMT.int backing_fmt=IMGFMT
> > > > >   { 'execute': 'qmp_capabilities' }
> > > > >
> > > > >
> > > > > any ideas?
> > > >
> > > > Hmm, interesting.. Is it always 161 and always exactly this diff?
> > >
> > > Its always 161 and only 161. I would need to check if its always the same error.
> > >
> > > > First, this place in 161 is usual: we just create and image, like in many other 
> > > > tests.
> > > >
> > > > Second, why _make_test_img trigger "Failed to get write lock"? It should just 
> > > > create an image. Hmm. And probably starts QSD if protocol is fuse. So, that start of QSD 
> > > > may probably fail.. Is that the case? What is image format and protocol used in test run?
> > > >
> > > > But anyway, tests running in parallel should not break each other as each test 
> > > > has own TEST_DIR and SOCK_DIR..
> >
> > Unless you run into the issue that Hanna described here:
> >
> >   https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01735.html
>
> Yes, we can't execute same test several times (for different formats) in 
> parallel.. But that's about any test, not only 161.
>
> And I don't think that it's currently possible that we run same test in 
> parallel several times somewhere, do we? In tests/check-block.sh we have a 
> sequential loop through $format_list ..

FWIW, I was able to bisect this and it came in with

bcda7b178fde7797f476e3b066fe5fc76bfa1c43 is the first bad commit
commit bcda7b178fde7797f476e3b066fe5fc76bfa1c43
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date:   Thu Dec 23 19:39:33 2021 +0100

    check-block.sh: passthrough -jN flag of make to -j N flag of check
    
    This improves performance of running iotests during "make -jN check".
    
    Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
    Message-Id: <20211223183933.1497037-1-vsementsov@virtuozzo.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

 tests/check-block.sh | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)



With

make check-block -j 100

it reproduced pretty quickly for me.