Re: [PATCH] scripts: display how long each test takes to execute

[Daniel P . Berrangé]

On Mon, Sep 14, 2020 at 01:19:20PM +0200, Paolo Bonzini wrote:
> On 14/09/20 13:09, Daniel P. Berrangé wrote:
> > I'm unclear if meson's native test runner can print timings. If not,
> > we might want to submit an RFE there too.
> 
> I agree that any holes should be filled in there.  In this case it does,
> so I think we should start using it in CI so that RFEs can be sent there.
> 
> mtest2make-style output has been in use (and good enough) for years so
> I'd rather avoid piling up more hacks on top.  "meson test" is not
> perfect but I'd rather improve it instead.

When I run "meson test" it doesn't use the results from the previous
"make", instead it re-compiles the entire codebase using ninja.

If we're telling users to continue to use "make" and "make check" though,
I don't think we should be using "meson test" in the CI systems, as it
means CI is not testing the same build process as our users, which is
defeating the point of CI.

If using "meson compile" and "meson test" already works though, what
is our current ninja -> make convertor doing for us, besides letting
people have a facade to pretend nothing has changed ?


> >  scripts/mtest2make.py | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> > index 9cbb2e374d..9103ae65b9 100644
> > --- a/scripts/mtest2make.py
> > +++ b/scripts/mtest2make.py
> > @@ -20,7 +20,7 @@ print('''
> >  SPEED = quick
> >  
> >  # $1 = environment, $2 = test command, $3 = test name, $4 = dir
> > -.test-human-tap = $1 $(if $4,(cd $4 && $2),$2) < /dev/null | 
> > ./scripts/tap-driver.pl --test-name="$3" $(if $(V),,--show-failures-only)
> > +.test-human-tap = export then=`date +%s` ; $1 $(if $4,(cd $4 && $2),$2) < 
> > /dev/null | ./scripts/tap-driver.pl --test-name="$3" $(if 
> > $(V),,--show-failures-only) ; export now=`date +%s` ; delta=`expr $$now - 
> > $$then` ; $(if $(V),echo "TIME $$delta seconds",true)
> >  .test-human-exitcode = $1 $(PYTHON) scripts/test-driver.py $(if $4,-C$4) 
> > $(if $(V),--verbose) -- $2 < /dev/null
> >  .test-tap-tap = $1 $(if $4,(cd $4 && $2),$2) < /dev/null | sed 
> > "s/^[a-z][a-z]* [0-9]*/& $3/" || true
> >  .test-tap-exitcode = printf "%s\\n" 1..1 "`$1 $(if $4,(cd $4 && $2),$2) < 
> > /dev/null > /dev/null || echo "not "`ok 1 $3"
> > -- 2.26.2
> 
> 
> Paolo
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|