[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Proposal for a regular upstream performance testing

From: Lukáš Doktor
Subject: Re: Proposal for a regular upstream performance testing
Date: Thu, 26 Nov 2020 12:16:04 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1

Dne 26. 11. 20 v 11:17 Peter Maydell napsal(a):
On Thu, 26 Nov 2020 at 08:13, Lukáš Doktor <ldoktor@redhat.com> wrote:
The goal of this initiative is to detect system-wide performance
regressions as well as improvements early, ideally pin-point the
individual commits and notify people that they should fix things.
All in upstream and ideally with least human interaction possible.

So, my general view on this is that automated testing of performance
is nice, but unless it also comes with people willing to do the
work of identifying and fixing the causes of performance regressions
there's a risk that it degrades into another automated email or set
of graphs that nobody pays much attention to (outside of the obvious
"oops this commit dropped us by 50%" mistakes). As with fuzz testing,
I'm a bit wary of setting up an automated system that just pushes
work onto humans who already have full plates. Is RedHat also
planning to set performance requirements and assign engineers to
keep us from falling below them ?

-- PMM

In the proposed "solution 1" my role would be to maintain, judge and help 
analyzing the reports if needed. As for the fixing the code I can not serve, that would 
have to be on the individual contributors/maintainers, the best I can do is to bisect and 
CC the authors.

In "solution 2" that would be on the other volunteer with my assistance, if 
needed. Note that currently the pipeline is not that clever so it requires manual 
interaction for bisection but I do have plan on improving that soon.

Also note that the purpose of this email is also a call for ideas, because 
maybe there is a better tool out there and if it fitted our needs better I 
wouldn't mind to switching to it.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]