[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] QEMU Summit 2017: minutes
From: |
Cornelia Huck |
Subject: |
Re: [Qemu-devel] QEMU Summit 2017: minutes |
Date: |
Tue, 28 Nov 2017 10:36:32 +0100 |
On Tue, 28 Nov 2017 09:33:52 +0100
Thomas Huth <address@hidden> wrote:
> On 27.11.2017 23:03, John Snow wrote:
> >
> > On 11/23/2017 11:31 AM, Peter Maydell wrote:
> [...]
> >> Continuous Integration:
> >> * Christian Borntraeger: qemu-iotests have broken a lot, they should be
> >> run before patches are merged
> >
> > This, rather unfortunately, is a huge testing burden. I try to make sure
> > I do it for everything I submit, but for the volume of block patches it
> > really does rely CI. The more we add (to our pitifully sparse iotesting,
> > I might add) the longer it takes. Ensuring per-patch testing begins to
> > take prohibitively long.
> >
> > Perhaps per-pull or per-merge becomes more feasible. Maybe if we do
> > implement a block-next amalgam we'd be able to batch our testing on a
> > weekly basis.
>
> I think you block-layer folks should do at least run the qemu-iotests
> before sending a pull request to Peter. The iotests should really not be
> broken in upstream master.
This is unlikely to cover the iotest failures on s390 (due to usage of
ccw, strange backing devices, etc.), though. We have basically two
options here:
- Continue to rely on the IBM folks finding those problems (which will
likely be post-merge, but better than nothing.)
- Have patchew (which has a bot on s390) execute the iotests - which is
time-consuming.
>
> >> * Peter Maydell: If it isn't tested by "make check" then it isn't tested:
> >> so if something is regularly regressing then it needs to be added to
> >> "make check".
> >
> > Is this tenable long term? We can't conceivably state that we will never
> > test things that aren't in "make check" -- we ought to have different
> > tiers, at least. The full testing suite should run for RC tags at least,
> > but it's not feasible (I think?) to run the entire battery of tests on
> > every commit... but that shouldn't stop us from running them /sometimes/...
> >
>
> We've already got "make check SPEED=slow" for running tests that take a
> lot of time. So maybe you could do that in the iotests as well, so that
> the normal, quick tests can be run during "make check" and the full
> iotest suite is only run during "make check SPEED=slow" ?
+1 to that. Having a subset covered by default is better than nothing
at all.