Re: [Qemu-devel] [Qemu-block] [PATCH] tests/qemu-iotests/235: Allow fallback to tcg and remove it from quick group

[John Snow]

On 3/21/19 4:23 AM, Thomas Huth wrote:
> On 20/03/2019 18.32, John Snow wrote:
>>
>>
>> On 3/1/19 7:20 AM, Thomas Huth wrote:
>>> iotest 235 currently only works with KVM - this is bad for systems where
>>> it is not available, e.g. CI pipelines. The test also works when using
>>> "tcg" as accelerator, so we can simply add that to the list of accelerators,
>>> too. But still, there might be the case that someone compiled their
>>> QEMU with --disable-tcg and still try to run the iotests in a CI pipeline
>>> where KVM is not available - in that case it would be best to use the
>>> "qtest" accelerator for this test. However, that currently hangs and I
>>> did not succeed to get it working with "accel=qtest" yet. Thus, as long
>>> as this is not fixed, it's likely better to remove this test from the
>>> "quick" group so that it does not fail on CI pipelines.
>>>
>>> Signed-off-by: Thomas Huth <address@hidden>
>>
>> What happens if we set accel=kvm:tcg but leave it in quick group? It
>> will still fail in those cases where we have neither accelerator for
>> whichever reason, but maybe that's better than just trying to prevent it
>> from running ... ?
> 
> ... but since it is failing, you can't use "make check-block" in the CI
> pipelines in that case. I had to manually select the test cases in the
> .gitlab-ci.yml file due to this. I don't mind too much, but IMHO "make
> check-block" should simply run everywhere, with every QEMU binary, and
> not only if you've compiled it with the right options and/or KVM is
> available. Otherwise, we maybe need a "make check-block-simple" or "make
> check-block-ci" target, that uses another new group beside the "quick"
> group? I.e. introduce a "ci" or "simple" group into
> tests/qemu-iotests/group ?
> 

It still feels like papering over the problem to me; if the test is
failing on qtest that feels like an accurate test result we shouldn't
hide or ignore...?

I realize that's inconvenient, but is it common for us to have neither
TCG nor KVM? (I genuinely don't know. I'd have thought that's fairly
rare? Clearly not for you.)

If the problem is that the test suite stops on first failure, should we
amend the execution to continue on in those cases?

--js

(I guess I should look at why the test doesn't work under qtest, but I
have some downstream work to finish before I could prioritize that.)