Re: [Qemu-devel] [PATCH for-4.0 0/2] test: replace gtester with a TAP driver

[Cleber Rosa]

On 11/30/18 4:58 AM, Paolo Bonzini wrote:
> On 30/11/18 08:21, Thomas Huth wrote:
>> On 2018-11-29 21:43, Eric Blake wrote:
>>> On 11/29/18 11:45 AM, Paolo Bonzini wrote:
>>>> gtester is deprecated by upstream glib and it does not support tests
>>>> that call g_test_skip in some glib stable releases.
>>>>
>>>> glib suggests instead using Automake's TAP support.  We do not support
>>>> Automake, but we can copy the code that beautifies the TAP output and
>>>> use it.  I chose to use the Perl copy rather than the shell/awk one,
>>>> in order to reuse Perl's TAP parsing package, but I'm open to suggestions
>>>> about which language to use.
>>>
>>> I'm less familiar with the TAP protocol than I'd like to admit, but I
>>> did find:
>>>
>>> https://blog.gtk.org/2018/07/11/news-from-glib-2-58/
>>>
>>> which corroborates your claim that switching to Automake's TAP parser is
>>> indeed a recommended approach to avoid the now-deprecated gtester.
>>
>> Hmm, do we really have to maintain our own version of a test runner now?
> 
> Well, that's what you get for using a custom build system but it's
> "just" an output beautifier really.  I could just "grep -ve ^# -e ^ok"
> but it wouldn't look very nice (plus things that start as 1-line shell
> scripts end up being the same as the thing you're trying to replace).
> 
> I don't think it will be a big maintenance headache---just like most
> other files in scripts/, some of which have had exactly zero commits
> since they were added.  In fact, Automake has hardly changed its driver
> since the first commit.
> 
>> There's already some effort going on to use avocado in "make
>> check-acceptance" ... maybe we could use avocado as replacement for
>> gtester, too?
> 
> Avocado does not have the required functionality, unfortunately.  It can
> produce TAP, but not consume it.  Also, Python libraries for TAP do not
> abound and are generally not shipped with the OS (apart from Perl, which
> basically invented TAP, the language with the most TAP libraries seem to
> be JavaScript!).
> 

Avocado can't currently consume TAP (it produces it, though), but it has
some integration with glib based tests:

https://avocado-framework.readthedocs.io/en/66.0/optional_plugins/glib.html

But, there may be bits missing to fully replae gtests.  Given that we
have been adding features to Avocado based on QEMU requirements, we can
certainly look at adding missing pieces there.  If this sounds like
something you'd be interested in, I can come up with an integration
proposal.

> Paolo
> 

Cheers,
- Cleber.