[Koha-devel] QA manager

[Pierrick LE GALL]

Hi koha-devel,

On the last Koha team meeting of March 20th, 2006, we discussed about
the position INEO (me for instance) could officialy take on the
project. Joshua and Paul proposed me to be Quality Assurance (QA)
manager, and I personnaly accepted this role for a two months try. I've
just asked my INEO managers for a formal validation.

On IRC, the definition about QA manager was discussed, because
everybody doesn't define it the same way. Paul said "responsible for
heavy testing" and I answered I did not agree with this point of view,
not in OSS model. thd asked what difference I made between open and
closed source QA. Let's answer this question by mail, as requested by
Russ. See details on IRC logs [1].

I first answer to the question "what difference I make between open and
closed source QA", then I'll describe what is my vision of QA manager
for an OSS project like Koha.

QA, proprietay Vs OSS
=====================

I've worked on the two sides of softwares : proprietary and OSS. I
think I understand the advantages and drawbacks of each model,
concerning QA.

In a proprietary model, the software *must* be /stable/ when reaching
customers. In my vision, /stable/ means:

- without bug that may corrupt customers data
- without blocking bug

Consequently, a proprietary software vendor needs to have a QA team.
This team assures the stability of releases. No right to mistakes
because financial penalties will be requested by customers.

OSS model is different because it *involves* customer on QA. This is a
very important point in my opinion. An OSS project must provide
efficient tools to communicate with core dev team. Customers/users are
willing to propose patches.

OSS model partially moves QA to customers. Not completely of course,
and I'll describe muy vision of QA management later. I'm sure Koha
developers have all read Eric Raymond essay "The Cathedral & the
Bazaar" [2] (if not, I encourage the reading). In this essay, you can
read the summarize of Linus's law : "Given enough eyeballs, all bugs
are shallow.". Koha team will never have enough eyeballs, this is why
customers must be involved in bug tracking, from notification to test &
validation.

Having worked nearly 3 years in a proprietary software company, I can
assure you that QA is one of the most critical task, and very often
underestimated since it costs a lot while not giving back money
directly. QA team needs to grow at the same speed as development team
output, and even more because complexity increases more than linearly.

OSS offers a solution to assure software quality without costing a lot
more. Indeed, the more customers, the more requested features, the more
tests required. If customers are involved in QA process, the number of
testers will increase the same as the number of features. My conclusion
is that in OSS, QA dedicated team don't need to be as big as
development team.

My explanations are very theoretical, in the reality OSS vendors have
to "release often, release early" [2] and involve their customer at an
early stage of deployment, not only before production deployment. This
is why HEAD builds, 3.0-RCx are necessary. I think some customers will
accept to play the role of beta tester. I ask Paul, Joshua, Russ and
other Koha services vendor to give their opinion.

My QA manager vision on Koha
============================

I come to my vision of QA manager on Koha project. The most important
of the QA manager is to know the content of the bug tracker:

1. being notified on every update
2. dispatch issue to developers (with developer authorization)
3. check that each issue is assigned/treated
4. reduce time between bug notification and its closure

It is important that no bug remain a too long time
unassigned/untreated. If this happens, customer won't feel useful to
involve herself. A bug rapidly corrected with a efficient communication
between reporter and developer will encourage customers to report bugs,
thus improving Koha quality in the long run.

An additionnal task of the QA manager is Bug Squashing Party (BSP)
coordination. I need to be explained how Koha worked on this in the
past.


I'm waiting for your feedback.

[1] http://tinyurl.com/n4234
[2] http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/

Cheers,

-- 
Pierrick LE GALL
INEO media system