Re: [Koha-devel] QA manager

[Joshua Ferraro]

Hi Pierrick,

Sorry so late in getting back to you on this ... here we go:

On Tue, Mar 21, 2006 at 02:32:51PM +0100, Pierrick LE GALL wrote:
> On the last Koha team meeting of March 20th, 2006, we discussed about
> the position INEO (me for instance) could officialy take on the
> project. Joshua and Paul proposed me to be Quality Assurance (QA)
> manager, and I personnaly accepted this role for a two months try. I've
> just asked my INEO managers for a formal validation.
Have you head back? Can we announce it as official?

> On IRC, the definition about QA manager was discussed, because
> everybody doesn't define it the same way. Paul said "responsible for
> heavy testing" and I answered I did not agree with this point of view,
> not in OSS model. thd asked what difference I made between open and
> closed source QA. Let's answer this question by mail, as requested by
> Russ. See details on IRC logs [1].
Right ... see comments below ...

> I first answer to the question "what difference I make between open and
> closed source QA", then I'll describe what is my vision of QA manager
> for an OSS project like Koha.
> 
> QA, proprietay Vs OSS
> =====================
> 
> I've worked on the two sides of softwares : proprietary and OSS. I
> think I understand the advantages and drawbacks of each model,
> concerning QA.
> 
> In a proprietary model, the software *must* be /stable/ when reaching
> customers. In my vision, /stable/ means:
> 
> - without bug that may corrupt customers data
> - without blocking bug
> 
> Consequently, a proprietary software vendor needs to have a QA team.
> This team assures the stability of releases. No right to mistakes
> because financial penalties will be requested by customers.
> 
> OSS model is different because it *involves* customer on QA. This is a
> very important point in my opinion. An OSS project must provide
> efficient tools to communicate with core dev team. Customers/users are
> willing to propose patches.
> 
> OSS model partially moves QA to customers. Not completely of course,
> and I'll describe muy vision of QA management later. I'm sure Koha
> developers have all read Eric Raymond essay "The Cathedral & the
> Bazaar" [2] (if not, I encourage the reading). In this essay, you can
> read the summarize of Linus's law : "Given enough eyeballs, all bugs
> are shallow.". Koha team will never have enough eyeballs, this is why
> customers must be involved in bug tracking, from notification to test &
> validation.
> 
> Having worked nearly 3 years in a proprietary software company, I can
> assure you that QA is one of the most critical task, and very often
> underestimated since it costs a lot while not giving back money
> directly. QA team needs to grow at the same speed as development team
> output, and even more because complexity increases more than linearly.
> 
> OSS offers a solution to assure software quality without costing a lot
> more. Indeed, the more customers, the more requested features, the more
> tests required. If customers are involved in QA process, the number of
> testers will increase the same as the number of features. My conclusion
> is that in OSS, QA dedicated team don't need to be as big as
> development team.
>
> My explanations are very theoretical, in the reality OSS vendors have
> to "release often, release early" [2] and involve their customer at an
> early stage of deployment, not only before production deployment. This
> is why HEAD builds, 3.0-RCx are necessary. I think some customers will
> accept to play the role of beta tester. I ask Paul, Joshua, Russ and
> other Koha services vendor to give their opinion.
I think in general you're right, our customers often play the role of
beta testers for the specific modules that we are developing for them.
However, I can't tell you how many times a developer has written 
something that works perfectly for _their_ client, but it also breaks
some function that someone else's client needs (for instance, budget-
based acquisitions have been broken no less than 3 times).

So, what this means is that the customer can't be the only beta tester.
We need someone to take a broad overview and make sure _all_ functions
are working before a release.

Paul, Chris, Russ, anyone have any further comments on this?

> My QA manager vision on Koha
> ============================
> 
> I come to my vision of QA manager on Koha project. The most important
> of the QA manager is to know the content of the bug tracker:
> 
> 1. being notified on every update
> 2. dispatch issue to developers (with developer authorization)
> 3. check that each issue is assigned/treated
> 4. reduce time between bug notification and its closure
> 
> It is important that no bug remain a too long time
> unassigned/untreated. If this happens, customer won't feel useful to
> involve herself. A bug rapidly corrected with a efficient communication
> between reporter and developer will encourage customers to report bugs,
> thus improving Koha quality in the long run.
100% agreed.

> An additionnal task of the QA manager is Bug Squashing Party (BSP)
> coordination. I need to be explained how Koha worked on this in the
> past.
Also a great idea. I was holding weekly BSP's a while back but have
gotten out of the habit of scheduling them -- feel free to start this
asap...it might also help us to get back to using bugzilla again.

Cheers,

-- 
Joshua Ferraro               VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE
President, Technology       migration, training, maintenance, support
LibLime                                Featuring Koha Open-Source ILS
address@hidden |Full Demos at http://liblime.com/koha |1(888)KohaILS