[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Suggestion] GNUstep-test for quality control
From: |
Stefan Urbanek |
Subject: |
Re: [Suggestion] GNUstep-test for quality control |
Date: |
Thu, 16 Oct 2003 07:56:52 +0200 |
On 2003-10-15 22:58:38 +0200 Lars Sonchocky-Helldorf
<Lars.Sonchocky-Helldorf@bbdo-interone.de> wrote:
On 15.10.2003 20:58:37 Stefan Urbanek wrote:
<snip>
Simplier solution: What about lots of users as a 'testing framework and
quality
controll'? Advantage of 'user based testing framework' is that it tests
continualy. That 'framework' should also contain users, that were marked
as
'kind of users that gnustep does not need' in a gworskpace thread few
days ago
on this list. More users means more testers and it does not matter what
kind of
users. At this time and state, gnustep needs any kind of users. More
eyes see
more.
I think, that if at least half of the effort devoted to writing test
suites (i
am not against them) is also devoted to make gnustep more attractive, it
can
also help to remove bugs and have a quality controll.
That's a circular dependency: less bugs mean more users, more users mean less
bugs. But the flipside is also true: lots of bugs mean less users, less users
mean lots of bugs :-(. Currently the situation tends more to the latter...
Yes, and we need to get out of it.
The problem with a "unorganised user based testing" approach is that it needs
committed, disciplined users to work. That's something you can't take for
granted. My experience tells me instead that far most users simply turn away
if things don't work out of the box: You even can be thankful if about one
percent fire their mail client up and send you just complaints, the users
that are able to give you a qualified bug report you can count with the
fingers of both hands. Another result introduced by the nature of your
approach is that you'll get many reports for easy bugs you already know, but
virtually nobody is willing to invest a lot of time to track down a bug which
a.) occurs irregulary or b.) involves a lot of work. In the end it is like in
any chaotic multi agent system: the path of least resistance is chosen. The
already long history of GNUstep is another proof for this.
True, users go away if it does not work. So what? We should at least have the
first-user-experience parts bug-free, to let them go in and start play with the other
stuff. Like a trap :-) You need a trap, hook or attraction to get users. Is there any in
GNUstep? "The great framework desing with no alternative? Yes, but ... it does not
work (as expected), so what?" You get the point.
About the discipline and organisation: you need that and you do not. Point is, that
currently there is no need to fix bugs. Why? Because noone wants them to be fixed, or
there are just a few people that want them to be fixed. So why waste an energy? More
users means more pressure, and perhaps a chance that there is someone who will likely
want to fix it. This is something about that famous "it works on my machine, so why
bother?". True, why to bother, if there are just one or two users with this
behaviour and noone else complains?
You make a mistake when you think that the "effort devoted to writing test
suites" is basically lost time and done just for respectability. Let me tell
you: it is different. Any amount of time invested in a proper test pays off.
For a long time I also was a opponent of unit testing, I just saw no benefit
from it, I resisted doing it and left it to the other guys here at work. But
one day I *had* to do it and it came out to be much less complicated then I
had feared. In fact I realized that I was so reluctant towards it because it
was unknown to me. Today I have the habbit to test early and often.
Unit-testing just helps you to make rapid progress while coding. I can tell
you that I discovered lots of bugs, even 'thinkos' in my code doing testing
(often borderline cases where some special treatment for certain values was
needed) that would have gone undiscovered and would have caused bugs hard to
track down later on (if you build on a faulty foundation you building will
most likely be faulty to). I wouldn't start a project (even a private)
without unit testing today.
No, I haven't said that the effort devoted to writing test is lost time, just that at
least half of the time should be devoted to gaining more users. This is not a commercial
software for particular requestor where you already have a customer, this is free
software where "customers" are not certain. And if you write a SW that is
really perfect (according to tests), why is it good for if it is for noone just those who
developed it? Then the SW will die. I'm saying that testing by users can be complimentary
to testing with a testing framework. GNUstep needs both. Now a question is, what is
easier to get at this time?
Btw., to paraphrase your last question ... Would you start a project without
users (even potential) today? Are there going to be any future users with this
state and attitude of GNUstep?
Moreover, there are functionalities that can be tested only by users, so saying
that users are not good for testing is not a way to go.
To sum it up:
- user testing is complimentary to unit testing
- more users will result in greather pressure/need of bug fixing
- to get more users, there should be a trap/hook/attraction - is there any?
- without users, project dies with its developers
Have a nice day,
Stefan Urbanek
--
First they ignore you, then they laugh at you, then they fight you, then you
win.
- Mahatma Gandhi