[Chicken-hackers] Re: [Chicken-users] Integrating unit tests into source code

[Brandon J. Van Every]

Peter Bex wrote:

On Thu, Dec 14, 2006 at 12:13:19PM +0100, Peter Busser wrote:
  

Hi!

I have no ideas for the exact mechanism or the syntax. But I like the
idea of having test cases embedded in the code. That makes proper
maintenance of the test cases more likely IMHO.
    

You're not worried about the proliferation of huge test cases which
clutter up the code?  You'd have to go looking for the thing that's most
important: the actual code.  If you're trying to find something quickly
and you're not familiar with the code it's much easier if you have a
few functions on your screen.  If you include all the tests, it's
very likely one function will fill up an entire screen so you'll have
to keep scrolling all over the place.
  

Non-issue.  Understanding anyone else's goddamn code requires viewing lotsa functions, and hence lotsa scrolling.  What you need are COMMENTS and search tools.  If there aren't good comments, or rather straightforward code structure for the problem at hand, well you're just SOL.  You'll have to scratch your head and figure it out.  For search tools, I strongly prefer class browsers when working with C++ code.  I still haven't got such a capability for Scheme code.  Started up the Etags learning curve at one point, but like everything else in Emacs-land, quickly tired of it.  Went back to improving the CMake build.  I get by with grepping everything.

To me, tests are a completely different beast than the algorithms that
make up the actual program.  You can leave out the tests and the program
won't stop working.

Ahem.  If it *is* working, which you don't really know unless you keep running the tests all the time.  Just having some user poking about day-to-day isn't testing.  I worked a game testing gig recently, the bottom rung of the game industry at $10/hr.  We sat around thinking up really twisted shit to throw at the UIs.  All we really had to do was think a little more broadly than a programmer.  Not hard to find bugs in most people's code.

Why do people do QA at all?  Well, they're trying to prevent some customer from screaming at their software, and losing that customer.  They're hoping the QA guy finds the butt stinker of a problem first.  Now, how does this apply to open source?  Or even more specifically, open source programmer tools?  At least in the latter case, you stand a chance of a programmer actually bothering to report a bug.  Maybe even attempt to fix the bug.  But there is a general labor problem.  There are way more bugs than people have time or willingness to deal with.  Implementing a bug tracker will improve Chicken's QA, but I don't expect miracles.

  Some tests need additional code for support (setting
things up in a particular way etc).  Why would one want to put all that
code (which may be twice as big as the code itself, or even bigger, if done
properly) in one huge file?
  

You do it if you want your code to actually work, and you think it's better to run the automated tests than to pay someone $XX/hour after-the-fact to show you your bugs.

If you're writing one-shot code, or you don't care much about other users or programmers, then you may rightly see all the tests as just bloat in the way.  Whether they're part of the code or separate.

There are political issues here.  The strong split between "developers" and "QA," particularly in terms of responsibilities and payscale, is likely due to the paradigm of code-test separation you espouse.  Or to put it another way, most developers throw their problems over the fence to someone else.


Cheers,
Brandon Van Every