Re: [Chicken-users] Nursery sizing considered stupid

[Brandon J. Van Every]

Brandon J. Van Every wrote:
> So, I'm bumping the samples to 100, and implementing a low-high 
> variance test.  The low-high variance test won't take any more time, 
> and will demonstrate how random the sampling actually is.  100 samples 
> is kinda slow on my box.  I was hoping to make due with fewer samples 
> and have faster build times.  But this apparently won't work in the 
> general case.  If 100 samples isn't good enough to create testing 
> stability for machines in general, then nsample is junk.  If proven to 
> be junk, then we will need to either improve it or scrap it.

Implemented and pushed to Darcs.  The low-high reporting has revealed a 
further flaw in nsample testing during the build.  The benchmarks are 
highly sensitive to system load.  I've seen large variances when I've 
switched tasks to check e-mail or browse web stuff.  This example is way 
above the usual average of ~390 ms on my machine:

-- Testing the default stack size
-- stack size 131072 bytes ->
-- lowest  390.0 ms
-- average 506.620 ms
-- highest 1702.0 ms

A user has no reason to expect that the build is going to be sensitive 
to the machine load at the time of the build.  If he starts a build, 
then does the usual sorts of multitasking things people do when waiting 
for a long, boring build to complete, he's going to get complete crap 
results from nsample.

I think for this reason, nsample must not be part of the standard build 
procedure.  Rather, it can be an available make target, and generate a 
stack-size.h file for a user who knows what he's doing.  Warning 
messages aplenty should be spewed when a person types "make stack-size," 
informing him how much things are going to suck if he runs off to check 
his e-mail.  Since the absence of such a policy leads to insanity, I 
will be implementing it in the CMake build shortly.  That is how it will 
remain unless someone is more clever than I am.


Cheers,
Brandon Van Every