Re: Test Failure

[Ben Pfaff]

Jason Stover <address@hidden> writes:

> On Mon, Apr 17, 2006 at 10:21:12AM -0700, Ben Pfaff wrote:
>> The catch is that this isn't what chisq is actually computing.
>> It is actually printing pdf.chisq(idf.chisq(0.01, 2), 2).
>> Here is code that illustrates that[+]:
>
> Pardon me if this is too far off topic, but the fact that pspp
> evaluates pdf.chisq(idf.chisq(0.01, 2), 2), and the fact that it
> delays writing data to the active file has me wondering: Does pspp try
> to delay any kind of evaluation until the 'end'? I have heard that
> gurus view delayed evaluation (and the consequent fewer assignments)
> favorably. Is that part of its design? (If my question makes no sense,
> do not hesitate to say so.)

The question does makes sense.  PSPP does do this in at least one
sense: there is an "optimizer" for expressions that tries to make
expressions less expensive.  Any expression with constant
operands will be evaluated only once (except for random
functions), x**2 will be rewritten as x*x, and so on.

But PSPP doesn't currently do any inter-command optimization like
the kind you envision.  There is plenty of leeway to do so; for
example, SELECT IF can often moved earlier in the transformation
stream.

>> So the real question may be, how should we adjust the chi-square
>> test to avoid this edge case?  Perhaps we should just pick a
>> different initial value of P that does not come so close to a
>> rounding error.
>
> I kind of like having the 'edge' cases in the test, partly
> to show where the computation depends on architecture. Would
> it be possible to leave those cases in the tests, but without causing
> a test failure? Perhaps if the user wants to see the differences,
> they could type 'make paranoid-check' or something.

I'd be willing to do that if we can come up with a heuristic to
figure out how much variance we allow before complaining.
Perhaps any change +/- 0.01 is allowable?
-- 
Ben Pfaff 
email: address@hidden
web: http://benpfaff.org