[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Precision of output

From: John Darrington
Subject: Precision of output
Date: Wed, 25 Jan 2006 10:22:27 +0800
User-agent: Mutt/1.5.9i

An issue which arose yesterday when I was trying to use pspp in my

It seems that many procedures output their results with a fixed
precision. I happened to need a greater precision than PSPP was

The relevant section of pspp code (which I wrote) uses tab_float with
a literal interger for its W and D parameters.  Perhaps it should have
used tab_value, and passed the variable's print member to  FMT_SPEC ;
the problem is, in that particular routine, the variable is out of
scope (however see below!).

A quick browse through the code of other procedures shows that there
are perhaps quite a lot of places where this problem has crept in.
Perhaps we should do an audit of these sometime --- although I'd have
to think hard about the general problem to know what the correct
precision and width should be an all cases.  For example, given a
bunch of integers, it might be necessary to know the mean value with
better than integer precision ...

Maybe it'd be better to modify tab_float so that the sentinel value
-1, displays values with variable precision ? ie using %g.


PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See or any PGP keyserver for public key.

Attachment: signature.asc
Description: Digital signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]