[Octave-bug-tracker] [bug #49961] sign_test() returns incorrect p-value

[Kirill Pushkaryov]

URL:
  <http://savannah.gnu.org/bugs/?49961>

                 Summary: sign_test() returns incorrect p-value
                 Project: GNU Octave
            Submitted by: kpushkaryov
            Submitted on: Sat 31 Dec 2016 05:53:38 AM GMT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 4.0.0
        Operating System: GNU/Linux

    _______________________________________________________

Details:

The function sign_test() from core Octave (ver. 4.0.0) returns a seemingly
incorrect p-value.
In Octave command window:

>> x = [1 1 1 -1 -1]
x =

   1   1   1  -1  -1

>> [pval, b, n] = sign_test(x, -x)
pval =  0.37500
b =  3
n =  5


At the same time, signtest() function from the statistics package (ver. 1.2.4)
gives a different result:

>> pval2 = signtest(x, -x)
pval2 =  1.00000


SciPy's scipy.stats.binom_test(3, 5) returns 1.0 too.

I inspected the source code of sign_test() and believe that it improperly
handles the boundary points. In sign_test.m we have:

  n   = sum (x != y);
  b   = sum (x > y);
  cdf = binocdf (b, n, 1/2);
...
  if (strcmp (alt, "!=") || strcmp (alt, "<>"))
    pval = 2 * min (cdf, 1 - cdf);
  elseif (strcmp (alt, ">"))
    pval = 1 - cdf;
  elseif (strcmp (alt, "<"))
    pval = cdf;
  else
    error ("sign_test: option %s not recognized", alt);
  endif

By default alt = "!=". So, mathematically cdf = F(b) = P(X <= b) and 1 - cdf =
P(X > b), where X has binomial distribution with parameters n, 0.5. Depending
on whether cdf < 1 - cdf (i.e. cdf < 0.5) or cdf > 0.5 we have pval = 2 * P(X
<= b) or pval = 2 * P(X > b). The first case correctly includes X = b, but the
second does not. A similar problem will occur with alt = ">".

sign_test.m in Octave 4.2.0 contains the same code.



    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Sat 31 Dec 2016 05:53:38 AM GMT  Name: sign_test_bug.m  Size: 203B   By:
kpushkaryov
Demonstration code
<http://savannah.gnu.org/bugs/download.php?file_id=39335>

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?49961>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/