[Octave-bug-tracker] [bug #60682] betainc is inaccurate

[Michael Leitner]

URL:
  <https://savannah.gnu.org/bugs/?60682>

                 Summary: betainc is inaccurate
                 Project: GNU Octave
            Submitted by: mleitner
            Submitted on: Fri 28 May 2021 08:51:45 AM UTC
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Inaccurate Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: 6.1.0
         Discussion Lock: Any
        Operating System: GNU/Linux

    _______________________________________________________

Details:

For the second parameter b equal to one, betainc(x,a,1) should be equal to
x^a. Double precision corresponds to eps(1)=2.2204e-16, so the third column of
the output when doing


format long e
x=2.^(-(0:20))';[x x.^2 betainc(x,2,1)./x.^2-1]


should not be much larger than this value. Note that for x equal 2 to the
power of an integer and also a an integer, x^a is exactly representable, so it
could also be zero. However, in this display of not so extreme values I see
deviations of up to 14 eps, namely at x=2^-18, and if you go to smaller x or
larger a, you get even larger deviations. 

Further, if you do


format long e
x=2.^-18*(1+eps(1)*(0:9)');[x x.^2 betainc(x,2,1) betainc(x,2,1)./x.^2-1]


you see that indeed the second column is continuously growing, but the third
shows only a single jump.

All of the numbers I report here were obtained on /octave-online.net.

I did not look into the code, but betainc will be computed by some series
expansion or continued fraction, and it seems that the aborting condition of
this computation is a bit too tight.





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?60682>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/