[Octave-patch-tracker] [patch #8119] Allow variable tolerance and improve error messages for assert.m script

[Dan Sebald]

URL:
  <http://savannah.gnu.org/patch/?8119>

                 Summary: Allow variable tolerance and improve error messages
for assert.m script
                 Project: GNU Octave
            Submitted by: sebald
            Submitted on: Wed 10 Jul 2013 09:02:05 AM GMT
                Category: None
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any

    _______________________________________________________

Details:

Only the comparisons which fall outside of tolerance are displayed.  The error
messages are formatted better and make sense in terms of what is out of
tolerance and by how much.

For that reason I've removed the FIXME about formatting at the top of the
script file.  Let's tweak the output format as desired and be done with that. 
I think it is pretty close as changed.

Instead of keeping track of error state via variable "iserror" all the way to
the end of the routine, it uses a subfunction for printing the error.  In
addition to a little more flexibility of output error message, this has the
added advantage of displaying the line number about where the tests fails. 
With the former approach, the line number was always somewhere at the end of
the file.

The varying tolerance comes in handy for testing function results for which
there is a wide numerical range--when a large tolerance value (say eps)
doesn't really test most of the routine and a small tolerance value (say
eps(min(x)) is so small that the vast majority of the other values will fail. 
To illustrate, here is a test for the recent changes to binocdf.m:


>> p = 1-1e-3;
>> pval = binopdf ([0:50]', 50, p);
>> assert (cumsum (pval), binocdf ([0:50]', 50, p), 1000 * eps (pval))


Notice that the comparisons are within tolerance for all values, and I've had
to relax the tolerance a bit because as we discovered with the discussions
about binocdf there are different numerical techniques at the heart of the two
routines binopdf and binocdf.

The following is what results on binocdf.m prior to the recent change:


>> assert (cumsum (pval), binocdf ([0:50]', 50, p), 1000 * eps (pval))
error: assert (cumsum (pval),binocdf ([0:50]', 50, p),1000 * eps (pval))
expected
   1.0000e-150
   4.9951e-146
   1.2226e-141
   1.9542e-137
   2.2940e-133
   2.1084e-129
   1.5798e-125
   9.9203e-122
   5.3270e-118
   2.4835e-114
   1.0172e-110
   3.6955e-107
   1.1999e-103
   3.5039e-100
    9.2515e-97
    2.2182e-93
    4.8477e-90
    9.6862e-87
    1.7741e-83
    2.9851e-80
    4.6225e-77
    6.5974e-74
    8.6884e-71
    1.0567e-67
    1.1877e-64
    1.2341e-61
    1.1855e-58
    1.0528e-55
    8.6403e-53
    6.5488e-50
    4.5801e-47
    2.9523e-44
    1.7514e-41
    9.5451e-39
    4.7686e-36
    2.1781e-33
    9.0685e-31
    3.4287e-28
    1.1721e-25
    3.6041e-23
    9.9053e-21
    2.4146e-18
    5.1720e-16
    9.6196e-14
    1.5303e-11
    2.0408e-09
    2.2198e-07
but got
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   0.0000e+00
   5.5511e-16
   9.6145e-14
   1.5303e-11
   2.0408e-09
   2.2198e-07
maximum absolute error 1e-150 exceeds tolerance 1.35666e-163
maximum absolute error 4.9951e-146 exceeds tolerance 8.89103e-159
maximum absolute error 1.2226e-141 exceeds tolerance 1.45671e-154
maximum absolute error 1.95425e-137 exceeds tolerance 2.38667e-150
maximum absolute error 2.294e-133 exceeds tolerance 3.91032e-146
maximum absolute error 2.10842e-129 exceeds tolerance 3.20333e-142
maximum absolute error 1.57977e-125 exceeds tolerance 2.62417e-138
maximum absolute error 9.92031e-122 exceeds tolerance 2.14972e-134
maximum absolute error 5.32698e-118 exceeds tolerance 8.80525e-131
maximum absolute error 2.48351e-114 exceeds tolerance 3.60663e-127
maximum absolute error 1.01725e-110 exceeds tolerance 1.47728e-123
maximum absolute error 3.69551e-107 exceeds tolerance 6.05092e-120
maximum absolute error 1.19988e-103 exceeds tolerance 2.47846e-116
maximum absolute error 3.50395e-100 exceeds tolerance 5.07588e-113
maximum absolute error 9.25151e-97 exceeds tolerance 1.03954e-109
maximum absolute error 2.21823e-93 exceeds tolerance 4.25796e-106
maximum absolute error 4.84772e-90 exceeds tolerance 8.7203e-103
maximum absolute error 9.68615e-87 exceeds tolerance 1.78592e-99
maximum absolute error 1.7741e-83 exceeds tolerance 3.65756e-96
maximum absolute error 2.98511e-80 exceeds tolerance 3.74534e-93
maximum absolute error 4.62254e-77 exceeds tolerance 7.67046e-90
maximum absolute error 6.59738e-74 exceeds tolerance 7.85455e-87
maximum absolute error 8.68837e-71 exceeds tolerance 1.60861e-83
maximum absolute error 1.05672e-67 exceeds tolerance 1.64722e-80
maximum absolute error 1.1877e-64 exceeds tolerance 1.68675e-77
maximum absolute error 1.23407e-61 exceeds tolerance 1.72723e-74
maximum absolute error 1.18551e-58 exceeds tolerance 1.76869e-71
maximum absolute error 1.05282e-55 exceeds tolerance 1.81114e-68
maximum absolute error 8.64034e-53 exceeds tolerance 1.8546e-65
maximum absolute error 6.54884e-50 exceeds tolerance 9.49557e-63
maximum absolute error 4.58011e-47 exceeds tolerance 9.72346e-60
maximum absolute error 2.95232e-44 exceeds tolerance 4.97841e-57
maximum absolute error 1.75142e-41 exceeds tolerance 2.54895e-54
maximum absolute error 9.54508e-39 exceeds tolerance 1.30506e-51
maximum absolute error 4.76856e-36 exceeds tolerance 6.68191e-49
maximum absolute error 2.17814e-33 exceeds tolerance 3.42114e-46
maximum absolute error 9.06846e-31 exceeds tolerance 1.75162e-43
maximum absolute error 3.42871e-28 exceeds tolerance 4.48416e-41
maximum absolute error 1.17214e-25 exceeds tolerance 2.29589e-38
maximum absolute error 3.60415e-23 exceeds tolerance 5.87747e-36
maximum absolute error 9.90534e-21 exceeds tolerance 1.50463e-33
maximum absolute error 2.41464e-18 exceeds tolerance 3.85186e-31
maximum absolute error 3.7911e-17 exceeds tolerance 9.86076e-29
maximum absolute error 5.02326e-17 exceeds tolerance 1.26218e-26
maximum absolute error 7.983e-18 exceeds tolerance 3.23117e-24
maximum absolute error 5.36716e-17 exceeds tolerance 4.1359e-22
maximum absolute error 1.89715e-17 exceeds tolerance 2.64698e-20


Notice that only the comparisons that failed are shown so that one doesn't
have to weed through all the values.  Also notice that the comments about the
error and tolerance now make sense, and furthermore the tolerance nicely
scales relative to the magnitude of the expected item.  In this case, Rik knew
about the string of zeros at the front.  However, I think this sort of setup
would be a good test for the plethora of other routines using 1 - betainc().




    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Wed 10 Jul 2013 09:02:05 AM GMT  Name:
octave-assert_vector_tolerance-2013jul10.patch  Size: 9kB   By: sebald

<http://savannah.gnu.org/patch/download.php?file_id=28527>

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/patch/?8119>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/