[Octave-bug-tracker] [bug #53199] hist returns different results depending on 2nd 1:n/n input argument

[Michael Leitner]

Follow-up Comment #9, bug #53199 (project octave):

@Rik: You said that the error is 8 eps, which indeed would be larger than
expected and larger than necessary, but my point was that for floating-point
numbers the discretization step grows with the magnitude of the number. In
this case the number with the deviation is 16 (actually 15.9999...), and the
deviation is 0.5 eps(16), that is, 1 eps(15.9999...), which is the best
possible accuracy after the ideal result.

I did not know that Octave now can do scalar operations on ranges (sorry,
Debian Stable here with Octave 4.0.3), but see it in the recent bug reports.
Seemingly 4.2.1 on https://octave-online.net/ also cannot do it yet. Yes, of
course it is sensible to do these calculations on ranges. But first I would
not call the problem of the reporter a bug, and I would not be sure that with
the range solution it is equivalent to Matlab in every possible situation
without trying a very large number of possibilities. For instance, consider


[y,x]=hist(20*[rand(10,1);0;1],6)


You get a bin size of 3.333..., and you would ideally have a bin division at
exactly 10. However, you generate a range of half-integers (that is, with an
increment of 1 and a starting value of 0.5, both of which can be exactly
represented), but then multiply with 3.333..., which cannot be exactly
represented. Perhaps here the result differs depending on whether you do this
multiplication on the range or on the corresponding vector (definitely there
will be equivalent examples where they differ), and I could not say that
Matlab will always give the result that corresponds to the operation on the
range, because, as far as I know, Matlab does not have a range type. So the
patch is fine, but I would not claim that it fixes a Matlab incompatibility in
the general case. 

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53199>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/