[Octave-bug-tracker] [bug #53644] Possible Octave or Nouveau image plot bug (not sure which) discovered via ASAN

[Dan Sebald]

Follow-up Comment #2, bug #53644 (project octave):

Not sure this is specific to images.  Nonetheless, I'm writing notes here to
keep track of things.

It seems that the ASAN consistently exits at "spo 4" in the following location
in gl-render.cc:


  void
  opengl_renderer::set_polygon_offset (bool on, float offset)
  {
    if (on)
      {
std::cerr << "spo 1\n";
        glEnable (GL_POLYGON_OFFSET_FILL);
std::cerr << "spo 2\n";
        glEnable (GL_POLYGON_OFFSET_LINE);
std::cerr << "spo 3\n";
        glPolygonOffset (offset, offset);
      }
    else
      {
std::cerr << "spo 4\n";
        glDisable (GL_POLYGON_OFFSET_FILL);
std::cerr << "spo 5\n";
        glDisable (GL_POLYGON_OFFSET_LINE);
      }
std::cerr << "spo 6\n";
  }


The image() routine causes the above hunk of code to run three times (in
pairs, i.e., the "on" is true, some drawing, then "on" is false) and on the
third time that "glDisable (GL_POLYGON_OFFSET_FILL);" is where ASAN complains
and exits.

That routine glDisable() is quite simple, it's basically disabling the polygon
features or drawing a filled surface and an outlining or border.  So I can't
imagine that routine is the actual problem, just where ASAN recognizes the
problem.  Also, glPolygonOffset() is pretty innocuous, as it controls a little
z-depth offset to bring the lines nearer and prevent the stitching
phenomenon.

So, as is typical with these ASAN like programs, we look prior to the
occurrence of the dump.  I worked backward and put pre-process conditional
around the following hunk of code


  opengl_renderer::draw_axes_planes (const axes::properties& props)
  {
[snip]
#if 0
    if (! is2d)
      {
        // X plane
        glVertex3d (xPlane, yPlaneN, zPlaneN);
        glVertex3d (xPlane, yPlane, zPlaneN);
        glVertex3d (xPlane, yPlane, zPlane);
        glVertex3d (xPlane, yPlaneN, zPlane);

        // Y plane
        glVertex3d (xPlaneN, yPlane, zPlaneN);
        glVertex3d (xPlane, yPlane, zPlaneN);
        glVertex3d (xPlane, yPlane, zPlane);
        glVertex3d (xPlaneN, yPlane, zPlane);
      }

    // Z plane
    glVertex3d (xPlaneN, yPlaneN, zPlane);
    glVertex3d (xPlane, yPlaneN, zPlane);
    glVertex3d (xPlane, yPlane, zPlane);
    glVertex3d (xPlaneN, yPlane, zPlane);
#endif
[snip]


and the image() routine works fine, can be repeated, have different plotting
methods in-between, etc.  So, the next step is to look for anything peculiar
about the memory buffer for the drawing of the axes in this code hunk.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53644>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/