Re: Loading in hdf5 format skips every other variable.

[Steven G. Johnson]

Ah, I think I've found the problem.  The release notes for HDF5 1.2.2 contain 
the item:

   * Fixed error in H5Giterate which was not updating the "index" parameter
     correctly.

I had noticed exactly this problem, but assumed it was the correct behavior and
so I incremented the index manually (H5Giterate is the function to step through
the objects in an HDF5 file).  However, I sent a note to the HDF5 developers
complaining about the documentation for this function (which seemed to imply
that the index would be incremented for me), and apparently they decided it was
a bug.

So, in the new version, H5Giterate increments the index itself, which means I am
incrementing it a second time and items are skipped.  (Sigh...I wish that they
would make a more prominent notice when they change function semantics.)

The patch below should fix the problem.  It detects at runtime whether we are
linked to a version of HDF5 with the bug (by looking at the version number),
and uses that to determine whether or not to update the index manually.
Can you try it to make sure it works?  (I don't have the new version installed
yet, because I need to fix other software on my system first.)

Steven

PS. You don't need to specify "load -hdf5". "load" should detect the file type.

Index: load-save.cc
===================================================================
RCS file: /cvs/octave/src/load-save.cc,v
retrieving revision 1.124
diff -u -r1.124 load-save.cc
--- load-save.cc        2000/07/03 01:23:02     1.124
+++ load-save.cc        2000/07/07 21:57:12
@@ -1307,6 +1307,11 @@
   bool import;
 };
 
+// This variable, set in read_hdf5_data(), tells whether we are using
+// a version of HDF5 with a buggy H5Giterate (i.e. which neglects to
+// increment the index parameter to the next unread item).
+static bool have_h5giterate_bug = false;
+
 // This function is designed to be passed to H5Giterate, which calls it
 // on each data item in an HDF5 file.  For the item whose name is NAME in
 // the group GROUP_ID, this function sets dv->tc to an Octave representation
@@ -1688,7 +1693,8 @@
          if (dsub.doc)
            delete [] dsub.doc;
 
-         current_item++;  // H5Giterate returned the last index processed
+         if (have_h5giterate_bug)
+           current_item++;  // H5Giterate returned the last index processed
        }
 
       if (retval2 < 0)
@@ -1799,12 +1805,20 @@
   d.range_type = hdf5_make_range_type (H5T_NATIVE_DOUBLE);
   d.import = import;
 
+  // Versions of HDF5 prior to 1.2.2 had a bug in H5Giterate where it
+  // would return the index of the last item processed instead of the
+  // next item to be processed, forcing us to increment the index manually.
+  unsigned int vers_major, vers_minor, vers_release;
+  H5get_libversion(&vers_major, &vers_minor, &vers_release);
+  have_h5giterate_bug = vers_major <= 1 && vers_minor <= 2 && vers_release < 2;
+
   herr_t retval = H5Giterate (hs.file_id, "/", &hs.current_item,
                              hdf5_read_next_data, &d);
 
-  // H5Giterate sets current_item to the last item processed; we want
-  // the index of the next item (for the next call to read_hdf5_data)
-  hs.current_item++;
+  if (have_h5giterate_bug)
+    // H5Giterate sets current_item to the last item processed; we want
+    // the index of the next item (for the next call to read_hdf5_data)
+    hs.current_item++;
 
   if (retval > 0)
     {