Re: [h5md-user] Specifying the data type

[Pierre de Buyl]

Hi all,

On Thu, Aug 29, 2013 at 11:24:59PM -0400, Peter Colberg wrote:
> Hi Olaf, hi all!
> 
> Your proposal of explicitly specifying the data types of elements in
> the specification is a very good idea. Before we work out the details,
> I would like to add two minor remarks to the discussion, since there
> seems to be some confusion over HDF5’s derived data types.
> 
> * Array
> 
> A dataset element of any data type can only be read or written
> atomically. For the case of an Array data type, this means that the
> element itself cannot be sliced using a hyperslab selection. The
> ability to slice vectors is important for certain classes of parallel
> simulations, e.g., with OpenCL, where particle coordinates are stored
> in memory as 4-dimensional vectors for alignment reasons (e.g.,
> cl_double3 has a size of 4×8 bytes), while only 3 components are
> stored in the file.

Just to check: does it mean that we should stick to "regular" dataset? If so,
it's all good. I never intended to include "arrays" or understood that it was
anyone's intention.

> * Variable Length
> 
> A variable length data type allows each dataset element to have a
> variable size. This is not to be confused with a variable-length
> dataspace, as used for time-dependent datasets. An example for
> variable-length data types that implicitly appear in the specification
> are variable-length strings. As mentioned earlier, I would prefer to
> explicitly specify string attributes to be of variable length string
> data type, which eases handling in low-level languages.

Variable length is part of the type specification so it should be made explicit.
I propose that variable length data is prefixed by VL.

The case of scalar dataspaces can be specified by "[]".

Regarding the case issue, the HDF5 spec
http://www.hdfgroup.org/HDF5/doc/UG/11_Datatypes.html goes by "Integer", "Float"
or "Character" in Table 1. Only the named constant "H5T_*" are capitalized so we
may use what we prefer :-)

\-- <data_group>
     \-- step: Integer[variable]
     \-- time: Float[variable]
     \-- value: Float[variable][...]
          +-- unit: VL String[]
\-- <data_item>: VL String[]

Pierre