Re: [h5md-user] subsystems with varying number of particles

[Felix Höfling]

Am 12.12.2013, 08:56 Uhr, schrieb Konrad Hinsen
<address@hidden>:

> Felix Höfling writes:
>
>  > In commit a3ee98a, the fill value of the particle ID has been given a
>  > meaning which I find very useful.
>
> Where can I find those commits, i.e. where is the repository backing
> the Web site?
It's on Savannah, try this one: http://git.savannah.gnu.org/cgit/h5md.git/

For contributing, a registration as project member is probably necessary.

>  > Just one question: what if there is no fill value defined? Does
>  > H5Pget_fill_value() return the default value? I think it is 0 for  
> Integer
>  > types, which would conflict with regular particle IDs starting at 0.
>
> The default fill value is whatever an all-zero bit pattern means for
> the type of a given dataset. That's what H5Pget_fill_value() returns
> if H5Pset_fill_value() has never been called for a dataset.
>
>  > Is this an issue that needs to be addressed somehow?
>
> If setting a fill value is a requirement for H5MD conformity, then
> this should of course be in the specification.
>
> Konrad.

Since the default fill value and the regular ID 0 are conflicting, the
current spec is ambiguous and needs to be fixed. I suggest one of the
following:

i) an attribute attached to the "id" element stating that placeholders are
used and what the fill value is

ii) query the HDF5 library whether the fill value has been set by the user
(it is not the default, see below), and only then placeholders are assumed

iii) mandate undefining the fill value (see below) for the "id" dataset or
"id/value" if no placeholders are used. Getting the fill value must (in
any case) be preceded by a call to H5Pfill_value_defined().

iv) postpone placeholders after 1.0, their introduction could be achieved
in a forward-compatible way.

H5Pfill_value_defined() gives us the possibility to query whether a fill
value has been defined by the user, whether the default is used, or
whether it is undefined. Undefining the fill value is achieved by passing
NULL to H5P_set_fill_value().

Let me comment on these choices:

i) is in the spirit of H5MD of being explicit about additional semantics,
the presence of placeholders is transparent to high-level APIs as well. No
need for H5P* functionality, which is just an implementation detail

ii) is the implicit solution, more HDF5-ish. Additional action is needed
reader-wise to interpret placeholders, a writer can't do wrong. A reader
ignoring the possibility of placeholders wouldn't do so badly either, but
find many non-unique IDs 0.

iii) the writer always has to do some action (either undefining or setting
the fill value). If it "forgets" undefining, a placeholder-aware reader
will find 1 placeholder (since ID 0 is unique) and will miss particle #0.
The reader won't break because of non-unique IDs.

iv) would give us time to cross-check, e.g., with visualisation tools. It
is not really a solution, but would allow us to release 1.0 without
further delays.

After some reflections, I'm in favour of i), the attribute solution

Please vote!

Felix