Re: [h5md-user] Make "time" optional?

[Felix Höfling]

Am 14.07.2014, 11:33 Uhr, schrieb Pierre de Buyl <address@hidden>:

> > Am 11.07.2014, 17:51 Uhr, schrieb Olaf Lenz  
> > <address@hidden>:
> >
> > >...and adding to that, can we also make the "step" optional? Weird
> > >as this
> > >may sound, we would also have to invent step numbers.
> > >
> > >Olaf
> > >
> >
> > Dear Konrad and Olaf,
> >
> > We have had some discussion on the time dataset in the context of Monte
> > Carlo simulations. If I remember well the outcome was that in the case  
> > of
> > no physical time, time is simply identical (=linked) to step. Our
> > intentation was to have no optional parts in the core H5MD element---for
> > the sake of making reading simple.
> >
> > Whether such a decision was wise or not, I don't know. But it has been
> > fixed now for H5MD 1.x. Making step or time optional would break
> > compatibility with 1.0 and would make 1.0 basically obsolete. Thus I  
> > don't
> > think it is a good idea. Nevertheless, I am open to extend the
> > interpretation of step/time (but the fields must be present). For  
> > example,
> > step could also just numerate the snapshots stored, without reference to
> > any simulation order.
>
> Actually, allowing time to be missing we could make the reading software  
> default
> to use time instead. Would that break anything?

Maybe my conclusion that any changes w.r.t. step/time would break  
backwards compatibility was premature. I still have to get used to what  
kind of changes are allowed and which are not ...

A reader for 1.1 could be advised to proceed as follows:

  * missing "time" => use "step"

  * missing "step" => use array index, i.e., step = (0, 1, 2, … ) by  
default

Reading files of version 1.0 is unchanged. A reader expecting 1.0, of  
course, will break if step/time is missing. For the writer, it is even  
simpler: writing an optional field doesn't break anything. But omitting  
the field requires that H5MD version 1.1 is stored.

> > Note that for a single snapshot only, the data are stored in a simple  
> > HDF5
> > dataset without step/time.
>
> In a more general idea about step/time, I have an idea since a long  
> time. I
> didn't want it for H5MD 1.0 to avoid any confusion. But storing step and  
> time
> when step is simply step[i] = STEP_SIZE*i and time[i] = STEP_SIZE*DT*i  
> is a bit
> of a waste. We could define a proper setup for regularly sampled data,  
> for which
> step[0], STEP_SIZE, time[0] and DT should be given.
>
> P

This is a very reasonable extension. So far we relied on data compression  
for such simple (and prevalent cases), which is not a good solution.

I prefer to keep step and time separate. Thus it is possible to specify,  
e.g., time as a list and step by an offset/increment pair. (Of course,  
STEP_SIZE*DT is replaced by DT in your example.) The question is how to  
indicate the different scheme in the file? (two attributes step_offset,  
step_increment, one attribute "step" with a pair, a dataset step[0] with  
an attribute "increment", ...)

Shall we also support piecewise linearly spaced grids? (0, 1, 2, 3, 4, 5,  
10, 15, 20, 25, 30, 35, 50, 65 ...)

Felix