Re: [h5md-user] Strings again / parallel issue

[Peter Colberg]

On Tue, Feb 04, 2014 at 06:19:20PM +0100, Pierre de Buyl wrote:
> On Tue, Feb 04, 2014 at 10:38:55AM -0500, Peter Colberg wrote:
> > Maybe you could use an enumeration for the datasets in question?
> > 
> > The variable-length string type is currently used with attributes,
> > which cannot be written in parallel. So I would not expect any
> > problem for the H5MD specification.
> 
> Well, the code that I attach (for real now) writes an attribute in parallel. 
> The
> operation is collective (as it must).

>   !
>   ! Create property list for collective dataset write
>   !
>   CALL h5pcreate_f(H5P_DATASET_XFER_F, plist_id, error) 
>   CALL h5pset_dxpl_mpio_f(plist_id, H5FD_MPIO_COLLECTIVE_F, error)

>   call h5acreate_f(dset_id, 'author', filetype, aspace_id, attr_id, error)
>   if (mpi_rank.eq.0) then
>      call h5awrite_f(attr_id, filetype, author, adims, error)
>   end if

Here the attribute is written by the process on rank 0.

Collective reading and writing is only possible for datasets.

>   call H5Sselect_hyperslab_f(filespace, H5S_SELECT_SET_F, offset, count, 
> error)
>   !
>   ! Write the dataset collectively. 
>   !
>   CALL h5dwrite_f(dset_id, H5T_NATIVE_INTEGER, data, dimsfi, error, &
>        mem_space_id=memspace, file_space_id=filespace, xfer_prp = plist_id)

The above call uses collective I/O, but it is for a dataset.

So far there is no conflict with the current version of H5MD, which
mandates the vlen string type only for the specified attributes.

For large datasets (scaling with the number of particles) of string
type that we specify in a later version, fixed-length strings can of
course be used. But again, if the string values form a small set,
an enumeration data type is more efficient.

> > Could you describe the data layout of your topology?
> 
> In another thread if you wish :-)

The description certainly belongs to this thread.

How is the data layed out, in terms of data space and data type?

Peter