On Tue, Oct 07, 2014 at 09:35:57AM +0200, address@hidden wrote:
Before going further with "particle_number", did you test your
situation with a
compressed "id" element?
No, the particles are indistiguishable, so there is not id. The
compression should reduce the size dramatically; for a 64-bit “step”
dataset and a chunk size of 32 kB the compression ratio can be >100.
Alternatively, a better compression could be achieved if we decided to
allow
"species" to contain also a fill value. This way, you'd compress a
dataset of
fill_value, 0, 1 (for two species, for instance) and that should be more
efficient.
That is a good idea. We should add this as a minimal solution.
In any case, I am not concerned with the efficiency of storage, but
with the convenience of reading the particle data. It would be nice if
one would not have to loop over species/id to find non-empty regions.
Instead one reads the particle number, checks that both species and id
(if present) do not define a fill value (= no holes), and then selects
a single hyperslab for reading a particle dataset.
Peter