Re: [Openexr-devel] Huge Files & Tiles

[Florian Kainz]

Hi Gordon,

the minimum frame buffer size required to read a tiled OpenEXR file
is one tile.  You a use larger frame buffer, but this is not
required.  Please see the second example, function writeRgbaONE2(),
in section 5.1 of our "Reading and Writing OpenEXR ..." document.
(http://www.openexr.com/ReadingAndWritingImageFiles.pdf)

With tiled files, you could read and write individual columns if you
made the tiles one pixel wide and as high as the entire image, but I
guess that's not what you really want.

Tiled OpenEXR files support random access to tiles.  When you write
a file, you can write the tiles in any order (but each tile must be
written exactly once).  When you read a file, you can read the tiles
in any order; and individual tiles can be skipped or read more than
once.

The IlmImf library does not currently support updating an OpenEXR
in-place.  The file format would allow in-place updating of uncompressed
files, but this is not implemented.  Updating an OpenEXR image requires
copying the file.  (You can copy the file one tile at a time; you don't
have to keep the entire image in main memory.)

Florian



Gordon Wetzstein wrote:
> Dear OpenEXR developers,
>
> I am using OpenEXR for reading and writing simple floating point images for
> quite a while now and I am totally happy with it. Recently, I started
> working with huge amounts of data, let's say an image has a resolution of
> 640x480 by 1024x768. Until now I stored every column of these images in
> separate files, resulting a many files that are very slow to read and
> cumbersome to handle anyway. I have to process the data, so the problem is
> to hold all of it in the main memory. Actually this is possible but not very
> convenient. My idea was to swap all the data onto the harddisk in one file
> using a tiled EXR image and only use the data that I currently need. This
> didn't work out as I wanted it to, thus I have a couple of questions (I use
> the C-Interface because I could not link EXR to our software)
>
> 1. Is it possible to allocate an array of ImfRgba that is as big as one tile
> and read and write only a particular tile from/to the
> ImfTiledInputFile/ImfTiledOutputFile into it. I cannot set the data window
> of the input file, which makes sense, but it seems that I have to allocate
> an array of the complete size of the image and can read in tiles into the
> framebuffer. This is not nice, because I waste a lot of memory, if I only
> want to process one tile at a time.
>
> 2. It thought to store each column as a tile, which would allow to read the
> individual columns. However, it would be fantastic to be able to read
> individual rows as well, even after I stored the tiles as columns. The same
> for writing the data. What I need is one big file, that allows reading
> arbitrary rectangles and storing them back in the file without reading the
> whole data and allocating a complete framebuffer.
>
> So my question: is the tiled EXR image format the right thing for me or does
> anyone have alternative suggestions? I would appreciate any hint.
>
> Kind Regards,
>
> Gordon Wetzstein
> Media System Science
> Faculty of Media
> Bauhaus-University Weimar