openexr-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Openexr-devel] Technical Introduction update ?


From: Ed Hanway
Subject: Re: [Openexr-devel] Technical Introduction update ?
Date: Tue, 29 Mar 2016 00:30:52 +0000

The DWA compressor will store info in the file indicating which channels were compressed lossily (which theoretically allows a custom writer to use more sophisticated logic to determine how to handle each channel, while maintaining compatibility with all 2.2 readers) but I don't think there's an API for getting at that information.

 

From: openexr-devel-bounces+address@hidden [mailto:openexr-devel-bounces+address@hidden On Behalf Of Deke Kincaid
Sent: Monday, March 28, 2016 5:22 PM
To: address@hidden
Subject: Re: [Openexr-devel] Technical Introduction update ?

 

Ahh got it.  I thought exrheader would explicitly say "zip, individual scanlines" next to those parts and not "dwa, small scanline". blocks".  Is there a way to introspect the compression for each part that dwaa is doing auto-magically? 

Richard: sorry for hijacking your thread ðŸ˜ˆ  It appears a lot of this is explained in the header for ImfDwaCompressor.cpp.  Most of the time the top of the EXR source is just copy-write data and basic comments so I didn't realize there as a whole book chapter in there on the topic. 😀

 

On Mon, Mar 28, 2016 at 4:43 PM, Karl Rasche <address@hidden> wrote:

 

Ahh; I see what you are saying.

 

The filtering we're talking about happens internally when you set the compressor = DWAA or DWAB. You won't see the affect in exrheader.

 

So in your example there are multiple parts, all set to DWAB. But only part 3 will be lossy compressed, and of that, only the A channel.

 

Parts 0-2 will be lossless.

 

Karl

 

On Mon, Mar 28, 2016 at 4:35 PM, Deke Kincaid <address@hidden> wrote:

The channel filtering you and Piotr mentioned as being automatic does not occur.  Every part whether it has XYZ, UV, Z, etc... is written with dwa compression.

 

On Mon, Mar 28, 2016 at 4:29 PM, Karl Rasche <address@hidden> wrote:

 

So.. what's the issue at hand then?

 

On Mon, Mar 28, 2016 at 4:17 PM, Deke Kincaid <address@hidden> wrote:

They must be implementing something incorrectly then.  Below is a truncated exrheader on a file out of Nuke with the "interleave" knob set to "channels" which means each layer is put into a separate part..

 

file testMultiPartChannelsdwaa.exr:

file format version: 2, flags 0x1000

 part 0:
channels (type chlist):
    x, 16-bit floating-point, sampling 1 1
    y, 16-bit floating-point, sampling 1 1
    z, 16-bit floating-point, sampling 1 1
chunkCount (type int): 34                
compression (type compression): dwa, small scanline blocks
dataWindow (type box2i): (0 240) - (2047 1315)           
displayWindow (type box2i): (0 0) - (2047 1555)          
dwaCompressionLevel (type float): 45                     
lineOrder (type lineOrder): increasing y                 
name (type string): "N.left"                             
nuke/full_layer_names (type int): 0                      
nuke/node_hash (type string): "368b9d1b87a4b787"         
nuke/version (type string): "9.0v8"                      
pixelAspectRatio (type float): 1                         
screenWindowCenter (type v2f): (0 0)                     
screenWindowWidth (type float): 1                        
type (type string): "scanlineimage"                      
version (type int): 1                                    
view (type string): "left"                               

 part 1:
channels (type chlist):
    x, 16-bit floating-point, sampling 1 1
    y, 16-bit floating-point, sampling 1 1
    z, 16-bit floating-point, sampling 1 1
chunkCount (type int): 34                
compression (type compression): dwa, small scanline blocks
dataWindow (type box2i): (0 240) - (2047 1315)           
displayWindow (type box2i): (0 0) - (2047 1555)          
dwaCompressionLevel (type float): 45                     
lineOrder (type lineOrder): increasing y                 
name (type string): "P.left"                             
pixelAspectRatio (type float): 1                         
screenWindowCenter (type v2f): (0 0)                     
screenWindowWidth (type float): 1                        
type (type string): "scanlineimage"                      
version (type int): 1                                    
view (type string): "left"                               

 part 2:
channels (type chlist):
    Z, 16-bit floating-point, sampling 1 1
chunkCount (type int): 34                
compression (type compression): dwa, small scanline blocks
dataWindow (type box2i): (0 240) - (2047 1315)           
displayWindow (type box2i): (0 0) - (2047 1555)          
dwaCompressionLevel (type float): 45                     
lineOrder (type lineOrder): increasing y                 
name (type string): "depth.left"                         
pixelAspectRatio (type float): 1                         
screenWindowCenter (type v2f): (0 0)                     
screenWindowWidth (type float): 1                        
type (type string): "scanlineimage"                      
version (type int): 1                                    
view (type string): "left"                               

 part 3:
channels (type chlist):
    u, 16-bit floating-point, sampling 1 1
    v, 16-bit floating-point, sampling 1 1
chunkCount (type int): 34                
compression (type compression): dwa, small scanline blocks
dataWindow (type box2i): (0 240) - (2047 1315)           
displayWindow (type box2i): (0 0) - (2047 1555)          
dwaCompressionLevel (type float): 45                     
lineOrder (type lineOrder): increasing y                 
name (type string): "motion.left"                        
pixelAspectRatio (type float): 1                         
screenWindowCenter (type v2f): (0 0)                     
screenWindowWidth (type float): 1                        
type (type string): "scanlineimage"                      
version (type int): 1                                    
view (type string): "left"                               

 part 4:
channels (type chlist):
    A, 16-bit floating-point, sampling 1 1
    B, 16-bit floating-point, sampling 1 1
    G, 16-bit floating-point, sampling 1 1
    R, 16-bit floating-point, sampling 1 1
chunkCount (type int): 34                
compression (type compression): dwa, small scanline blocks
dataWindow (type box2i): (0 240) - (2047 1315)           
displayWindow (type box2i): (0 0) - (2047 1555)          
dwaCompressionLevel (type float): 45                     
lineOrder (type lineOrder): increasing y                 
name (type string): "rgba.left"                          
pixelAspectRatio (type float): 1                         
screenWindowCenter (type v2f): (0 0)                     
screenWindowWidth (type float): 1                        
type (type string): "scanlineimage"                      
version (type int): 1                                    
view (type string): "left"                  

 

On Mon, Mar 28, 2016 at 3:16 PM, Piotr Stanczyk <address@hidden> wrote:

+1 .. there is nothing that the client code really has to do. It's purely based off the channel names. 

 

-Piotr

 

 

On 28 March 2016 at 15:05, Karl Rasche <address@hidden> wrote:

Deke -

 

All the channel rule setup should happen automagically; there isn't anything for Nuke to setup.

 

That said, if your channels are all named RGB going into the write node, they'll all be handled the same by the compressor.

 

Karl


On Monday, March 28, 2016, Deke Kincaid <address@hidden> wrote:

Thanks for the info Karl.  Currently Nuke doesn't follow these rules and when you pick DWAA and just compresses all channels the same instead of selectively based on channel names. So we are having to write some hackery to work around it at the moment.

 

I'll put this into my feature request/bug to the Foundry.

 

On Mon, Mar 28, 2016 at 2:15 PM, Karl Rasche <address@hidden> wrote:

 

This won't help on the documentation front, but for your question about how channels are handled...

 

This is setup in initializeChannelRules(). That logic is basically saying:

 

  • If the channel name is A, use a lossless RLE compression on it
  • If the channel name is {Y, BY, RY}, and the data is {FLOAT, HALF}, lossy compress the channel
  • If the channel name is {R, G, B}, and the data is {FLOAT, HALF}, lossy compress the channels. 
  • If we have R, G, and B channels, convert to a color difference format prior to quantization. Otherwise, deal with the channels independently as-is.
  • Otherwise, use a lossless compression on the data

So channels named {y, u, U, v, V} should not be lossy compressed.

 

The rule set that Jonathan is referring to is in initializeLegacyChannelRules(), and are slightly different from what 2.2 does by default.

 

Most of this is hold-over from the time before multi-part files allowed more control over compression on different channel sets.

 

Karl

 

On Mon, Mar 28, 2016 at 1:56 PM, Richard Hadsell <address@hidden> wrote:

There has been discussion on the [nuke-prerelease] e-mail list about DWAA and DWAB compression with OpenExr 2.2.  I checked the OpenExr Technical Introduction, but was disappointed to see that it has not been updated since November 2013.  Is there any chance that someone is working on an update?

It is not difficult to use DWAA and DWAB compressions, which are available in the compression header.  That is not the problem.  The questions that I would like to see answered pertain more to lossy vs. lossless compression.  The [nuke-prerelease] discussion included interesting information like this:

On Thu, Mar 17, 2016 at 1:03 PM, Jonathan Egstad <address@hidden> wrote:
...
> 3. Does the lossyness cause issues with data aov's, motion vectors, uv, position, normals, etc...?

 

Yup - it will destroy data AOVs like normals, positions, etc.  I believe the codec recognizes typical color-channel names like 'r/R/red/RED', etc and only lossy compresses those.  Channels it doesn't recognize will get lossless compressed with I believe ZIP - alpha for example is lossless compressed.

One gotcha is I think it traps 'Y/y, u/U, v/V' as valid color channels and lossy-compresses those, which is a problem for xyz and uv channels...

This is how our internal codec works so you might want to check that the one in the official OpenEXR2.2+ release has these behaviors.

...
(end quote)

It would be great to see a list of those special channel names.  Are they the only channels that are compressed with DWAA/DWAB?  What compression would be applied to the other channels?  Does this behavior apply to all lossy compressions?

I hope to see answers to these questions in an updated Technical Intro (eventually).

-- 
Dick Hadsell                 203-992-6320  Fax: 203-992-6001
Reply-to:                    address@hidden
Blue Sky Studios                http://www.blueskystudios.com
1 American Lane, Greenwich, CT 06831-2560

 

_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

 


_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

 


_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

 

 


_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

 

 


_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

 

 


reply via email to

[Prev in Thread] Current Thread [Next in Thread]