| Oy, have I really had this hanging over my head since 2014? Sorry, Thiago, if you are reading this.
Tip: I think that in addition to --fail and --failpercent, it may also be wise to set --hardfail as well (generally to a larger value than you used for --fail). The complexity here reflects that it's quite tricky to design good "image is close enough" criteria. You want to allow some pixels to differ by a bit -- but not too many -- but also that if any one pixel is TOO far off, that's also a failure even if there is just one. It's tricky to keep straight, and I have to read the docs carefully every time I use it to remind myself which flags mean what.
Hi Larry,
Thank you for taking the time to provide such a thorough response. Sorry, you had to repeat yourself. I just found Thiago’s post about this from 2014 ( http://lists.openimageio.org/pipermail/oiio-dev-openimageio.org/2014-August/013430.html). Yes, the --diff option with --fail 0.05 and --failpercent 0.1 are working well to address LSB error for my automated tests. For anyone new to oiiotool, --fail and --failpercent must go before —diff.
Thank you, Ryan
I should add that, while -pdiff seemed like a good idea at the time, I'm not sure it's any better in practice than using regular --diff with wisely chosen epsilons to allow a certain amount of LSB error to not be counted as failure. I do that all the time for testsuite image comparisons, where I expect just a little bit of error against the reference output, for different platforms or different versions of dependent libraries.
Oh my, this is embarrassing. I see now that the oiiotool docs do not even explain how those thresholds can be set. So sorry. The undocumented commands are --fail, --failpercent, --warn, --warnpercent, --hardfail. It may be undocumented in the oiiotool section of the OIIO docs, but the same commands are explained in the "idiff" section of the docs (they work the same in the two programs). I'll get that oversight fixed.
It's also been suggested that I look into a replacement (or additional) perceptual diff based on CIEDE2000 or DE2000K (google them for details). Seems like a good idea, but I haven't had time to do it. I'd certainly be very supportive if anybody felt like it would be a project they wanted to work on and submit it as a PR.
Ugg… Sorry, I overlooked
oiiotool --help ... --pdiff Print report on the perceptual difference of two images
Is there a popular or de facto industry standard tool for OpenEXR perceptual image differencing?
_______________________________________________ Openexr-devel mailing list address@hiddenhttps://lists.nongnu.org/mailman/listinfo/openexr-devel
|