Question hdr cubemaps questions

Hello there,

It was unclear to me how values from hdr cubemaps would transport into unity so I had a more in depth look.

As far as I can tell there are two shemes of encoding, dLDR and rgbM where the latter has a range of 0-5
(judging by https://community.arm.com/developer...gh-quality-rgbm-texture-compression-with-astc )

dLDR seems to match that range 0-5, albeit with less precision

Any pixel value beyond 5 seems to be clamped.

There seem to be still changes in the convoluted hdr, I guess this comes from the convolution process where there would be more bleed in the mipmaps in my example (?)

If anybody could confirm the above (or not), I'd appreciate it!
Now some questions I still have:

Something I don't understand: Setting the reflection intensity to 0.5 puts a pixel value of 5 in the original hdr to exactly 1. I would have imagined an intensity of 0.2 would do that. (maybe this is because the final value is color corrected?)

Another thing I don't get is the srgb toggle. On windows this does not have any effect at all but when switching to android, turning off srgb will give a washed out look. Not sure this is intended?

Tests:

Changing intensities: 3,4,5,6,20 (of the pixels marked by the arrow)
hdr_tests_intensity.gif

(I know it's hard to see in the gif, but colorpicking results in 1 above a source intensity of 5)

rgbaM vs dLDR :
hdr_tests_encode.gif

srgb setting on android:
hdr_tests_srgb.gif



The project is URP, Unity 2019.4 ( I tried later, same results), linear rendering. I've attached that it as well.

 

It's a bug ... though humorously it's actually a feature some of us have been asking for on desktop. Having linear data stored in HDR native formats (BC6H, RGBAHalf, etc) is incredibly useful, and supported by Unity. But there's no way to import an .hdr or .exr file without Unity color doing color correction to it in some specific situations. HDR textures packed using dLDR or rgbM should always be sRGB so it's definitely a bug that check box does anything.

Gamma color space vs linear color space can be weird to think about.

"Gamma" space is actually sRGB space. The conversion between sRGB and linear color values is weird and more complex than just a multiply or power. Which color space you're using depends on a setting in the project, and mainly changes if the render buffer used for rendering stores the values in sRGB or linear, and if textures that are marked as being sRGB should be color corrected when sampled by the shader (when using linear color space).

The range for rgbM is between 0 and 5 in sRGB gamma color space. That's a peak brightness of 34.49 (5^2.2) in linear color space. The RGB channels are used to store the hue & saturation of a color value, and the Alpha channel is used to store the brightness. To decode it in the shader the RGB and A are multiplied together, then multiplied by either 5 or 34.49 depending on if you're using gamma or linear color space.

The range for dLDR (Double Low Dynamic Range) is just between 0 and 2 in gamma color space. That's a peak brightness of 4.595 (2^2.2) in linear color space. It's much less than rgbM. The decoding step is taking the texture and either multiplying it by 2 or 4.595 depending on if you're using gamma or linear color space.

However that should make that "intensity of 0.5 == 1.0" make more sense, as it's affecting the intensity in gamma space. And 2 * 0.5 = 1.

https://docs.unity3d.com/Manual/Lightmaps-TechnicalInformation.html

It should be noted that multiplying the color value by the gamma space max raised to a power of 2.2 is a complete hack. And really both rgbM and dLDR are kind of hacks too. They're both lossy conversions, obviously in the color precision, but also in the brightness representation within the precision they can represent, at least when using linear color space. The problem is dLDR and rgbM are both going to be encoded expecting to be used with gamma space rendering. A sRGB value of 2.0 is actually 4.954 in linear color space. But as I said above, the conversion between sRGB and linear is a little funny, and you can't just multiply or apply a power to one value to get the other. So if you have HDR values that were encoded with the expectation of being used in sRGB space and you want to use them in linear space multiplying by 4.954 means the values that were expected to decode to 1.0 will be too bright! Where as multiplying them by 4.595 will be very closer to 1.0.

Here are the steps for encoding a dLDR image:

1. You start with a linear space HDR image.
   
2. Convert the pixels from linear to sRGB.
3. Clamp value range between 0.0 and 2.0 in sRGB.
4. Divide values by 2.
5. Store in 8 bit sRGB texture.

If the value is 1.0, linear to sRGB conversion results in a value of 1.0 still. The clamp and divide produces a value of 0.5. Storing in an 8 bit texture produces a value of 127/255, or 0.498 in sRGB space.

The steps for decoding a dLDR texture in sRGB gamma color space rendering is read texture, get the value of 0.498 in the shader, multiply by 2.0, and you get 0.996. Pretty close considering the precision loss.

The steps for decoding a dLDR texture in linear color space rendering is read the texture, which does an sRGB to linear conversion and results in a value of 0.2122. Multiply that by 4.594 and you get 0.975. Still pretty close, but slightly dimmer. The problem is HDR values below about 0.5 end up brighter than they originally were, and values above that all get dimmer.


TLDR: HDR on mobile is imperfect and it's totally expected that different HDR encoding schemes aren't going to look exactly the same as it does in other color spaces / HDR encoding options.

 

@bgolus Thanks so much for the big writeup making more clear what I'm seeing. You're a star!

 

Appreciate for the detailed explanation! I still have some confussion for the hdr sky usage. If the target is mobile platform , it is a appropriate way to just to clamp the output color value of skybox shader to make things consistent for editor and the installed, maybe could be used in lookdev?

 

That could certainly get you closer, yes. Though it may have some impact on the ambient lighting if that's generated from the skybox material (which is the default). I believe the prebaked ambient probe will take into account the full HDR brightness of the skybox even if the platform isn't capable of displaying the skybox textures with the same HDR range on device.

 

I have tested the baked case and you are right. And I believe the full hdr is far more better when used for baking. I will try to explore some useful workflows in this case. Appreciate again!