Search Unity

Linear vs Gamma color space for ui font rendering

Discussion in 'General Graphics' started by IcyHammer, Jun 4, 2019.

  1. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    I noticed when using Linear color space, the alpha values in UI including font and textures are wrong. The main issue is with fonts, since the soft antialiased edges just get more or less cut. I've tried using shaders but the damage done is ireversible and you can't get it back to correct values. Does anyone know if this issue is being addressed?
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,039
    It’s an unfortunate side effect of linear space rendering. Alpha blending no longer acts the way one would expect as the blending is being done in linear space and not sRGB space.

    You can apply some faux gamma correction to the alpha values, but it’s highly content dependent.

    The only real way to fix it would be to use a second camera to render your UI to a separate render texture, and composite the two together with a post process shader that correctly handles the blend in sRGB space.
     
  3. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    Thanks for taking your time, that's the first solution I've seen on this forum, it's strange that almost nobody else is dealing with this issue, i've seen some posts about images but none about text. Regarding the blending of UI to framebuffer, do I just need to use pow function on alpha of UI render texture and then blend it to frambuffer or is there any other step I missed?
     
    Last edited: Jun 6, 2019
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,039
  5. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    Thanks, I'll try to blend whole ui to rt and then to fb, since the background and text will be different, sometimes lighter, sometimes darker, I also hope I'll be able to see that in editor while it's not playing, otherwise this would be a nightmare to work with.
     
  6. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    @bgolus So I tried what we discussed and here are results: https://imgur.com/a/LbFlDVQ

    I created a gradient background from black to white and wrote "A"s on it. In first row they are white and in second they are black. First image is rendered in gamma color space, second in linear and the third is the manual correction where I probably messed something up since it's the worst. What I did to render 3rd image was create render texture, set it to ui camera, and then I added this code to the ui camera:

    Code (CSharp):
    1. private void OnPreRender()
    2. {
    3. uiCamera.targetTexture = uiRt;
    4. }
    5.  
    6. private void OnPostRender()
    7. {
    8. uiCamera.targetTexture = null;
    9. Graphics.Blit(uiRt, null, blitMaterial);
    10. }
    The code in the vertex shader:

    fixed4 fragment_shader(VS_Output input) : COLOR
    {
    float4 bcgCol = tex2D(_BackgroundTexture, input.uv);
    float4 uiCol = tex2D(_MainTex, input.uv);
    bcgCol = pow(bcgCol, 2.2);
    bcgCol = lerp(bcgCol, uiCol, uiCol.a);
    bcgCol = pow(bcgCol, 0.454545);
    return bcgCol;
    }

    What went wrong here, and if I understood correctly, this should work for all cases?
     
  7. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,039
    Try swapping those two pows.

    Also, is the UI render texture sRGB?
     
  8. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    This is the result if I swap them:
    upload_2019-6-24_19-45-39.png

    I did set the srgb, but I had to go to debug menu to enable srgb and than the color format changed to the one you can see below, but for some strange reason it made no visual difference what so ever.

    This is the render texture
    upload_2019-6-24_19-46-37.png
    and the debug view
    upload_2019-6-24_19-47-14.png

    this is the format when i disable srgb:
    upload_2019-6-24_19-49-27.png
     
  9. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    @bgolus Out of curiosity I checked out UE4 to see how they are solving this and they are also using linear space but they somehow manage to also properly blend the alpha from antialiased fonts but I suspect that this should be included somewhere in the rendering pipeline, I'll check if I can do something with the SRP regarding this issue.
     
  10. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,039
    So, two issues.
    One is you need to be doing the in shader gamma correction to the UI texture as well as the main scene texture. Because you're using linear space rendering for the main rendering path, the sRGB render texture is getting gamma corrected into linear space when it's sampled.
    Second is render textures with alpha are premultiplied values, so a straight lerp (which reproduces a traditional alpha blend) is the wrong way to blend the texture together.

    So try this:
    Code (csharp):
    1. bcgCol.rgb = LinearToGammaSpace(bcgCol.rgb);
    2. uiCol.rgb = LinearToGammaSpace(uiCol.rgb);
    3. bcgCol.rgb = bcgCol.rgb * (1 - uiCol.a) + uiCol.rgb; // premultiplied alpha
    4. bcgCol.rgb = GammaToLinearSpace(bcgCol.rgb);
    5. return bcgCol;
    However that won't be all of it. Technically the UI is still being rendered in linear space, just to an sRGB target. When you're rendering with your project set to linear color space to an sRGB render texture, I believe the shader and the blending is still all functioning in linear space, the GPU is just additionally doing linear to sRGB conversions when reading from (before the blend) and writing to (after the blend) the render texture. This is a little more annoying to deal with. Supposedly you can add a script to your UI camera to set GL.sRGBWrite = true OnPreRender and turn it off OnPostRender, but to be honest I've never gotten that to work.
    https://docs.unity3d.com/ScriptReference/GL-sRGBWrite.html

    edit: It looks like setting sRGBWrite to false OnPreRender and back to true OnPostRender gets the correct output ... I'm so confused.

    My guess is Unreal does the compositing for you. It certainly used to for UE3, I don't have a lot of experience with UE4. Once the main scene was rendered I seem to remember it swapped to rendering using sRGB color space for the screen space UI elements.
     
    Last edited: Jun 25, 2019
  11. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    @bgolus thanks for such an extensive explanation, I totally forgot about also converting ui to the gamma space but unfortunately even with all those 3 fixes things do not look correct.

    GL.sRGBWrite doesn't seem to have any visual effect, I also tried reversing the order and still nothing.
    This is the current result when using the shader code that you suggested:
    upload_2019-6-26_19-15-36.png

    And this is the one where I used "pow" with 2.2 and 1/2.2 instead of
    LinearToGammaSpace andGammaToLinearSpace from "UnityCG.cginc"
    upload_2019-6-26_19-14-45.png

    I'm not entirely sure where to go from here, but I'll try one more thing with matlab.
     
  12. andysaia

    andysaia

    Joined:
    Nov 2, 2015
    Posts:
    20
    @IcyHammer did you ever end up finding a solution to this?
     
  13. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    Nope, neither did I get any answer from unity about how should we handle this, or if they are aware of the problem.
     
  14. andysaia

    andysaia

    Joined:
    Nov 2, 2015
    Posts:
    20
    That's too bad. Hopefully, the new scriptable render pipeline gives us more control. We decided to have our artists reauthor the UI assets that look bad.

    One frustrating thing is Photoshop is displaying alpha as if it's blending in Gamma space. After more research then I'd like to admit we figured out you can change this setting in Photoshop by going to Edit -> Color Settings -> Then check Blend RGB Colors Using Gamma with a value of 1.00.

    At least now our artists can see what alpha blending will look like in engine from within photoshop so they can manually tweak alpha to compensate.
     
  15. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    The main problem is font rendering, there is just no way to fix this.
     
  16. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,039
    I spent a little time on this. Here's what I came up with.
    upload_2020-2-10_23-40-7.png

    This is a single screenshot.
    One of those lines is the text in gamma space, screen grabbed, and placed on a texture for reference, rendered by the main camera.
    One of those lines is a TextMeshPro being rendered by a UI only camera that is being composited on top of the main camera as an image effect.
    One of those lines is the same screen grab, but in the UI camera.

    The order of those lines happens to be the order they are in the image, but it doesn’t matter since all 3 are identical.

    So, what am I doing?
    Render the main scene normally.
    I'm rendering the UI camera elements to a render texture using linear color space, but any textures used have sRGB disabled.
    Composite the render texture over the main scene as an image effect, first converting the main camera's image into sRGB space, doing an premultiplied blend in-shader, and outputting the result.
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. [ExecuteInEditMode]
    6. public class GammaSpaceUI : MonoBehaviour
    7. {
    8.     public Camera UICamera;
    9.     public Material UICompositeMaterial;
    10.  
    11.     private int uitex_id = Shader.PropertyToID("_UITex");
    12.  
    13.     void Awake()
    14.     {
    15.         UICamera.enabled = false;
    16.     }
    17.  
    18.     void OnRenderImage(RenderTexture src, RenderTexture dst)
    19.     {
    20.         RenderTexture UIRenderTexture = RenderTexture.GetTemporary(src.width, src.height, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
    21.         UICamera.targetTexture = UIRenderTexture;
    22.         UICamera.Render();
    23.         UICamera.targetTexture = null;
    24.  
    25.         UICompositeMaterial.SetTexture(uitex_id, UIRenderTexture);
    26.  
    27.         // GL.sRGBWrite used to avoid an additional conversion in the shader. Probably won't work on mobile.
    28.     #if !UNITY_ANDROID && !UNITY_IOS
    29.         GL.sRGBWrite = false;
    30.     #endif
    31.  
    32.         Graphics.Blit(src, dst, UICompositeMaterial, 0);
    33.  
    34.     #if !UNITY_ANDROID && !UNITY_IOS
    35.         GL.sRGBWrite = true;
    36.     #endif
    37.  
    38.         RenderTexture.ReleaseTemporary(UIRenderTexture);
    39.     }
    40. }
    Code (CSharp):
    1. Shader "Hidden/GammaSpaceUI"
    2. {
    3.     Properties
    4.     {
    5.         _MainTex ("Texture", 2D) = "white" {}
    6.         _UITex ("Texture", 2D) = "black" {}
    7.     }
    8.     SubShader
    9.     {
    10.         Cull Off ZWrite Off ZTest Always
    11.  
    12.         Pass
    13.         {
    14.             CGPROGRAM
    15.             #pragma vertex vert
    16.             #pragma fragment frag
    17.  
    18.             #include "UnityCG.cginc"
    19.  
    20.         #if defined(SHADER_API_METAL) || defined(SHADER_API_GLES3)
    21.             #define MOBILE_USE_POST_CORRECTION 1
    22.         #endif
    23.  
    24.             struct appdata
    25.             {
    26.                 float4 vertex : POSITION;
    27.                 float2 uv : TEXCOORD0;
    28.             };
    29.  
    30.             struct v2f
    31.             {
    32.                 float4 vertex : SV_POSITION;
    33.                 float2 uv : TEXCOORD0;
    34.             };
    35.  
    36.             v2f vert (appdata v)
    37.             {
    38.                 v2f o;
    39.                 o.vertex = UnityObjectToClipPos(v.vertex);
    40.                 o.uv = v.uv;
    41.                 return o;
    42.             }
    43.  
    44.             sampler2D _MainTex, _UITex;
    45.  
    46.             half3 LinearToGammaSpace3(half3 col)
    47.             {
    48.                 col.r = LinearToGammaSpaceExact(col.r);
    49.                 col.g = LinearToGammaSpaceExact(col.g);
    50.                 col.b = LinearToGammaSpaceExact(col.b);
    51.                 return col;
    52.             }
    53.  
    54.             half3 GammaToLinearSpace3(half3 col)
    55.             {
    56.                 col.r = GammaToLinearSpaceExact(col.r);
    57.                 col.g = GammaToLinearSpaceExact(col.g);
    58.                 col.b = GammaToLinearSpaceExact(col.b);
    59.                 return col;
    60.             }
    61.  
    62.             fixed4 frag (v2f i) : SV_Target
    63.             {
    64.                 fixed4 col = tex2D(_MainTex, i.uv);
    65.                 col.rgb = LinearToGammaSpace3(col.rgb);
    66.                 fixed4 ui = tex2D(_UITex, i.uv);
    67.  
    68.                 col.rgb = col.rgb * (1.0 - ui.a) + ui.rgb;
    69.  
    70.                 // need conversion back to linear space if GL.sRGBWrite doesn't work on your platform (mobile)
    71.             #if defined(MOBILE_USE_POST_CORRECTION)
    72.                 col.rgb = GammaToLinearSpace3(col.rgb);
    73.             #endif
    74.                 return col;
    75.             }
    76.             ENDCG
    77.         }
    78.     }
    79. }
    I've also attached a copy of the assets & scene I used as a zip file.

    So, why do I render the UI to a linear space render target and not an sRGB one? Because it removes an extra step of in-shader color correction. But, as mentioned, it does require all textures used in the UI to not use sRGB. This was true for Unreal too as I remember, probably for the same reason.

    In the c# I'm using
    GL.sRGBWrite
    which I now understand a little better. This appears to allow you to disable the color conversion on shader output, but will still do color conversion on texture sampling. Because we don't have any control over the destination render texture's sRGB settings, we can't disable it entirely. But this means we only have to convert the main camera's output from linear to gamma space, and can use the UI camera's output and the results of the blend directly without having to convert back to linear space. There are warnings in Unity's documentation that this doesn't work on some mobile devices, so I just always do the final conversion back to linear space in-shader for mobile. I don't account for your project being in Gamma (sRGB) color space, but you shouldn't be needing to do any of this in that case anyway.
     

    Attached Files:

  17. IcyHammer

    IcyHammer

    Joined:
    Dec 2, 2013
    Posts:
    64
    @bgolus Amazing work, thanks! This is the first and only solution that I've ever seen on this topic.
     
    unnanego likes this.
  18. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,276
    Man.. how would we do this for URP pipeline. I don't even think there is option to go Gamma in URP.

    How would I render ScreenSpace Overlay UI into a rendertexture.. :(
     
    Noxury likes this.
unityunity