Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Fastest way of Grayscaling a image in editor time? Maybe compute shaders?

Discussion in 'Shaders' started by carcasanchez, Apr 15, 2021.

  1. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Hello, fellow Uniters! I'm in the situation where I have and editor script that generates a grayscaled, graded and blurred copy of a texture. It's a standard operation, and doesn't take too much time (like, 3 or 5 seconds in total). The problem it's that I usually convert a lot of textures at once, and easily scalates to a wait of 1 or 2 minutes. I'm investigating if its possible to optimize this process, but the only idea I have in mind is to use compute shaders to do the operation.
    My doubt is if this is a viable option (I know compute shaders has some speed issues comunicating between CPU and GPU), if someone has some documentation at hand, or if there are better alternatives.
    I drop the code here:
    Code (CSharp):
    1. private static void GrayScale(Texture2D _currentDiffuse)
    2.         {
    3.             Color32[] pixels = _currentDiffuse.GetPixels32();
    4.             Color32[] pixelsDest = _currentDiffuse.GetPixels32();
    5.  
    6.             for (int x = 0; x < _currentDiffuse.width; x++)
    7.             {
    8.                 for (int y = 0; y < _currentDiffuse.height; y++)
    9.                 {
    10.                     Color32 pixel = pixels[x + y * _currentDiffuse.width];
    11.                     int p = (int)(((256 * 256 + pixel.r) * 256 + pixel.b) * 256 + pixel.g);
    12.                     int b = p % 256;
    13.                     p = Mathf.FloorToInt(p / 256);
    14.                     int g = p % 256;
    15.                     p = Mathf.FloorToInt(p / 256);
    16.                     int r = p % 256;
    17.                     float l = (0.2126f * r / 255f) + 0.7152f * (g / 255f) + 0.0722f * (b / 255f);
    18.  
    19.                     Color c = new Color(l, l, l, pixel.a);
    20.                     pixelsDest[x + y * _currentDiffuse.width] = c;
    21.                 }
    22.             }
    23.             _currentDiffuse.SetPixels32(pixelsDest);
    24.             _currentDiffuse.Apply();
    25.         }
    Code (CSharp):
    1.  protected static void CreateGradientMap(Texture2D t, float gradientStrenght)
    2.         {
    3.             Color[] pixels = new Color[t.width * t.height];
    4.             Color[] pixelsNew = new Color[t.width * t.height];
    5.  
    6.             pixels = t.GetPixels();
    7.  
    8.             for (int y = 0; y < t.height; y++)
    9.             {
    10.                 for (int x = 0; x < t.width; x++)
    11.                 {
    12.                     if (x == 0 || y == 0 || x == t.width - 1 || y == t.height - 1)
    13.                     {
    14.                         pixelsNew[x + y * t.width] = Color.black;
    15.                     }
    16.                     else
    17.                     {
    18.                         Color tc = pixels[(x - 1) + (y - 1) * t.width];
    19.                         Vector3 cSampleNegXNegY = new Vector3(tc.r, tc.g, tc.g);
    20.                         tc = pixels[(x) + (y - 1) * t.width];
    21.                         Vector3 cSampleZerXNegY = new Vector3(tc.r, tc.g, tc.g);
    22.                         tc = pixels[(x + 1) + (y - 1) * t.width];
    23.                         Vector3 cSamplePosXNegY = new Vector3(tc.r, tc.g, tc.g);
    24.                         tc = pixels[(x - 1) + (y) * t.width];
    25.                         Vector3 cSampleNegXZerY = new Vector3(tc.r, tc.g, tc.g);
    26.                         tc = pixels[(x + 1) + (y) * t.width];
    27.                         Vector3 cSamplePosXZerY = new Vector3(tc.r, tc.g, tc.g);
    28.                         tc = pixels[(x - 1) + (y + 1) * t.width];
    29.                         Vector3 cSampleNegXPosY = new Vector3(tc.r, tc.g, tc.g);
    30.                         tc = pixels[(x) + (y + 1) * t.width];
    31.                         Vector3 cSampleZerXPosY = new Vector3(tc.r, tc.g, tc.g);
    32.                         tc = pixels[(x + 1) + (y + 1) * t.width];
    33.                         Vector3 cSamplePosXPosY = new Vector3(tc.r, tc.g, tc.g);
    34.  
    35.                         float fSampleNegXNegY = cSampleNegXNegY.x;
    36.                         float fSampleZerXNegY = cSampleZerXNegY.x;
    37.                         float fSamplePosXNegY = cSamplePosXNegY.x;
    38.                         float fSampleNegXZerY = cSampleNegXZerY.x;
    39.                         float fSamplePosXZerY = cSamplePosXZerY.x;
    40.                         float fSampleNegXPosY = cSampleNegXPosY.x;
    41.                         float fSampleZerXPosY = cSampleZerXPosY.x;
    42.                         float fSamplePosXPosY = cSamplePosXPosY.x;
    43.  
    44.                         float edgeX = (fSampleNegXNegY - fSamplePosXNegY) * 0.25f + (fSampleNegXZerY - fSamplePosXZerY) * 0.5f + (fSampleNegXPosY - fSamplePosXPosY) * 0.25f;
    45.                         float edgeY = (fSampleNegXNegY - fSampleNegXPosY) * 0.25f + (fSampleZerXNegY - fSampleZerXPosY) * 0.5f + (fSamplePosXNegY - fSamplePosXPosY) * 0.25f;
    46.  
    47.                         float fValue = (edgeX + edgeY) / 2.0f;
    48.                         fValue = 1 - fValue / gradientStrenght;
    49.  
    50.                         Color c = new Color(fValue, fValue, fValue, tc.a);
    51.  
    52.                         pixelsNew[x + y * t.width] = c;
    53.                     }
    54.                 }
    55.             }
    56.  
    57.             t.SetPixels(pixelsNew);
    58.             t.Apply();
    59.         }
    Code (CSharp):
    1.         protected static void BlurImage(Texture2D t)
    2.         {
    3.             Color[] pixels = new Color[t.width * t.height];
    4.             Color[] pixelsNew = new Color[t.width * t.height];
    5.             pixels = t.GetPixels();
    6.  
    7.             for (int y = 0; y < t.height; y++)
    8.             {
    9.                 for (int x = 0; x < t.width; x++)
    10.                 {
    11.                     if (x == 0 || y == 0 || x == t.width - 1 || y == t.height - 1)
    12.                     {
    13.                         pixelsNew[x + y * t.width] = Color.black;
    14.                     }
    15.                     else
    16.                     {
    17.                         Color tc = pixels[(x - 1) + (y - 1) * t.width];
    18.                         Vector3 cSampleNegXNegY = new Vector3(tc.r, tc.g, tc.g);
    19.                         tc = pixels[(x) + (y - 1) * t.width];
    20.                         Vector3 cSampleZerXNegY = new Vector3(tc.r, tc.g, tc.g);
    21.                         tc = pixels[(x + 1) + (y - 1) * t.width];
    22.                         Vector3 cSamplePosXNegY = new Vector3(tc.r, tc.g, tc.g);
    23.                         tc = pixels[(x - 1) + (y) * t.width];
    24.                         Vector3 cSampleNegXZerY = new Vector3(tc.r, tc.g, tc.g);
    25.                         tc = pixels[(x + 1) + (y) * t.width];
    26.                         Vector3 cSamplePosXZerY = new Vector3(tc.r, tc.g, tc.g);
    27.                         tc = pixels[(x - 1) + (y + 1) * t.width];
    28.                         Vector3 cSampleNegXPosY = new Vector3(tc.r, tc.g, tc.g);
    29.                         tc = pixels[(x) + (y + 1) * t.width];
    30.                         Vector3 cSampleZerXPosY = new Vector3(tc.r, tc.g, tc.g);
    31.                         tc = pixels[(x + 1) + (y + 1) * t.width];
    32.                         Vector3 cSamplePosXPosY = new Vector3(tc.r, tc.g, tc.g);
    33.  
    34.                         tc = pixels[x + y * t.width];
    35.                         Vector3 cSampleXY = new Vector3(tc.r, tc.g, tc.g);
    36.  
    37.                         float fSampleNegXNegY = cSampleNegXNegY.x;
    38.                         float fSampleZerXNegY = cSampleZerXNegY.x;
    39.                         float fSamplePosXNegY = cSamplePosXNegY.x;
    40.                         float fSampleNegXZerY = cSampleNegXZerY.x;
    41.                         float fSamplePosXZerY = cSamplePosXZerY.x;
    42.                         float fSampleNegXPosY = cSampleNegXPosY.x;
    43.                         float fSampleZerXPosY = cSampleZerXPosY.x;
    44.                         float fSamplePosXPosY = cSamplePosXPosY.x;
    45.                         float fSampleXY = cSampleXY.x;
    46.  
    47.  
    48.                         float fBlurredValue = (fSampleNegXNegY + fSampleZerXNegY + fSamplePosXNegY + fSampleNegXZerY + fSamplePosXZerY + fSampleNegXPosY + fSampleZerXPosY + fSamplePosXPosY + 3 * fSampleXY) / 11.0f;
    49.                         Color c = new Color(fBlurredValue, fBlurredValue, fBlurredValue, tc.a);
    50.  
    51.  
    52.                         pixelsNew[x + y * t.width] = c;
    53.                     }
    54.                 }
    55.             }
    56.             t.SetPixels(pixelsNew);
    57.             t.Apply();
    58.  
    59.         }
     
  2. Phy_rax

    Phy_rax

    Joined:
    Oct 20, 2018
    Posts:
    27
    Pretty sure that using a compute shader is way faster and definitely viable, could not think of a reason why not. If one Image takes you 3-5 seconds it will be a huge boost. I would guess at least x100-1000 for the process itself.

    Also the bottleneck between CPU and GPU shouldn't be too much of a problem if I'm not mistaken, although surely the one that takes the longest, since your algorithm isn't really computationally intensive.

    What quite detailed tutorial and explanation of compute shaders can be found here: https://catlikecoding.com/unity/tutorials/basics/compute-shaders/

    Shouldn't take too long to set it up since you already have your algorithm.

    Don't know if you could actually also do that with Unity's ShaderGraph (if you use URP or HDRP) - don*t know how to save the image then though.
     
  3. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Thank for the link! I have started some experiments and seems very promising.
    Edit: I have stumbled upon a curious problem. Everything goes perfect, and its incredibly fast, but the final texture is smaller than the input one!
    Don't mind the colored part, that is added in another step, but the grayed part of the image should occupy all the texture.
    I have checked the code, and everything is 2048x2048, so I don't know where the problem could be.
    Code (CSharp):
    1.  private static void GrayScale(Texture2D _currentDiffuse)
    2.         {
    3.             float timer = Time.realtimeSinceStartup;
    4.  
    5.             RenderTexture t = new RenderTexture(_currentDiffuse.width, _currentDiffuse.height, 24);
    6.             t.enableRandomWrite = true;
    7.          
    8.             t.Create();
    9.  
    10.             int k = grayscaleShader.FindKernel("CSMain");
    11.             grayscaleShader.SetTexture(k, "inputTexture", _currentDiffuse);
    12.             grayscaleShader.SetTexture(k, "outputTexture", t);
    13.             grayscaleShader.Dispatch(k, _currentDiffuse.width, _currentDiffuse.height, 1);
    14.  
    15.             RenderTexture.active = t;
    16.             _currentDiffuse.ReadPixels(new Rect(0, 0, t.width, t.height), 0, 0);
    17.             _currentDiffuse.Apply();
    18.             RenderTexture.active = null;
    19.                Debug.Log("Grayscaling took: " + (Time.realtimeSinceStartup - timer).ToString("f0"));
    20.         }
     
    Last edited: Apr 15, 2021
    lilacsky824 likes this.
  4. Phy_rax

    Phy_rax

    Joined:
    Oct 20, 2018
    Posts:
    27
    Glad to hear it, no problem!
    Would love to hear how well the results work out then :)
     
  5. lilacsky824

    lilacsky824

    Joined:
    May 19, 2018
    Posts:
    171
    Can you share compute shader that you use?
    Probably is texture sample coordinate wrong cause resize?
     
  6. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Of course
    Code (CSharp):
    1. #pragma kernel CSMain
    2.  
    3. Texture2D inputTexture;
    4. RWTexture2D<float4> outputTexture;
    5.  
    6. [numthreads(1, 1, 1)]
    7. void CSMain(uint3 id : SV_DispatchThreadID)
    8. {
    9.     float R = inputTexture[id.xy].r;
    10.     float G = inputTexture[id.xy].g;
    11.     float B = inputTexture[id.xy].b;
    12.     float A = 1;
    13.  
    14.     float Y = R * 0.299 + G * 0.587 + B * 0.114; // RGB to grayscale
    15.  
    16.     outputTexture[id.xy] = float4(Y, Y, Y, A); // throw back to the CPU
    17. }
     
  7. lilacsky824

    lilacsky824

    Joined:
    May 19, 2018
    Posts:
    171
    mmm. Strange. I can't reproduce problem.
    Can you use Load instead?
    Like this.
    Code (CSharp):
    1. inputTexture.Load(int3(id.xy, 0));
     
  8. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Same result. I have been doing an isolated test to see where the problem could be. Just a script editor where I send the texture to GPU and return it, no extra operations. It just happens with every texture. I tried also to hardcode the size, and still the same. The final texture size is the correct, but the image is shrinked to exactly 1/2.
    upload_2021-4-16_11-58-39.png
    Code (CSharp):
    1. #pragma kernel CSMain
    2.  
    3. Texture2D inputTexture;
    4. RWTexture2D<float4> outputTexture;
    5.  
    6. [numthreads(1, 1, 1)]
    7. void CSMain(uint3 id : SV_DispatchThreadID)
    8. {
    9.     outputTexture[id.xy] = inputTexture.Load(int3(id.xy, 0));//float4(Y, Y, Y, A); // throw back to the CPU
    10. }
    Code (CSharp):
    1. public class DoGrayscaleTest : MonoBehaviour
    2. {
    3.     public Texture2D original;
    4.     public RenderTexture processed;
    5.     public ComputeShader grayscaleShader;
    6.    public void DoTest()
    7.     {
    8.  
    9.         processed = new RenderTexture(original.width, original.height, 24);
    10.         processed.enableRandomWrite = true;
    11.  
    12.         processed.Create();
    13.  
    14.         int k = grayscaleShader.FindKernel("CSMain");
    15.         grayscaleShader.SetTexture(k, "inputTexture", original);
    16.         grayscaleShader.SetTexture(k, "outputTexture", processed);
    17.         grayscaleShader.Dispatch(k, original.width, original.height, 1);
    18.     }
    19.  
    20. }
     
  9. lilacsky824

    lilacsky824

    Joined:
    May 19, 2018
    Posts:
    171
    It is correct on my computer.
    Probably is GPU side problem
    未命名-1.png

    Can you change Thread Group size?
    Like.
    Code (CSharp):
    1. int threadGroupAmountX = Mathf.CeilToInt(original.width / 8.0f);
    2. int threadGroupAmountY = Mathf.CeilToInt(original.height / 8.0f);
    3. grayscaleShader.Dispatch(k, threadGroupAmountX, threadGroupAmountY, 1);
    4.  
    And change numthread inside your shader
    Code (CSharp):
    1. [numthreads(8, 8, 1)]
     
  10. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Same result. I tried with 2 and 8, and the behaviour is the same. Strangely, I don't see any cases of this error around the web.
    I think I will make another thread, to see if I can catch the attention of someone from Unity Tech, as it seem rather obscure.
     
  11. lilacsky824

    lilacsky824

    Joined:
    May 19, 2018
    Posts:
    171
    I just found when I change Texture Quality to half will trigger this problem. 未命名-1.jpg
     
    bgolus likes this.
  12. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Oh god, you are right, I had the Texture Quality set to half. Returned it to full quality, and everything worked perfectly.
    Thank you very much. I think I will make a separated post anyway, this seems very strange and I don't know if it is the intended behaviour.
     
  13. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Yeah, that's the intended behavior. When you set the texture quality to less than full res it changes what gets sent to the GPU. The CPU side asset isn't being changed, and the resolution & texture seen by c# is going to be the asset's imported resolution for the current build platform selected. But the GPU is only getting a version of the texture with N mip levels removed from the top (depending on what texture quality setting you've chosen).

    For anything you do in-game, you'll want to check what the current texture quality setting is and adjust the texture size you send to your shader properties accordingly. But for in-editor I would suggest one of two options: set the texture quality to full res in the script and set it back afterward, or change the import settings for the source textures to disable mip maps.
     
    lilacsky824 likes this.
  14. carcasanchez

    carcasanchez

    Joined:
    Jul 15, 2018
    Posts:
    177
    Ok, thanks of the explanation