Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Texture2D.ReadPixels has some Y offset from center

Discussion in 'General Graphics' started by LazloBonin, Dec 1, 2021.

  1. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    795
    I'm trying to write a simple method that crops a Texture2D based on a RectInt.

    The method is:

    Code (CSharp):
    1. public static Texture2D Crop(this Texture2D source, RectInt crop)
    2. {
    3.     var previousActiveRenderTexture = RenderTexture.active;
    4.  
    5.     var sourceRenderTexture = RenderTexture.GetTemporary(source.width, source.height, 0, source.graphicsFormat);
    6.     RenderTexture.active = sourceRenderTexture;
    7.     GL.Clear(false, true, Color.clear);
    8.     Graphics.Blit(source, sourceRenderTexture);
    9.  
    10.     var texture2D = new Texture2D(crop.width, crop.height, source.format, source.mipmapCount, linear: false);
    11.     texture2D.filterMode = source.filterMode;
    12.     texture2D.hideFlags = HideFlags.HideAndDontSave;
    13.  
    14.     var srcX = crop.x;
    15.     var srcY = crop.y;
    16.     var srcW = crop.width;
    17.     var srcH = crop.height;
    18.     var dstX = 0;
    19.     var dstY = 0;
    20.     Debug.Log($"[{srcX}, {srcY}, {srcW}, {srcH}] => ({dstX}, {dstY})");
    21.  
    22.     texture2D.ReadPixels(new Rect(srcX, srcY, srcW, srcH), dstX, dstY);
    23.     texture2D.Apply();
    24.  
    25.     RenderTexture.active = previousActiveRenderTexture;
    26.     RenderTexture.ReleaseTemporary(sourceRenderTexture);
    27.  
    28.     return texture2D;
    29. }
    30.  
    However, I'm running into an issue where the cropped texture is being sampled in the wrong place. It has an unexplainable Y offset. From observation, it seems that the offset is bigger the farther my crop rect Y center is from the source Y center. The offset is proportional to the distance somehow.

    I've read about every Google / StackOverflow thread I could find on ReadPixels and coordinates and none seem to explain the problem.

    What's baffling me is that the X coordinate has no offset and is cropped perfectly fine.

    My closest lead is platform specific rendering differences where the vertical UV coordinates for render textures may be flipped in DirectX, but to my understanding that would manifest as a flipped Y in the cropped texture, not an offsetted Y. Even with that in mind, I tried variations on srcY/dstY, but I always get errors that I'm reading out of bounds when I do.

    What am I missing?! I'm going a bit crazy here.

    Extra details:
    - I've measured the logged src/dst values on screen, and they are correct in my understanding
    - This method gets called in play mode, during an Update() method
    - Scene view resolution or aspect ration does not seem to affect the bug
    - I'm on Windows
    - Unity 2020.3.20f1 LTS
     
    Last edited: Dec 1, 2021
  2. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    795
    Alright, turns out it's a very simple issue: the documentation for ReadPixels lied! The documentation says:

    However, that's not true. In the source rect parameter, (0, 0) is actually top left.

    Therefore, in my code above, to get a behaviour where coordinates are both sampled from bottom left, you need to change:

    Code (CSharp):
    1. var srcY = source.height - crop.height - crop.y;
     
    sildeflask likes this.
  3. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    795
    Reported the documentation error, case 1385050.
     
  4. UselessGameDev

    UselessGameDev

    Joined:
    Jul 15, 2018
    Posts:
    3
    Hi, sorry to bump this up but this is the first search engine result and there is news regarding this topic.

    I have the same use case (crop an image).
    I am also on Windows.
    I also came up with the coordinate shift offset thing and it worked very well.

    Until a colleague played our game using MacOS devices (both Apple M1 and Intel) and their image is shifted. If I remove the coordinate offset, it works for them.
    It seems their device does place (0,0) at the bottom left. So the documentation didn't lie if the writer was using a MacOS computer.

    This issue only seems to affect RenderTextures and not the main Camera texture (or the other way around depending how you look at the problem).

    So we ended up using a good old #if !UNITY_STANDALONE_OSX but I fear this will come back to bite us when porting to other platforms such as consoles.

    Extra Details :
    - I did find SystemInfo.graphicsUVStartsAtTop but it did not solve the problem I think it's more an issue of a discrepancy between RTs and the main camera texture on Windows
    https://docs.unity3d.com/ScriptReference/SystemInfo-graphicsUVStartsAtTop.html
    - 2021.3.7f1 LTS
    - The MacOS version uses Metal Graphics API. Couldn't test it with OpenGLCore API because we're building for Apple silicon
     
    seobyeongky likes this.
  5. sildeflask

    sildeflask

    Joined:
    Aug 16, 2023
    Posts:
    156
    this saved me, still bugged on 2021 LTS
     
  6. unity_TEkEHKcvu5sRAw

    unity_TEkEHKcvu5sRAw

    Joined:
    Sep 11, 2023
    Posts:
    7
    just to add to Lazlos explanation abouve and because it took me THREE DAYS! to get this to work I wanted to note my findings here also for anyone having no idea how this is working (like me)

    Be careful to get the Y position correct, else you will have weird offset when using ReadPixels


    I created a rect for the position where the user is clicking for a size of 10 pixel:

    Rect rect = new Rect(click.x, click.y, 10, 10)

    That will create the rect here starting from the x,y point
    ................
    ::::::::::::::::
    :::::::::::::::x,y

    Then I called ReadPixels(myRect, 0, 0);

    But seems you need the Rect constructed from left bottom up

    :::::::::::::::x,y
    ::::::::::::::::
    ::::::::::::::::


    So for the rect creation, I took the same X position of the click, but for Y I took the application Height minus y like so:
    gameCamera.scaledPixelHeight - click.y

    To find out exactly what happens when I click a certain point on screen, I created a JPG screenshot out of the Texture2D I was working with, to see what was actually created on click. That helped me figure out, which parameter was inverted.
    So if you have weird offsets, try saving the picture and actually looking at it while you click :)
     
    Last edited: Oct 6, 2023
  7. sildeflask

    sildeflask

    Joined:
    Aug 16, 2023
    Posts:
    156
    your explanation is incorrect, since you used read pixels to check your results, your 1st finding is already tarnished, your first rect is correct, but read pixels interprets it in reverse
     
  8. unity_TEkEHKcvu5sRAw

    unity_TEkEHKcvu5sRAw

    Joined:
    Sep 11, 2023
    Posts:
    7
    sorry, adapted the "rect graphic" to correctly show what i meant. at least now it works perfect for me and it was inversed before.
     
  9. sildeflask

    sildeflask

    Joined:
    Aug 16, 2023
    Posts:
    156
    nono what I mean is this
    this is incorrect, this creates a normal rect from the bottom left

    you can test it with any other feature of unity that is not readpixels

    just the read pixels will process the rect incorrectly, and thats why you need to input a different rect

    its read pixels that is flawed, not the original rect