Search Unity

  1. Are you interested in providing feedback directly to Unity teams? Sign up to become a member of Unity Pulse, our new product feedback and research community.
    Dismiss Notice

Resolved RenderTexture ReadPixels GetPixel shortcut

Discussion in 'General Graphics' started by Abended, May 3, 2021.

  1. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Code (CSharp):
    1. RenderTexture.active = tex;
    2. currenttexture = new Texture2D(2048, 2048, TextureFormat.RGBA32, false, true);
    3. currenttexture.ReadPixels(new Rect(0, 0, tex.width, tex.height), 0, 0, false); ;
    4. currenttexture.Apply(false);
    I have been reading about this all day. I have come to understand it's about GPU data vs CPU data.
    The reason I need to perform this heavy action is to run GetPixel on the new Texture2D to get a single pixel.
    The X and Y of that pixel is derived after I create the new texture and get its height and width.
    I am dealing with a 2048x2048 Rendertexture and the hit is noticeable.

    Is it possible to determine that Getpixel X and Y first, and then define ReadPixels so it only pulls back a minimal pixel group 16x16 or 8x8 (I don't know how small a texture2d can be) so my resulting Texture2D is the minimum size and my pixel sits at 0,0?

    It really feels like a waste to copy a 16mb image for 1 pixel. I have seen a lot of code today for getting a pixel from a RenderTexture, none suggested this. Is there a reason this would be an issue? You could essentially create a getpixel for rendertextures without taking the hit.
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
    There's no reason you couldn't copy a single pixel from the render texture using ReadPixels.
    Code (csharp):
    1. currenttexture = new Texture2D(1, 1, TextureFormat.RGBA32, false, true);
    2. currenttexture.ReadPixels(new Rect(0,0,1,1), xOff, yOff, 0, 0, false);
    Also no reason to call
    Apply()
    unless you need to send that single pixel back to the GPU. And if that's the case you can avoid this whole slow
    ReadPixels()
    thing by copying that single pixel value into a single pixel texture on the GPU.
    https://docs.unity3d.com/ScriptReference/Graphics.CopyTexture.html
    Code (csharp):
    1. // don't need to recreate it every frame
    2. if (currenttexture == null)
    3.   currenttexture = new Texture2D(1, 1, TextureFormat.RGBA32, false, true);
    4.  
    5. // copy the single pixel value from the render texture to the texture2D on the GPU
    6. Graphics.CopyTexture(tex, 0, 0, xOff, yOff, 1, 1, currenttexture, 0, 0, 0, 0);
    Do not call
    Apply()
    with this setup! That copies from the CPU data to the GPU, and the CPU data isn't updated here, so it'll just reset the texture back to the default black and it'll seem like nothing is working.

    The other question is, if you do need the data on the CPU, do you need to have the data "right now", or can you wait a frame or two and still be acceptable? In that case you can use async readback. Keijiro has a great example of how to do this here:
    https://github.com/keijiro/AsyncCaptureTest
    https://github.com/keijiro/AsyncCaptureTest/blob/master/Assets/AsyncCapture.cs
    In some cases it might even be the same frame if you do the request early enough in the frame, though it's best not to rely on that. You can also use the above trick of getting only the single pixel value if you create a 1x1 temporary render texture and use
    CopyTexture()
    like above, and request a readback on that 1x1 render texture.
     
    AljoshaD likes this.
  3. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Man, I was hoping you'd be the one to answer me.

    I will try without the Apply() and see if I still get a read. I've learned that the pixels read (.804,.804,.804) when my new texture does not grab the data correctly, so that is an easy check.

    I suppose for my purposes, I do not need the result immediately. I was more looking to cut down the time and the memory I'm chewing up trying to get that pixel. I really appreciate your insight. This is going to make the whole process seem snappy and light. Glad I can ditch that giant texture.

    And just for a bit of color, my rendertexture is generating a dynamic fairway, and I am checking the point of impact to see if we are on the fairway or in the rough by looking at the alpha channel. I've really learned a lot this weekend. Thanks again!
     
  4. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    It seems that ReadPixels does not work the way we thought.

    Code (CSharp):
    1. currenttexture.ReadPixels(new Rect(0, 0, 1, 1), 1659, 384, false);
    I am getting an error: Trying to read pixels out of bounds

    after looking at the ReadPixels definition it looks like:
    public void ReadPixels(Rect source, int destX, int destY, bool recalculateMipMaps = true);

    I think that the Rect needs to specify the specific pixel, and the destX and destY are still 0,0.
    so based on the numbers above is it:
    Code (CSharp):
    1. currenttexture.ReadPixels(new Rect(1659, 384, 1, 1), 0, 0, false);
    *Update, that was it. Smooth as can be!
     
    bgolus likes this.
  5. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    @bgolus

    Turns out I've been getting some false positives. I can see that my original code to take the RT wholesale still seems to be working, but my new code implementing just the pixel size texture is having some issues. I put both scripts on at the same time so I can compare the log output. So far it seems like both are giving me the same x and y for the point of the pixel. I have to do more tests to understand where the failure is.
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
    I just copy/pastad your example code and quickly modified it. I never use
    ReadPixels()
    so I didn't remember the argument layout and didn't bother to check it. :rolleyes:
     
  7. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Is there any reason to think that snagging a pixel when setting the RT to active may somehow miss and get a pixel from the normal screen?
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
    Not that I can think of.
     
  9. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Full RT rect
    OLD result x: 1610 y: 668
    OLD 0.4039216
    OLD On Fairway: True 0.572549

    one pixel rect
    result x: 1610 y: 668
    0.4078431
    On Fairway: True 0.5803922

    here are results from both scripts on the same collision.
    they carry the same x and y.
    the second number is RGB Green
    the last number is Alpha.
     
  10. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    I think somehow the rect is not just capturing the area I am assuming it would with the same x and y values.
     
  11. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
    It's possible it's off by a pixel, or you're assuming the coordinate are starting from a different corner than they are.
     
  12. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    well the full rect is (0,0,2048,2048)

    I'm assuming we are going from the bottom left corner when we use

    new Rect(1659, 384, 1, 1)

    but I'm starting to suspect that it's off. I may bump it up to 256 or 512 and output the texture to png just so I can try to match it. Something's fishy.
     
  13. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Ok here's what I found. I made a 512x512 pixel to see if I could tell what is happening.
    The x and y are 212 and 297 respectively.
    upload_2021-5-3_22-5-50.png

    the red x is where the impact is. that is the pixel I'm looking for. the dark part is me finding where the 512x512 puzzle piece goes. seem like the rect is using the upper left corner as a starting point. so (0,0, 2048,2048) rect must go down and right. this gave me the result of on the fairway because the lower left pixel of the darker piece is of the fairway.
    so I have to now adjust for topleft origin.

    and here it gets the pixel point from top left and then moves down and right to fill it.

    I suspect that the x is correct, but the y has to be 2048 - y?
     
  14. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    So 2048 - Y worked to fix my issue, I reduced the Texture2D back down to 1x1. I was getting slightly different readings so considering the RT readpixels is topleft down and getpixel is bottomleft up, I subtracted 1 from x and y before the readpixels, and now I am pixel perfect. I wish there was an easier way to visualize what is happening, but I am more than happy with this quick and light solution to reading a single pixel from a RT without all the overhead. Thanks to @bgolus for all the help!
     
  15. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
  16. Abended

    Abended

    Joined:
    Oct 9, 2018
    Posts:
    119
    Docs schmocks, I believe it's the rect is starting in upper left.

    and the rect page shows exactly my experience:
    upload_2021-5-4_6-46-45.png
    I did run across a post that says rects should not act like this for readpixels, but it was from 2014 so I imagine they made it uniform at some point. working as expected now, has been a great learning week!
     
  17. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    10,900
    The rect documentation is a little confusing if you just look at the images. At the top of the documentation it says this:
    So someone decided the best way to show visual examples of how the Rect class works is to use the orientation that is the exception and not the common one. Though to be fair GUI related stuff is likely the most frequent usage of the Rect class.

    The thing to understand about the Rect class, or really anything that defines a position, has no inherent orientation on its own and depends entirely on the orientation of the coordinate system its being used in.
     
    Last edited: May 4, 2021
    tankorsmash likes this.
  18. Xine27

    Xine27

    Joined:
    Feb 20, 2019
    Posts:
    7
    After some frustrating debugging, I can also add another problem with rect orientation in ReadPixels:

    It would appear that OSX editor and player are reading from the bottom left corner, while Win editor and player are reading from the top left corner (at least in 2020.3.13) - certainly not the most intuitive setup...
     
  19. crowejohn20

    crowejohn20

    Joined:
    Oct 5, 2012
    Posts:
    8
    I know this is an old thread but I ran into this today while using the free aspect resolution settings in 2021.1.12f. It seems if you use that setting you need to reduce the width of your rect by 1 pixel in the X and Y. Otherwise you'll get the read pixel error.

    Setting to a standard resolution works without taking -1 from the X, Y. Never come across it before. Not sure the reason. I'll try regression testing it.
     
unityunity