Resolved RenderTexture ReadPixels GetPixel shortcut

Code (CSharp):

1. RenderTexture.active = tex;
2. currenttexture = new Texture2D(2048, 2048, TextureFormat.RGBA32, false, true);
3. currenttexture.ReadPixels(new Rect(0, 0, tex.width, tex.height), 0, 0, false); ;
4. currenttexture.Apply(false);

I have been reading about this all day. I have come to understand it's about GPU data vs CPU data.
The reason I need to perform this heavy action is to run GetPixel on the new Texture2D to get a single pixel.
The X and Y of that pixel is derived after I create the new texture and get its height and width.
I am dealing with a 2048x2048 Rendertexture and the hit is noticeable.

Is it possible to determine that Getpixel X and Y first, and then define ReadPixels so it only pulls back a minimal pixel group 16x16 or 8x8 (I don't know how small a texture2d can be) so my resulting Texture2D is the minimum size and my pixel sits at 0,0?

It really feels like a waste to copy a 16mb image for 1 pixel. I have seen a lot of code today for getting a pixel from a RenderTexture, none suggested this. Is there a reason this would be an issue? You could essentially create a getpixel for rendertextures without taking the hit.

 

There's no reason you couldn't copy a single pixel from the render texture using ReadPixels.

Code (csharp):

1. currenttexture = new Texture2D(1, 1, TextureFormat.RGBA32, false, true);
2. currenttexture.ReadPixels(new Rect(0,0,1,1), xOff, yOff, 0, 0, false);

Also no reason to call

Apply()

unless you need to send that single pixel back to the GPU. And if that's the case you can avoid this whole slow

ReadPixels()

thing by copying that single pixel value into a single pixel texture on the GPU.
https://docs.unity3d.com/ScriptReference/Graphics.CopyTexture.html

Code (csharp):

1. // don't need to recreate it every frame
2. if (currenttexture == null)
3.   currenttexture = new Texture2D(1, 1, TextureFormat.RGBA32, false, true);
4.  
5. // copy the single pixel value from the render texture to the texture2D on the GPU
6. Graphics.CopyTexture(tex, 0, 0, xOff, yOff, 1, 1, currenttexture, 0, 0, 0, 0);

Do not call

Apply()

with this setup! That copies from the CPU data to the GPU, and the CPU data isn't updated here, so it'll just reset the texture back to the default black and it'll seem like nothing is working.

The other question is, if you do need the data on the CPU, do you need to have the data "right now", or can you wait a frame or two and still be acceptable? In that case you can use async readback. Keijiro has a great example of how to do this here:
https://github.com/keijiro/AsyncCaptureTest
https://github.com/keijiro/AsyncCaptureTest/blob/master/Assets/AsyncCapture.cs
In some cases it might even be the same frame if you do the request early enough in the frame, though it's best not to rely on that. You can also use the above trick of getting only the single pixel value if you create a 1x1 temporary render texture and use

CopyTexture()

like above, and request a readback on that 1x1 render texture.

 

Man, I was hoping you'd be the one to answer me.

I will try without the Apply() and see if I still get a read. I've learned that the pixels read (.804,.804,.804) when my new texture does not grab the data correctly, so that is an easy check.

I suppose for my purposes, I do not need the result immediately. I was more looking to cut down the time and the memory I'm chewing up trying to get that pixel. I really appreciate your insight. This is going to make the whole process seem snappy and light. Glad I can ditch that giant texture.

And just for a bit of color, my rendertexture is generating a dynamic fairway, and I am checking the point of impact to see if we are on the fairway or in the rough by looking at the alpha channel. I've really learned a lot this weekend. Thanks again!

 

It seems that ReadPixels does not work the way we thought.

Code (CSharp):

1. currenttexture.ReadPixels(new Rect(0, 0, 1, 1), 1659, 384, false);

I am getting an error: Trying to read pixels out of bounds

after looking at the ReadPixels definition it looks like:
public void ReadPixels(Rect source, int destX, int destY, bool recalculateMipMaps = true);

I think that the Rect needs to specify the specific pixel, and the destX and destY are still 0,0.
so based on the numbers above is it:

Code (CSharp):

1. currenttexture.ReadPixels(new Rect(1659, 384, 1, 1), 0, 0, false);

*Update, that was it. Smooth as can be!

 

@bgolus

Turns out I've been getting some false positives. I can see that my original code to take the RT wholesale still seems to be working, but my new code implementing just the pixel size texture is having some issues. I put both scripts on at the same time so I can compare the log output. So far it seems like both are giving me the same x and y for the point of the pixel. I have to do more tests to understand where the failure is.

 

I just copy/pastad your example code and quickly modified it. I never use

ReadPixels()

so I didn't remember the argument layout and didn't bother to check it.

 

Is there any reason to think that snagging a pixel when setting the RT to active may somehow miss and get a pixel from the normal screen?

 

Not that I can think of.

 

Full RT rect
OLD result x: 1610 y: 668
OLD 0.4039216
OLD On Fairway: True 0.572549

one pixel rect
result x: 1610 y: 668
0.4078431
On Fairway: True 0.5803922

here are results from both scripts on the same collision.
they carry the same x and y.
the second number is RGB Green
the last number is Alpha.

 

I think somehow the rect is not just capturing the area I am assuming it would with the same x and y values.

 

It's possible it's off by a pixel, or you're assuming the coordinate are starting from a different corner than they are.

 

well the full rect is (0,0,2048,2048)

I'm assuming we are going from the bottom left corner when we use

new Rect(1659, 384, 1, 1)

but I'm starting to suspect that it's off. I may bump it up to 256 or 512 and output the texture to png just so I can try to match it. Something's fishy.

 

Ok here's what I found. I made a 512x512 pixel to see if I could tell what is happening.
The x and y are 212 and 297 respectively.


the red x is where the impact is. that is the pixel I'm looking for. the dark part is me finding where the 512x512 puzzle piece goes. seem like the rect is using the upper left corner as a starting point. so (0,0, 2048,2048) rect must go down and right. this gave me the result of on the fairway because the lower left pixel of the darker piece is of the fairway.
so I have to now adjust for topleft origin.

and here it gets the pixel point from top left and then moves down and right to fill it.

I suspect that the x is correct, but the y has to be 2048 - y?

 

So 2048 - Y worked to fix my issue, I reduced the Texture2D back down to 1x1. I was getting slightly different readings so considering the RT readpixels is topleft down and getpixel is bottomleft up, I subtracted 1 from x and y before the readpixels, and now I am pixel perfect. I wish there was an easier way to visualize what is happening, but I am more than happy with this quick and light solution to reading a single pixel from a RT without all the overhead. Thanks to @bgolus for all the help!

 

Yeah, it's bottom left corner.
https://docs.unity3d.com/ScriptReference/Texture2D.ReadPixels.html

 

Docs schmocks, I believe it's the rect is starting in upper left.

and the rect page shows exactly my experience:

I did run across a post that says rects should not act like this for readpixels, but it was from 2014 so I imagine they made it uniform at some point. working as expected now, has been a great learning week!

 

The rect documentation is a little confusing if you just look at the images. At the top of the documentation it says this:

So someone decided the best way to show visual examples of how the Rect class works is to use the orientation that is the exception and not the common one. Though to be fair GUI related stuff is likely the most frequent usage of the Rect class.

The thing to understand about the Rect class, or really anything that defines a position, has no inherent orientation on its own and depends entirely on the orientation of the coordinate system its being used in.

 

After some frustrating debugging, I can also add another problem with rect orientation in ReadPixels:

It would appear that OSX editor and player are reading from the bottom left corner, while Win editor and player are reading from the top left corner (at least in 2020.3.13) - certainly not the most intuitive setup...

 

I know this is an old thread but I ran into this today while using the free aspect resolution settings in 2021.1.12f. It seems if you use that setting you need to reduce the width of your rect by 1 pixel in the X and Y. Otherwise you'll get the read pixel error.

Setting to a standard resolution works without taking -1 from the X, Y. Never come across it before. Not sure the reason. I'll try regression testing it.

 

@bgolus
Hi you seem knowledgeable about this topic I was wondering if you could help assist with my use case if you have the time?
https://forum.unity.com/threads/mos...ned-viewport-coordinate.1192702/#post-7628218

I noticed you wrote about graphics.copytexture to avoid the slow readpixels()

So let’s say I have a camera that only renders the layers from which I want to sample some pixel colors.

can I just make that camera render into a render texture and use that as the source in graphics.copytexture.
So I’d run the function several times per frame to capture several pixels and access their colors.

would graphics.copytexture work on mobile devices? Or is it dependent on hardware?

If you can help clarify that’d be awesome, but if you don’t have time that’s ok I understand.

thanks.