Search Unity

  1. We would like to hear your feedback about Unity and our products. Click here for more information.
    Dismiss Notice

What are the alternatives for ReadPixels or how to optimise its performance?

Discussion in 'Scripting' started by HugoStudica, Jun 12, 2019.

  1. HugoStudica

    HugoStudica

    Joined:
    Aug 25, 2016
    Posts:
    4
    Hey guys! I'm trying to create a simple game about colors and I need to read some points on the screen and check the color these points are hovering. As you can see in the image below the background is b&w and the patterns will change and distort during the game (as crazy as possible). The player has two points and items only one point, this means I will have a maximum of 5 positions that I need to know the color at a time.



    The problem: It works exactly as I want using a RenderTexture from Camera or OnImageRender where I can use ReadPixels from the Texture2D to get the colors, but it costs too much processing for a mobile game.

    Failed solutions:

    1 -
    I tried to select smaller sections of the RenderTexture so it wouldn't need to read all the pixels from RenderTexture, this helped but not enough. Even when using 1px by 1px Rects to select exact pixels. The problem is calling "ReadPixels" multiple times is too expensive, it is better to call it once.

    2 - I tried to select the exact area needed (which is the player size) with only one "ReadPixels", still too expensive.

    3 - I tried to use the "ReadPixels" method on a different thread using Job System or AsyncOperation. I get this exception: "UnityException: get_isReadable can only be called from the main thread". Read Pixels can only be called from the main thread.

    Colliders:

    This is currently the best solution I found and I will do this if there are no other solutions. Using colliders for one color will solve part of the problem. It takes more time to set up colliders depending on the patterns but this is not the main problem which is the organic patterns and distortions.
     
  2. Kurt-Dekker

    Kurt-Dekker

    Joined:
    Mar 16, 2013
    Posts:
    3,694
    Unity isn't really set up to do fiddly pixel stuff like this. The obvious optimizations are going to be what you already tried: only do the ReadPixels once per frame (or once per when the underlying image changes), then read from there.

    Another alternative is to make a separate texture simultaneously using a separate RenderTexture, but one that is much lower-resolution than the insane high resolution of modern mobile devices.

    You should be able to easily do a ReadPixels on something like a 320x480 image, and the user's finger doesn't even have that much resolution in any case. You would never show this low resolution image (except perhaps for debugging), but rather just use it for "picking" the colors.
     
    HugoStudica likes this.
  3. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    6,470
    I recently was in a thread similar to this... and by recently I mean like April:
    https://forum.unity.com/threads/drawing-application-with-unity.667735/#post-4469914

    I suggested using 'GetRawTextureData', this returns a NativeArray that allows reading and writing (and I presume use the jobs system) directly to the texture in RAM. Instead of copying around RAM to modify it then applying back.

    From docs:
    It still requires calling apply at the end to upload your changes into graphics memory.

    https://docs.unity3d.com/ScriptReference/Texture2D.GetRawTextureData.html
     
    HugoStudica likes this.
  4. HugoStudica

    HugoStudica

    Joined:
    Aug 25, 2016
    Posts:
    4
    This is an interesting solution, but I don't have a Texture2D to use. When I get the texture from Camera it is a RenderTexture type, the method that I need to create a Texture2D is precisely the method that is lowering my performance. So, I don't think this will solve my problem specifically, but this is interesting for someone that have an instance of Texture2D.
     
  5. grizzly

    grizzly

    Joined:
    Dec 5, 2012
    Posts:
    217
    Calling ReadPixels will stall the CPU/GPU pipeline which is slow. The alternative solution is to use AsyncGPUReadback instead but at the expense of a few frames latency (depending on the size of the data).

    You can request a subset of data however, so if you calculate the positions on the CPU and feed that into the request(s) you'll be able to minimise the data to just the pixels that you need.
     
    palex-nx likes this.
  6. SparrowsNest

    SparrowsNest

    Joined:
    Apr 6, 2017
    Posts:
    1,649
    If you have a maximum of 5 position I'd assume using GetPixle (singular, https://docs.unity3d.com/ScriptReference/Texture2D.GetPixel.html) is the way to go, it's a lot slower if you need to look at the whole image (at any decent resolution) but in this case (again, I'd assume) it's gonna be faster by not dragging around all the other useless information.
     
  7. grizzly

    grizzly

    Joined:
    Dec 5, 2012
    Posts:
    217
    You can't call GetPixel(s) on a RenderTexture. :)
     
  8. GameDevCouple_I

    GameDevCouple_I

    Joined:
    Oct 5, 2013
    Posts:
    1,539

    This is probably the best solution, at most you will get about 4 frames latency which as long as you are running at 60fps will not be noticeable to the human eye for the most part (unless it happens multiple times in a row).

    To be honest I would combine this with @Kurt-Dekker idea of having a lower resolution render texure that you use for selecting the pixels. But you will lose accuracy, so try at various resolutions. You should be fine using at least a 1/2 size render texture to what the actual screen is, but really can probably go right down to like 1/8 size or something

    EDIT: another option would be to use a quad or something with a mesh renderer for the bit your trying to read and apply your pattern/color as a texture. Instead of using render textures, raycast and get the triangle hit, you can then get the color out from there :)

    Something like what is done in this thread: https://forum.unity.com/threads/trying-to-get-color-of-a-pixel-on-texture-with-raycasting.608431/

    This could be far faster
     
  9. HugoStudica

    HugoStudica

    Joined:
    Aug 25, 2016
    Posts:
    4
    I've followed your solution and I manage to make the game run at 60fps in mid to high devices. But in mid to low devices is still problematic.

    What I did:

    - The Camera is now using a custom target Render Texture
    - This Render Texture has a downscale parameter (I divided the width and height by 10 each) about 100 times fewer pixels.
    - The ReadPixels is extracting for this lower resolution Render Texture.
    - The ReadPixels occurs every 3 frames. (This actually did help because I was running the game by 20 fps and now 60 fps, meaning that I have the same results but with smoother rendering).

    I've lost some precision but It is not noticeable for the player.
    I will try other solutions you guys suggested to me, but this is an improvement.

    Thanks, guys!
     
    palex-nx and Kurt-Dekker like this.
  10. palex-nx

    palex-nx

    Joined:
    Jul 23, 2018
    Posts:
    605
    Have you tried AsyncGPUReadback suggested by grizzly? I'm curious on it's performance on low end devices