Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Bug Unable to get correct pixels from RenderTexture in Vulkan

Discussion in 'VR' started by weiner_monkey, Oct 18, 2022.

  1. weiner_monkey

    weiner_monkey

    Joined:
    Aug 1, 2022
    Posts:
    28
    2021.3 LTS (all versions), Quest 2, Vulkan (tested with/without URP)

    Made an empty project, added a new camera with a RenderTexture, and tried to save it to disk. Either get a completely black image or an image with a black line as attached below. The image sometimes shows completely black and sometimes as below randomly, with the black area changing in size.
    Code is simple and straightforward, and I have tried multiple ways, heres an example of the current one:

    Code (CSharp):
    1.     async void RunReadPixel(RenderTexture mainTex)
    2.     {
    3.         RenderTexture.active = mainTex;
    4.  
    5.       // Explicitly set to R8_UNorm on the RenderTexture and here, as mainTex.graphicsFormat gives incorrect format
    6.         // I have also tried multiple other formats and mainTex.graphicsFormat with the default RGBA32
    7.         var bridge = new Texture2D(mainTex.width, mainTex.height, GraphicsFormat.R8_UNorm, TextureCreationFlags.None);
    8.         bridge.ReadPixels(new Rect(0,0,mainTex.width,mainTex.height),0,0);
    9.         RenderTexture.active = null;
    10.         var bytes = bridge.EncodeToJPG();
    11.        
    12.         await File.WriteAllBytesAsync(SOME_PATH_HERE, bytes);
    13.     }
    Has anyone been able to achieve this?
     

    Attached Files:

  2. Shane_Michael

    Shane_Michael

    Joined:
    Jul 8, 2013
    Posts:
    148
    I believe there has been an issue with ReadPixels on Quest/Vulkan for quite a while. Looks like there is already a bug report. Couldn't hurt to vote on it. For now, the only workaround might be to use OpenGL.
     
  3. weiner_monkey

    weiner_monkey

    Joined:
    Aug 1, 2022
    Posts:
    28
    Yes I made this issue, Unity closed it today saying its a texture format mismatch, I tried explicitly setting the format the same on the texture and Textture2D implementation, as well as multiple formats. But it does not work.
     
  4. Shane_Michael

    Shane_Michael

    Joined:
    Jul 8, 2013
    Posts:
    148
    That's too bad. If you want to post a more complete test script (i.e. exactly where and how you are getting the RenderTexture from), I can try it out in my project.
     
  5. weiner_monkey

    weiner_monkey

    Joined:
    Aug 1, 2022
    Posts:
    28
    I made a default one through New > RenderTexture in the assets folder and also tried creating one at runtime so I could get the platform specific format. Both had similar results. The code above is essentially the entire script, the rendertexture settings attached below
    upload_2022-10-18_17-13-24.png

    Note: as you mentioned, don't have this issue when switching to OpenGL
     
    Last edited: Oct 18, 2022
  6. Shane_Michael

    Shane_Michael

    Joined:
    Jul 8, 2013
    Posts:
    148
    I took a look at this and ran into the same issues. Rendering into the render texture worked, saving a texture worked, but the ReadPixels call never retrieves valid data. I seem to only get black results. No combination of RenderTexture creation method or texture formats seem to work (either setting it explicitly or getting it from the RenderTexture).

    I would like to be able to switch to Vulkan on Quest and this is a nice feature to have so I will look into it a bit more. Let me know if you find a way to get this working and I will do the same.
     
  7. weiner_monkey

    weiner_monkey

    Joined:
    Aug 1, 2022
    Posts:
    28
    Yep, all actions on the GPU work fine (Blitting etc), but as soon as you try to get the data to the CPU then you're out of luck.

    Somethings I've tried -> Adding to coroutine and waiting a frame shows the half black image more often. WaitForEndOfFrame usually always shows a black image. Storing Opaque/Depth RenderTexture in URP settings occasionally makes GetRawTextureData get the full image, although the objects appear transparent/incorrect. Load/store also didnt seem to have an effect.

    Late-aquiring swapchain image didnt do anything, neither did decreasing swapchain buffers to 2. I'm not sure which buffer Unity reads from, or how its stored without looking at the source.

    Another point that might be benficial: I have made a custom Render Feature that culls and renders overlay objects, since the default overlay render feature (unsurprisingly) doesn't work at all. The execution timeframe is set to "After Rendering", and the objects on these layers usually show up in the images correctly while the rest of the image is black. Hence I'm quite certain its not format related.
     
  8. Shane_Michael

    Shane_Michael

    Joined:
    Jul 8, 2013
    Posts:
    148
    I didn't have any luck with ReadPixels() specifically; waiting a frame did sometimes capture some of the image but always partially corrupted as if it was grabbed mid-rendering. I did find a way to write out a render texture manually rendering a (disabled in the scene) camera with a CommandBuffer:

    Code (CSharp):
    1.    
    2. void Start()
    3. {
    4.      renderCamera = this.GetComponent<Camera>();
    5.      dynamicRT = new RenderTexture(512, 512, 24, RenderTextureFormat.ARGB32, 0);
    6.      texture = new Texture2D(dynamicRT.width, dynamicRT.height, TextureFormat.ARGB32, false, false);
    7. }
    8.  
    9. void TakeScreenshot()
    10. {
    11.      renderCamera.targetTexture = dynamicRT;
    12.      commandBuffer = new CommandBuffer();
    13.      var data = new NativeArray<byte>(dynamicRT.width * dynamicRT.height * 4, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);              
    14.  
    15.      commandBuffer.RequestAsyncReadbackIntoNativeArray(ref data, dynamicRT, 0, TextureFormat.ARGB32, (readbackRequest) =>
    16.      {
    17.           texture.LoadRawTextureData(data);
    18.           byte[] bytes = texture.EncodeToJPG();
    19.           var path = Application.persistentDataPath + "/test.jpg";
    20.           File.WriteAllBytesAsync(path, bytes);                  
    21.  
    22.           data.Dispose();
    23.           renderCamera.RemoveAllCommandBuffers();
    24.           commandBuffer.Dispose();
    25.      });
    26.  
    27.      renderCamera.AddCommandBuffer(CameraEvent.AfterEverything, commandBuffer);
    28.      renderCamera.Render();
    29. }
    30.  
     
    SimpleAssets likes this.
  9. weiner_monkey

    weiner_monkey

    Joined:
    Aug 1, 2022
    Posts:
    28
    Interesting, I was not aware of RequestAsyncReadbackIntoNativeArray. I will try it out.
     
  10. jakenicolaides

    jakenicolaides

    Joined:
    Sep 28, 2022
    Posts:
    10
    I used this code and it worked perfectly. Thank you for providing it. I am getting three red errors when I use it though:

    AsyncGPUReadback - NativeArray should not be undisposable
    AsyncGPUReadback - NativeArray does not have read/write access
    UnityException: No texture data provided to LoadRawTextureData

    The code works fine and I can live with the errors if necessary, just wondered if you knew a way to resolve them. From my understanding, the first error is because you are creating a persistant allocation for "data" and then not deallocating it in the same scope (it is deallocated inside the readbackRequest). The other two errors I have no idea.

    Appreciate any help with this.
     
  11. Shane_Michael

    Shane_Michael

    Joined:
    Jul 8, 2013
    Posts:
    148
    No, sorry, I haven't looked at this since. It was simply a proof-of-concept for the general method so I stopped once I saw actual textured being written to file on the headset. It can take some experimentation to find the way Unity expects you to interact with some APIs as the documentation itself is pretty skeletal.
     
  12. GamerTobbe

    GamerTobbe

    Joined:
    Apr 4, 2019
    Posts:
    6
    we reported this from our studio in august, it's fixed in 2021.3.12
     
  13. jakenicolaides

    jakenicolaides

    Joined:
    Sep 28, 2022
    Posts:
    10
    Okay thanks for letting me know!
     
  14. jakenicolaides

    jakenicolaides

    Joined:
    Sep 28, 2022
    Posts:
    10
    Ah okay great, will update and see if that fixes. Thanks for letting us know.