Bug Unable to get correct pixels from RenderTexture in Vulkan

2021.3 LTS (all versions), Quest 2, Vulkan (tested with/without URP)

Made an empty project, added a new camera with a RenderTexture, and tried to save it to disk. Either get a completely black image or an image with a black line as attached below. The image sometimes shows completely black and sometimes as below randomly, with the black area changing in size.
Code is simple and straightforward, and I have tried multiple ways, heres an example of the current one:

Code (CSharp):

1.     async void RunReadPixel(RenderTexture mainTex)
2.     {
3.         RenderTexture.active = mainTex;
4.  
5.       // Explicitly set to R8_UNorm on the RenderTexture and here, as mainTex.graphicsFormat gives incorrect format
6.         // I have also tried multiple other formats and mainTex.graphicsFormat with the default RGBA32
7.         var bridge = new Texture2D(mainTex.width, mainTex.height, GraphicsFormat.R8_UNorm, TextureCreationFlags.None);
8.         bridge.ReadPixels(new Rect(0,0,mainTex.width,mainTex.height),0,0);
9.         RenderTexture.active = null;
10.         var bytes = bridge.EncodeToJPG();
11.        
12.         await File.WriteAllBytesAsync(SOME_PATH_HERE, bytes);
13.     }

Has anyone been able to achieve this?

 

I believe there has been an issue with ReadPixels on Quest/Vulkan for quite a while. Looks like there is already a bug report. Couldn't hurt to vote on it. For now, the only workaround might be to use OpenGL.

 

Yes I made this issue, Unity closed it today saying its a texture format mismatch, I tried explicitly setting the format the same on the texture and Textture2D implementation, as well as multiple formats. But it does not work.

 

That's too bad. If you want to post a more complete test script (i.e. exactly where and how you are getting the RenderTexture from), I can try it out in my project.

 

I made a default one through New > RenderTexture in the assets folder and also tried creating one at runtime so I could get the platform specific format. Both had similar results. The code above is essentially the entire script, the rendertexture settings attached below


Note: as you mentioned, don't have this issue when switching to OpenGL

 

I took a look at this and ran into the same issues. Rendering into the render texture worked, saving a texture worked, but the ReadPixels call never retrieves valid data. I seem to only get black results. No combination of RenderTexture creation method or texture formats seem to work (either setting it explicitly or getting it from the RenderTexture).

I would like to be able to switch to Vulkan on Quest and this is a nice feature to have so I will look into it a bit more. Let me know if you find a way to get this working and I will do the same.

 

Yep, all actions on the GPU work fine (Blitting etc), but as soon as you try to get the data to the CPU then you're out of luck.

Somethings I've tried -> Adding to coroutine and waiting a frame shows the half black image more often. WaitForEndOfFrame usually always shows a black image. Storing Opaque/Depth RenderTexture in URP settings occasionally makes GetRawTextureData get the full image, although the objects appear transparent/incorrect. Load/store also didnt seem to have an effect.

Late-aquiring swapchain image didnt do anything, neither did decreasing swapchain buffers to 2. I'm not sure which buffer Unity reads from, or how its stored without looking at the source.

Another point that might be benficial: I have made a custom Render Feature that culls and renders overlay objects, since the default overlay render feature (unsurprisingly) doesn't work at all. The execution timeframe is set to "After Rendering", and the objects on these layers usually show up in the images correctly while the rest of the image is black. Hence I'm quite certain its not format related.

 

I didn't have any luck with ReadPixels() specifically; waiting a frame did sometimes capture some of the image but always partially corrupted as if it was grabbed mid-rendering. I did find a way to write out a render texture manually rendering a (disabled in the scene) camera with a CommandBuffer:

Code (CSharp):

1.    
2. void Start()
3. {
4.      renderCamera = this.GetComponent<Camera>();
5.      dynamicRT = new RenderTexture(512, 512, 24, RenderTextureFormat.ARGB32, 0);
6.      texture = new Texture2D(dynamicRT.width, dynamicRT.height, TextureFormat.ARGB32, false, false);
7. }
8.  
9. void TakeScreenshot()
10. {
11.      renderCamera.targetTexture = dynamicRT;
12.      commandBuffer = new CommandBuffer();
13.      var data = new NativeArray<byte>(dynamicRT.width * dynamicRT.height * 4, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);              
14.  
15.      commandBuffer.RequestAsyncReadbackIntoNativeArray(ref data, dynamicRT, 0, TextureFormat.ARGB32, (readbackRequest) =>
16.      {
17.           texture.LoadRawTextureData(data);
18.           byte[] bytes = texture.EncodeToJPG();
19.           var path = Application.persistentDataPath + "/test.jpg";
20.           File.WriteAllBytesAsync(path, bytes);                  
21.  
22.           data.Dispose();
23.           renderCamera.RemoveAllCommandBuffers();
24.           commandBuffer.Dispose();
25.      });
26.  
27.      renderCamera.AddCommandBuffer(CameraEvent.AfterEverything, commandBuffer);
28.      renderCamera.Render();
29. }
30.  

 

Interesting, I was not aware of RequestAsyncReadbackIntoNativeArray. I will try it out.

 

I used this code and it worked perfectly. Thank you for providing it. I am getting three red errors when I use it though:

AsyncGPUReadback - NativeArray should not be undisposable
AsyncGPUReadback - NativeArray does not have read/write access
UnityException: No texture data provided to LoadRawTextureData

The code works fine and I can live with the errors if necessary, just wondered if you knew a way to resolve them. From my understanding, the first error is because you are creating a persistant allocation for "data" and then not deallocating it in the same scope (it is deallocated inside the readbackRequest). The other two errors I have no idea.

Appreciate any help with this.

 

No, sorry, I haven't looked at this since. It was simply a proof-of-concept for the general method so I stopped once I saw actual textured being written to file on the headset. It can take some experimentation to find the way Unity expects you to interact with some APIs as the documentation itself is pretty skeletal.

 

we reported this from our studio in august, it's fixed in 2021.3.12

 

Okay thanks for letting me know!

 

Ah okay great, will update and see if that fixes. Thanks for letting us know.