Sending RGB Images from Unity to ROS2 slow performance

Hello,

Is there a way to quickly send color images from a camera in Unity in a ROS2 message? Currently, I am using a rendertexture to capture from a camera, using readpixels to get a texture2d, then using GetRawTextureData to get the raw byte data. ReadPixels is unfortunately very slow, causing lag in the simulation. Is there another way to get the output of a camera out of Unity besides this method?

Code (CSharp):

1. void captureRGGB()
2.     {
3.         if (Time.realtimeSinceStartup > nextTimePeriod)
4.         {        
5.             RenderTexture target = RT;
6.             var temp = RenderTexture.GetTemporary(target.descriptor);
7.             Graphics.Blit(target, temp, new Vector2(1,-1),new Vector2(0,1));
8.             Graphics.Blit(temp, target);
9.             RenderTexture.active = target;
10.             tempTexture = new Texture2D(RT.width,RT.height,TextureFormat.RGB24,false);
11.             tempTexture.ReadPixels( new Rect(0,0,RT.width,RT.height),0,0);
12.             RenderTexture.active = null;
13.             msg.data = tempTexture.GetRawTextureData();
14.             msg.header.stamp.sec =( int)(System.DateTime.UtcNow - new DateTime(1970,1,1)).TotalSeconds;
15.             msg.header.stamp.nanosec = (uint)(System.DateTime.UtcNow - new DateTime(1970,1,1)).Milliseconds*1000*1000;
16.             Destroy(tempTexture);
17.             ros.Publish(topicName,msg);
18.             nextTimePeriod += 1/renderTextureHz;
19.         }
20.      
21.      
22.  
23.     }

 

Hey, did you ever manage to find a way to optimize this?
I also have this problem.

 

Depending on the graphics API you are using, then yes.
https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html
This API is much faster, but does not work with OpenGL.

 

Thanks a lot!
I will look into it