Question Publish game camera stream via ROS

Hi marco, that would probably look something like this -

Code (CSharp):

1. void SendImage(Camera sensorCamera, string topicName)
2. {
3.   var oldRT = RenderTexture.active;
4.   RenderTexture.active = sensorCamera.targetTexture;
5.   sensorCamera.Render();
6.  
7.   // Copy the pixels from the GPU into a texture so we can work with them
8.   // For more efficiency you should reuse this texture, instead of creating a new one every time
9.   Texture2D camText = new Texture2D(sensorCamera.targetTexture.width, sensorCamera.targetTexture.height);
10.   camText.ReadPixels(new Rect(0, 0, sensorCamera.targetTexture.width, sensorCamera.targetTexture.height), 0, 0);
11.   camText.Apply();
12.   RenderTexture.active = oldRT;
13.  
14.   // Encode the texture as a PNG, and send to ROS
15.   byte[] imageBytes = camText.EncodeToPNG();
16.   var message = new MCompressedImage(new MHeader(), "png", imageBytes);
17.   RosConnection.instance.Send(topicName, message);
18. }

 

Yes, should work in any pipeline. Here's the equivalent code from our Object Pose Estimation demo, which uses the perception package -
https://github.com/Unity-Technologi...orialAssets/Scripts/TrajectoryPlanner.cs#L198