Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only.

    Please, do not make any changes to your username or email addresses at id.unity.com during this transition time.

    It's still possible to reply to existing private message conversations during the migration, but any new replies you post will be missing after the main migration is complete. We'll do our best to migrate these messages in a follow-up step.

    On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live.


    Read our full announcement for more information and let us know if you have any questions.

Question Displaying Point Cloud from a Realsense D435 in Unity through ROS

Discussion in 'Robotics' started by GerenMeric, Jun 1, 2023.

  1. GerenMeric

    GerenMeric

    Joined:
    May 15, 2023
    Posts:
    47
    Hello there,

    I have two pcs and i am trying to get live feed from Realsense D435 camera, publish it on ROS from one of the pcs with roslaunch realsene2_camera rs_camera.launch filters:=pointcloud, then, on the other computer using ROS TCP Connector and ROS TCP Endpoint, get ROS messages coming from the other pc into Unity, in Unity with a C# script which subscribes to "camera/depth/color/points" topic and displays the pointcloud in Unity. I tried to write the script. At the beginning of the script, i defined a MeshFilter object and a string which holds the value of the topic we are going to subscribe which is camera/depth/color/points, then in the start function, i assigned GetComponent<MeshFilter>() to the MeshFilter object. Then, i subscribed to the topic with ROSConnection.GetOrCreateInstance().Subscribe<PointCloud2Msg>(topic, UpdatePointCloud) then, in UpdatePointCloud function, i got a object named message of PointCloud2Msg class as the arguement of the function. Then, i tried to get number of points by doing this: (int)(message.data.Length / message.point_step) Then, i created a Vector3 and a Color32 array with size set to number of points i found. Then, i defined two integers vector_index and color_index. Then, looped through all the points and in each step tried to extract color and vectors from message.data using:

    Code (CSharp):
    1. vectors[] = new Vector3(System.BitConverter.ToSingle(message.data, vector_index), System.BitConverter.ToSingle(message.data, vector_index+4), System.BitConverter.ToSingle(message.data, vector_index+8))
    for getting vectors
    and
    Code (CSharp):
    1. colors[] = new Color32(message.data[color_index + 16], message.data[color_index + 17], message.data[color_index + 18], 255)
    for getting the colors.

    Also in each iteration i did += (int)message.point_step for both vector_index and color_index.

    Then, i defined a Mesh object and assigned sharedMesh of the MeshFilter object. Then if mesh is not null, i assign new Mesh() to the Mesh object Then, i set indexFormat of that Mesh object to UnityEngine.Rendering.IndexFormat.UInt32 then assigned sharedMesh of MeshFilter object to Mesh object. Then, finally, i did:

    [/I]
    Code (CSharp):
    1. meshObject.Clear();
    2. meshObject.SetVertices(vectors);
    3. meshObject.SetColors(colors);
    4. meshObject.SetIndices(New Int[0], MeshTopology.Points, 0)

    After writing the code, i created 2 empty GameObjects. To one of them i added ROSConnection script came with ROS TCP Connector package from Unity Robotics Hub. At the other game object i added the ROS subscriber script, a mesh filter and a mesh renderer component. I did't configure both mesh filter and mesh renderer components. So, both mesh field of mesh filter and materials field of mesh renderer is set to none.

    When i press that play button at the top in Unity, i see nothing displayed. But, when i move the D435 camera, i can see that GameObject with ROS subscriber script, mesh filter and mesh renderer component is moving around randomly. Also when i try to print number of points or colors by adding prints and press the play button, Unity just freezes. I wonder if what i did in the code was wrong. Could you help me on this issue? Thanks in advance.

    With kindest regards,

    Meriç Geren
     
    Last edited: Jun 2, 2023
  2. alessandro7111

    alessandro7111

    Joined:
    Jul 2, 2020
    Posts:
    6
  3. GerenMeric

    GerenMeric

    Joined:
    May 15, 2023
    Posts:
    47
    alessandro7111 likes this.