Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question ROS2 Depth Camera

Discussion in 'Robotics' started by seifmostafa7347, May 12, 2023.

  1. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I'm trying to use Ros2 Slam toolkit with Unity, this is similar to the Nav2 example that Unity provides , however, I'm trying to use a Depth camera instead of Lidar since it's not currently available in my country.
    to my understanding the Slam toolkit takes in Depth data and converts them into a Laser scan, but I can't find any good documentation on how to send the depth data from unity

    this is the message description I found on ros
    I've also written the script for creating the depth shader and can export the png image of depth
    Code (CSharp):
    1. # This message contains an uncompressed image
    2. # (0, 0) is at top-left corner of image
    3.  
    4. std_msgs/Header header # Header timestamp should be acquisition time of image
    5.                              # Header frame_id should be optical frame of camera
    6.                              # origin of frame should be optical center of cameara
    7.                              # +x should point to the right in the image
    8.                              # +y should point down in the image
    9.                              # +z should point into to plane of the image
    10.                              # If the frame_id here and the frame_id of the CameraInfo
    11.                              # message associated with the image conflict
    12.                              # the behavior is undefined
    13.  
    14. uint32 height                # image height, that is, number of rows
    15. uint32 width                 # image width, that is, number of columns
    16.  
    17. # The legal values for encoding are in file src/image_encodings.cpp
    18. # If you want to standardize a new string format, join
    19. # ros-users@lists.ros.org and send an email proposing a new encoding.
    20.  
    21. string encoding       # Encoding of pixels -- channel meaning, ordering, size
    22.                       # taken from the list of strings in include/sensor_msgs/image_encodings.hpp
    23.  
    24. uint8 is_bigendian    # is this data bigendian?
    25. uint32 step           # Full row length in bytes
    26. uint8[] data          # actual matrix data, size is (step * rows)
     
  2. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    Is your issue the creation of the message itself or the conversion from the shader image to required data in the message?
     
    seifmostafa7347 likes this.
  3. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    It's the conversation part that I'm having troubles with
     
  4. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    It is hard to debug without any code. Feel free to share some if you would like to get help
     
    Last edited: May 15, 2023
    seifmostafa7347 likes this.
  5. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
  6. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    One possible solution to send the distances is reading the texture pixel by pixel and generate an array of distances regarding the color (/ greyscale) of the pixel.
     
  7. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I don't think that the problem is with the shader, the problem is that the data array is too large (I've seen a published example from a package) for simple Row * Column RGBD, that's why I had trouble casting the Image itself to the Message Class

    Code (CSharp):
    1. header:   stamp:     sec: 360     nanosec: 891000000   frame_id: camera_depth_optical_frame height: 480 width: 640 encoding: 16UC1 is_bigendian: 0 step: 1280 data: - 178 - 5 - 182 - 5 - 187 - 5 - 192 - 5 - 196 - 5 - 201 - 5 - 206 - 5 - 212 - 5 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0...
     
  8. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I still can't seem to find any documentation and had no luck figuring it out , if anyone could help in this task or guide me I'd be thankful