Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question ROS2 Depth Camera

Discussion in 'Robotics' started by seifmostafa7347, May 12, 2023.

  1. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I'm trying to use Ros2 Slam toolkit with Unity, this is similar to the Nav2 example that Unity provides , however, I'm trying to use a Depth camera instead of Lidar since it's not currently available in my country.
    to my understanding the Slam toolkit takes in Depth data and converts them into a Laser scan, but I can't find any good documentation on how to send the depth data from unity

    this is the message description I found on ros
    I've also written the script for creating the depth shader and can export the png image of depth
    Code (CSharp):
    1. # This message contains an uncompressed image
    2. # (0, 0) is at top-left corner of image
    3.  
    4. std_msgs/Header header # Header timestamp should be acquisition time of image
    5.                              # Header frame_id should be optical frame of camera
    6.                              # origin of frame should be optical center of cameara
    7.                              # +x should point to the right in the image
    8.                              # +y should point down in the image
    9.                              # +z should point into to plane of the image
    10.                              # If the frame_id here and the frame_id of the CameraInfo
    11.                              # message associated with the image conflict
    12.                              # the behavior is undefined
    13.  
    14. uint32 height                # image height, that is, number of rows
    15. uint32 width                 # image width, that is, number of columns
    16.  
    17. # The legal values for encoding are in file src/image_encodings.cpp
    18. # If you want to standardize a new string format, join
    19. # ros-users@lists.ros.org and send an email proposing a new encoding.
    20.  
    21. string encoding       # Encoding of pixels -- channel meaning, ordering, size
    22.                       # taken from the list of strings in include/sensor_msgs/image_encodings.hpp
    23.  
    24. uint8 is_bigendian    # is this data bigendian?
    25. uint32 step           # Full row length in bytes
    26. uint8[] data          # actual matrix data, size is (step * rows)
     
  2. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    Is your issue the creation of the message itself or the conversion from the shader image to required data in the message?
     
    seifmostafa7347 likes this.
  3. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    It's the conversation part that I'm having troubles with
     
  4. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    It is hard to debug without any code. Feel free to share some if you would like to get help
     
    Last edited: May 15, 2023
    seifmostafa7347 likes this.
  5. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
  6. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    One possible solution to send the distances is reading the texture pixel by pixel and generate an array of distances regarding the color (/ greyscale) of the pixel.
     
  7. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I don't think that the problem is with the shader, the problem is that the data array is too large (I've seen a published example from a package) for simple Row * Column RGBD, that's why I had trouble casting the Image itself to the Message Class

    Code (CSharp):
    1. header:   stamp:     sec: 360     nanosec: 891000000   frame_id: camera_depth_optical_frame height: 480 width: 640 encoding: 16UC1 is_bigendian: 0 step: 1280 data: - 178 - 5 - 182 - 5 - 187 - 5 - 192 - 5 - 196 - 5 - 201 - 5 - 206 - 5 - 212 - 5 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0...
     
  8. seifmostafa7347

    seifmostafa7347

    Joined:
    Nov 2, 2021
    Posts:
    22
    I still can't seem to find any documentation and had no luck figuring it out , if anyone could help in this task or guide me I'd be thankful