Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  3. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Question Camera Intrinsic Matrix

Discussion in 'Robotics' started by dsriaditya999, Jul 18, 2021.

  1. dsriaditya999

    dsriaditya999

    Joined:
    Jul 5, 2021
    Posts:
    3
    Backstory:
    Hello, I have been trying to implement a Pose Estimation pipeline using the UR3 robot (similar to the Pose Estimation Tutorial). I have been using Perception Package for Data collection and 3D bounding box labeller for storing ground truth. I am trying to use a Pose Estimation Model similar to DOPE (https://arxiv.org/abs/1809.10790). In this method, we require the 2D ground-truth locations of the 3D Bounding boxes for training the model. Then, while inference, we can use a PnP algorithm (cv2.solvPnP()) to get the pose of the object. This can be later used for pick and place as shown in tutorial.
    (I am a little new to the field to the field of Comp. Vision so please bear with me :))

    Issue:

    I have read the documentation for reading the json dataset (https://docs.unity3d.com/Packages/c...6/manual/Schema/Synthetic_Dataset_Schema.html) where it is mentioned we can find the intrinsic matrix of the camera sensor under captures.sensor--camera_intrinsic. I also checked MathWorks reference mentioned in the same webpage. But there seems to be some discrepancy (I have a doubt) in the JSON captures.

    1. I found some negative value in the matrix (See attached example capture file). Can someone explain this? Is the intrinsic matrix of the camera ?

    2. Also, are the parameters of this matrix not expressed in pixels?

    3. I am planning to use cv2.projectPoints(object_vertex_coordinates, rvec, tvec, camera_intrinsic_matrix,..). So, I can get the rvec, tvec, vertex co-ordinates of the bounding box (from the size) in annotations. So, for camera_intrinsic_matrix do I use the captures.sensor.camera_intrinsic (as mentioned) earlier?

    Along with the answers, any resource for understanding more about these topics is also very helpful. Thankyou for your time!

     
  2. LaurieUnity

    LaurieUnity

    Unity Technologies

    Joined:
    Oct 23, 2020
    Posts:
    77