Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  3. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Question Camera Intrinsic Matrix

Discussion in 'Robotics' started by dsriaditya999, Jul 18, 2021.

  1. dsriaditya999


    Jul 5, 2021
    Hello, I have been trying to implement a Pose Estimation pipeline using the UR3 robot (similar to the Pose Estimation Tutorial). I have been using Perception Package for Data collection and 3D bounding box labeller for storing ground truth. I am trying to use a Pose Estimation Model similar to DOPE ( In this method, we require the 2D ground-truth locations of the 3D Bounding boxes for training the model. Then, while inference, we can use a PnP algorithm (cv2.solvPnP()) to get the pose of the object. This can be later used for pick and place as shown in tutorial.
    (I am a little new to the field to the field of Comp. Vision so please bear with me :))


    I have read the documentation for reading the json dataset ( where it is mentioned we can find the intrinsic matrix of the camera sensor under captures.sensor--camera_intrinsic. I also checked MathWorks reference mentioned in the same webpage. But there seems to be some discrepancy (I have a doubt) in the JSON captures.

    1. I found some negative value in the matrix (See attached example capture file). Can someone explain this? Is the intrinsic matrix of the camera ?

    2. Also, are the parameters of this matrix not expressed in pixels?

    3. I am planning to use cv2.projectPoints(object_vertex_coordinates, rvec, tvec, camera_intrinsic_matrix,..). So, I can get the rvec, tvec, vertex co-ordinates of the bounding box (from the size) in annotations. So, for camera_intrinsic_matrix do I use the captures.sensor.camera_intrinsic (as mentioned) earlier?

    Along with the answers, any resource for understanding more about these topics is also very helpful. Thankyou for your time!

  2. LaurieUnity


    Unity Technologies

    Oct 23, 2020