Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Unity Perception: Tracking corner positions by perception camera

Discussion in 'Computer Vision' started by titanbender, Jul 31, 2021.

  1. titanbender

    titanbender

    Joined:
    Jul 31, 2021
    Posts:
    1
    The objective is to get the pixel values of the corners of a cube object via unity perception. Here’s what has been setup so far:
    • Perception camera
    • Created 3D game object (‘cube’) with
      • Nested empty game objects under the 3D Object
      • Positioned empty game objects at each corner of the cube
    • The ID label configs have been associated with the KeypointLabeler from the perception camera and the nested objects
    • The simulation runs and captures are saved with corresponding images and json files.
    Attached is an example of the output json file of the simulation.

    Two challenges:

    - The json files doesn’t seem to include the coordinates of the joint labels from the nested objects with joint labels.
    - How can the intrinsic camera be set to 1024 by 1024 pixels? So that is the image output of the simulations.
     

    Attached Files:

  2. StevenBorkman

    StevenBorkman

    Joined:
    Jun 17, 2020
    Posts:
    16
    To be able to use the keypoint labeler with non-humanoid rig keypoints, you will need to define your own keypoint template file and then create joint labels for each of your new nodes to map them to the template file. Our perception Git repo a couple of test scenes here (for URP): https://github.com/Unity-Technologi...ster/TestProjects/PerceptionURP/Assets/Scenes. The 'PoseSample.unity' scene does exactly this.

    In that scene we have a KeypointLabeler specifically to label the box:
    upload_2021-8-2_12-7-3.png

    The BoxKeypointTemplate:
    upload_2021-8-2_12-9-54.png

    Basically all this file really needs is to define the points and to make sure that they are not associated to the rig. If they are not associated to the rig, then the 'Rig Label' field is never read.

    Then we set up our box gameobject like this:
    upload_2021-8-2_12-11-30.png

    In this example, only the 'KeypointCube' has a labeling component. Its children empty gameobjects each have a 'Joint Label' which is used to map them to the template. Here is the Inspector view of the 'FrontTopLeft' gameobject:
    upload_2021-8-2_12-13-32.png

    Just make sure that each one has a 'Joint Label' and the template information for it has a single entry mapping it to the correct template file and keypoint label.

    I hope this helps.
     
    BBraitling and RuiyuZhang_Unity like this.
  3. RuiyuZhang_Unity

    RuiyuZhang_Unity

    Unity Technologies

    Joined:
    Mar 17, 2021
    Posts:
    11
    For the second question:
    - How can the intrinsic camera be set to 1024 by 1024 pixels?

    Currently, the output image size is the same as the resolution of Target Display in the Camera component, which is "Display 1" by default. You can add a custom resolution to it like this:

    Screenshot 2021-08-02 145600.png
     
    BBraitling and StevenBorkman like this.