Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question How to get the whole 2d bbox of an asset

Discussion in 'Computer Vision' started by calin-nl, Mar 2, 2022.

  1. calin-nl

    calin-nl

    Joined:
    Sep 26, 2021
    Posts:
    1
    Hi guys, I hope you're doing well.

    I have the following question. Can unity perception output the 2d bbox of an object, without taking into account how much of it is visible? I noticed that the in-built 2d bbox labeler outputs the 2d bbox only around the visible part of an object of interest. However I'm interested in the whole 2d bbox as if the object was not occluded by anything.

    Reason is that we're interested in the visibility percentage of an object: mask area / total 2d bbox area. We need this to filter out annotations in which the object is not visible enough: similar to how human annotators will not annotate everything in a scene.

    I guess my questions are:
    • Can the perception package output the total 2d bbox?
    • Even better is there a way to extract directly the visibility percentage of an object of interest?
    If that's not the case, could I get some guidance on how to write my own custom labeler? Usually in order to get the 2d bbox of an object, I would just have to iterate over its vertices and take the min-max coords. But I don't know whether I have access to this kind of information in the labelling phase.

    Many thanks!
     
  2. StevenBorkman

    StevenBorkman

    Joined:
    Jun 17, 2020
    Posts:
    16
    Hi, sorry it took us a while to respond to your question. Unfortunately, it is not possible to do what you are trying to do with the 2D bounding box labeler. The 2D bounding box labeler uses a custom shader to determine the visible pixels of labeled objects, so it only can use the actual pixels that are rendered to the screen.

    You can take a look at how we have implemented the BoundingBox3D labeler to give you an example of how you can use the geometry of a labeled object in the labeling phase.

    Let me know if you need more info.

    -Steve