Bounding box occluded parts - Perception Package

Hey there,
Is there an option to get bounding boxes that span also the occluded areas of an object (blue box in the image)?

 

Asked the same question on GIthub.
"We presently do not have out-of-box support for this particular use-case of 2D Bounding Boxes in the Perception package."
https://github.com/Unity-Technologies/com.unity.perception/issues/401

 

Before using Unity Computer Vision I actually implemented manually an annotating system and had the opposite problem since my system annotated even occluded areas which is not what was required.

If you need this solution right now, I would say that implementing this yourself is be pretty easy. You could set 8 points as child of your Transform that represent its cubic bounds. Then just before rendering, use Camera.WorldToScreenPoint to get the 2D coordinates of these 8 points, find the 4 extents and write them in a file

Xmin, Ymin--------Xmax,Ymin
|............................................ |
|............................................ |
|............................................ |
Xmin, Ymax--------Xmax,Ymax

 

Thanks @NPSimulation for your response, I was about to suggest the exact same thing. The 3D bounding box labeler calculates a labeled objects bounds in 3D space. Building on top of that, you could take the 8 corners from the 3D bounding box labeler and covert them to screen space, and then find the screen bounding box that holds those points. In fact, the OnVisualize() method demonstrates how to convert the 3D bounding box points to screen space, because we have to do this to draw the boxes on screen. I hope this helps.

 

(Reposting from GitHub discussion for documenting here as well)
Using cubic bounds produces loose bounding boxes for some perspectives.

 

Hi, Can you help how to get the corner points of 3D bounding box?

 

Hi, We have created a tool for Shoe key point detection, issue we are facing is at 2 axis the rotation value calculation with 3d-3d and 3d-2d point with SolvePnP is correct, when 3rd axis (any) have a value, rotation value calculated goes wrong, can you suggest any solution for this. I am attaching the data we are generating