Unity Environmental Occlusion and Depth in ARFoundation

Updated: Occlusion and depth is now supported in both ARCore and ARKit.

The AR Foundation, ARCore, and ARKit packages have been published with version 4.1.0-preview.2. These packages add automatic environment occlusion functionality to the existing AROcclusionManager component. With this component configured in your scene, your virtual content is more integrated into the real world using depth information.

Generated for each video frame, depth information conveys an image in which each pixel represents the distance between the device and real-world objects. With automatic occlusion configured, the camera background rendering uses this depth image to populate the depth buffer prior to rendering. When the virtual content is rendered, Z-buffer occlusion allows for both the virtual content and real-world object to occlude each other, thus, giving the effect that virtual and real world appear as one.

For ARCore, Google maintains this list of ARCore compatible devices which notes the devices that support the depth API.

For ARKit, the new iPad Pro running ARKit 4 with iOS 14 supports automatic environment depth occlusion. Note that Xcode 12 is also required.


To use automatic occlusion, you simply add the AROcclusionManager component to the AR camera (along with both the ARCameraManager and ARCameraBackground components)



The AROcclusionManager has 3 parameters:
- Environment depth mode
- Human segmentation stencil mode
- Human segmentation depth mode

Human segmentation modes are discussed in this forum post.

Environment depth mode has four settings:
- Disabled
- Fastest
- Medium
- Best

Enable automatic environment occlusion by setting this property to anything other than Disabled. The other settings specify the quality of the depth occlusion to perform. However, each additional level of quality does come with more operations that impact your game's performance, so choose a setting that best meets your game performance needs.



The arfoundation-samples repository also has a SimpleOcclusion scene preconfigured with this setup. In this scene, you tap the screen to fire a red projectile into the scene and watch as it becomes occlude by real-world geometry.

 

We have released support for both ARCore and ARKit depth. The original forum post has been updated.

 

For ARKit, iOS 14 and Xcode 12 are both required for depth to work.

 

You may check Google's list of supported devices for compatibility with ARCore.

 

Which version of the ARCore app ("Google Play Services for AR") is installed on your device?

 

Yes.

 

The ARCameraBackground only looks for the AROcclusionManager on the same game object, the ARCamera.

If you add the AROcclusionManager to any other game object except for the ARCamera, the AROcclusionManager produces the environment depth texture (if enabled), but the background rendering does not use it for occlusion.

 

The pixel values are in meters.