Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Unity Environmental Occlusion and Depth in ARFoundation

Discussion in 'Handheld AR' started by todds_unity, Jun 24, 2020.

  1. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    Updated: Occlusion and depth is now supported in both ARCore and ARKit.

    The AR Foundation, ARCore, and ARKit packages have been published with version 4.1.0-preview.2. These packages add automatic environment occlusion functionality to the existing AROcclusionManager component. With this component configured in your scene, your virtual content is more integrated into the real world using depth information.

    Generated for each video frame, depth information conveys an image in which each pixel represents the distance between the device and real-world objects. With automatic occlusion configured, the camera background rendering uses this depth image to populate the depth buffer prior to rendering. When the virtual content is rendered, Z-buffer occlusion allows for both the virtual content and real-world object to occlude each other, thus, giving the effect that virtual and real world appear as one.

    For ARCore, Google maintains this list of ARCore compatible devices which notes the devices that support the depth API.

    For ARKit, the new iPad Pro running ARKit 4 with iOS 14 supports automatic environment depth occlusion. Note that Xcode 12 is also required.


    To use automatic occlusion, you simply add the AROcclusionManager component to the AR camera (along with both the ARCameraManager and ARCameraBackground components)

    arfoundation-arocclusionmanager-2020-06-24.png

    The AROcclusionManager has 3 parameters:
    - Environment depth mode
    - Human segmentation stencil mode
    - Human segmentation depth mode

    Human segmentation modes are discussed in this forum post.

    Environment depth mode has four settings:
    - Disabled
    - Fastest
    - Medium
    - Best

    Enable automatic environment occlusion by setting this property to anything other than Disabled. The other settings specify the quality of the depth occlusion to perform. However, each additional level of quality does come with more operations that impact your game's performance, so choose a setting that best meets your game performance needs.

    arfoundation-arocclusion-example.png

    The arfoundation-samples repository also has a SimpleOcclusion scene preconfigured with this setup. In this scene, you tap the screen to fire a red projectile into the scene and watch as it becomes occlude by real-world geometry.
     
    Last edited: Jun 25, 2020
    jbraam and newguy123 like this.
  2. Natzke

    Natzke

    Joined:
    Feb 5, 2017
    Posts:
    2
    Do you need XCode 12 to compile for ARKit4? Still waiting for iOS 14 beta to download for my iPad and haven't come across any development requirements (besides updated packages and device iOS beta)
     
    dimitris_baud likes this.
  3. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    37
    ARCore's depth API is now out of beta. Will the Occlusion Manager be updating to support that on Android at some point?
     
    ROBYER1 and peterfiveeight like this.
  4. peterfiveeight

    peterfiveeight

    Joined:
    Jun 24, 2013
    Posts:
    3
    I was just about to post the exact same question. Thanks for beating me to it! It would be great to get the support for ARCore as soon as possible for android.
     
  5. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    We have released support for both ARCore and ARKit depth. The original forum post has been updated.
     
  6. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    37
    Wonderful thanks!
     
  7. dimitris_baud

    dimitris_baud

    Joined:
    Jan 15, 2018
    Posts:
    21
    For iPad Pro with LiDAR running iOS13.x (not iOS14 Beta) this is not supported?
     
  8. spacemarine

    spacemarine

    Joined:
    Mar 28, 2013
    Posts:
    14
    My Galaxy Tab S6(2019) says "ARCore unsupported device".
    Please fix this. Thanks~
     
  9. DineshPunni

    DineshPunni

    Joined:
    Mar 31, 2020
    Posts:
    1
    I have the same issue with my Samsung Galaxy S8+
     
  10. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    For ARKit, iOS 14 and Xcode 12 are both required for depth to work.
     
    dimitris_baud likes this.
  11. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    You may check Google's list of supported devices for compatibility with ARCore.
     
  12. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    Which version of the ARCore app ("Google Play Services for AR") is installed on your device?
     
  13. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    362
    I'm not understanding this list
    Are all Android devices on that list supported, or only the ones with notes that says "supports depth API"?
     
  14. Cenda

    Cenda

    Joined:
    Jun 3, 2010
    Posts:
    58
    Yes, only with notes "Support depth API" support depth API :) Rest support just AR Core without Depth API.
     
    newguy123 likes this.
  15. dimitris_baud

    dimitris_baud

    Joined:
    Jan 15, 2018
    Posts:
    21
    Does Environmental Occlusion and Depth work out of the box (just with the AROcclusionManager) in URP?
     
  16. weirdmonkey2807

    weirdmonkey2807

    Joined:
    Nov 14, 2018
    Posts:
    1
    what a mess android is. as good as the depth api is, there is about 20 devices that support this out of hundreds of android phones ...
     
    newguy123 likes this.
  17. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    Yes.
     
  18. Tomzai

    Tomzai

    Joined:
    Oct 12, 2015
    Posts:
    3
    Is it possible to use the AROcclusionManager to get access to the environment depth texture without having automatic occlusion? My use cases are -
    - I want to use the depth texture for collision tests only
    - I want to generate my own occlusion from the depth texture so that I can try excluding around planar regions that my game content is sitting on
    - Also occlusion doesn't seem correct when a scale is applied to the AR Session Origin so I want to see if I can work around that
     
  19. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    The ARCameraBackground only looks for the AROcclusionManager on the same game object, the ARCamera.

    If you add the AROcclusionManager to any other game object except for the ARCamera, the AROcclusionManager produces the environment depth texture (if enabled), but the background rendering does not use it for occlusion.
     
  20. Tomzai

    Tomzai

    Joined:
    Oct 12, 2015
    Posts:
    3
    Perfect, thanks! Do you have any tips for converting the values in the depth texture into depth in metres?
     
  21. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    150
    The pixel values are in meters.
     
  22. Tomzai

    Tomzai

    Joined:
    Oct 12, 2015
    Posts:
    3
    Handy - thanks again.
     
unityunity