Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ARKit - Access rgb frame and depth map from Unity

Discussion in 'AR/VR (XR) Discussion' started by PROTOCUBE_JOB, Jan 8, 2020.

  1. PROTOCUBE_JOB

    PROTOCUBE_JOB

    Joined:
    Oct 22, 2014
    Posts:
    3
    Hi,

    I would like to access the RGB camera frame and the truedepth camera using ARKit.

    I know that in native it is possible to access the current frame, inside which we can find the 2 buffers related to the rgb image and the depth.

    I searched if there was a solution through ARFoundation but I did not find anything, can someone help me?

    Among the many attempts I also tried to take inspiration from this plugin

    https://github.com/AtsushiSuzuki/unity-depthcapture-ios

    which accesses the 2 frames through the use of AVSession and convert it by starting an ARKit session.
    I did it in part but the moment I pass the buffer to Unity and load it into the texture with this method

    https://docs.unity3d.com/ScriptReference/Texture2D.LoadRawTextureData.html

    I get the error "No texture data provided to LoadRawTextureData" even if I'm sure the starting data exists.
    If anyone is interested, you can write to me and I will gladly send you the unity and xcode project with the plugin.

    Thanks a lot.
    Giovanni
     
  2. ts352

    ts352

    Joined:
    Aug 1, 2019
    Posts:
    1
    Hi, I'm looking to do the same thing, obtaining the depth map (from the rear dual camera) from within an ARKit session. Did you ever get this working?

    Any help would be really appreciated. Thank you!
    Tim
     
  3. herra_lehtiniemi

    herra_lehtiniemi

    Joined:
    Feb 12, 2017
    Posts:
    133
    I’m also wondering if you managed to get the depth map somehow?