Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

Obtaining DepthData while ARKit is active

Discussion in 'AR' started by Blarp, Apr 6, 2019.

  1. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    212
    (I'm using REAR facing camera, iphone XS)

    I'm trying to pull the avdepthdata to analyze particular depth points while ARkit is running. This is so I can coreML obj/pose recognition, and then spatially place a 3d object where the pose/obj was detected.

    I ended up crashing after the first frame, and I'm hearing a lot of people tell me that you can't pull depthdata while arkit is running?

    Related Documentation: https://developer.apple.com/documen...and_media_capture/capturing_photos_with_depth

    Related Sample Proj (without arkit): https://github.com/AtsushiSuzuki/unity-depthcapture-ios

    Related Blog: https://richardstechnotes.com/2019/...dinates-for-openpose-or-anything-else-really/
     
    Last edited: Apr 7, 2019
  2. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    212
  3. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    212
    Last edited: Apr 7, 2019
  4. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    You could try raycast against feature points, and you can probably do a cone cast to be more accurate/get better results. You can also store feature points you have detected in previous frames to build up a depth cloud of your environment.
     
    Blarp likes this.
  5. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    212
    @jimmya

    Thx jim, conecast really brought it together for me.

    Using the technique below and applied colliders to the point clouds. The conecast shoots behind the screenspace detection point. I then filter out a point from those points to use for placement.

    https://github.com/walterellisfun/ConeCast
     
    janla and jimmya like this.
  6. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Great that it worked out for you! And nice find on that github source.
     
    Blarp likes this.
  7. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    212
    Ran into ARFoundation conecast, dropping this in here as a reference:

    public bool Raycast(Ray ray, List<ARRaycastHit> hitResults, TrackableType trackableTypeMask = TrackableType.All, float pointCloudRaycastAngleInDegrees = 5f);

    https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@1.0/manual/index.html#raycasting

     
    Last edited: Apr 12, 2019
  8. blanx

    blanx

    Joined:
    Feb 26, 2016
    Posts:
    60
    Sadly "pointCloudRaycastAngleInDegrees" was dropped in the recent version of Foundation
     
unityunity