Search Unity

Spatial Mapping on Mobile

Discussion in 'AR/VR (XR) Discussion' started by taylorbrown2686, May 15, 2020.

  1. taylorbrown2686

    taylorbrown2686

    Joined:
    Jun 14, 2019
    Posts:
    27
    Hello Unity Forums,

    I've been researching OpenCV recently, and ran across a lot of programs that map an image depending on what the camera sees. One of these which isn't included, is spatial mapping. I found some API's that support spatial mapping, but they require a HoloLens or NVIDIA card to work.

    My question is, can I get spatial mapping to work on Android/iOS with the phone camera, or is it too expensive of a calculation for a phone to handle? Since ARFoundation has ground plane tracking which can go vertical or horizontal, there must be some way to get spatial mapping on mobile, even if it isn't completely conforming to the terrain. Our use case is occlusion, so if it's not accurate enough to detect anything but walls and the floor, it's better than nothing.

    Let me know if y'all know anything about this topic. Thank you.
     
  2. Aseliot

    Aseliot

    Joined:
    May 7, 2019
    Posts:
    35
    EDIT: Think I found a similar thread for your problem?
    https://forum.unity.com/threads/ar-foundation-occlusion-object-or-shader.594394/

    I am also looking into something similar like this. But the Hololens uses sensors to construct it's spatial map which the phone does not have. As far as I can tell AR Core and the like use some SLAM algorithm(s) to determine planes etc. and most solutions use machine learning for object recognition.

    The best thing you can do is take one (or more) image(s), then somehow take depth from that image and do something with it.

    It depends on your goal wether you could also make use of something like photogrammetry.
    A term I have come across is "depth reconstruction".

    I am trying to place an object in my scene similar to like how it would work on the Hololens.
    I was also wondering if one single depth tested image would be accurate enough to place something, or that I should just use some kind of fixed plane/distance when no plane or any other trackable is hit by the racyast.

    But no as far as I can see it is not possible or easily achievable right now, phone's might not even be physically capable.
     
    Last edited: Jun 10, 2020