Search Unity

How to get world pixel position from AREnvironmentDepth image

Discussion in 'AR' started by Misnomer, Sep 22, 2020.

  1. Misnomer

    Misnomer

    Joined:
    Jul 15, 2013
    Posts:
    28
    Hello, I am trying to create a dense point cloud in Unity using iPad's Lidar scanner. To do so I need to create points at correct world position using depth image. I was able to obtain it using AROcclusionManager.TryGetEnvironmentDepthImage. Next I tried to use built-in Unity's methods like Camera.ScreenToWorldPoint, iterating through each pixel and sampling depth texture to get z value. That doesn't seem to produce correct results. Also I tried copying apple's shader code with a bunch of matrix transformations using native camera's view and projection matrices. That also didn't help.

    I would be very grateful if anyone can outline general steps or point to recources explaining how to produce a world point using camera's position and depth information. Bear in mind that depth is in meters and depth texture resolution is very small (256 x 190)