Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

How to convert ARFaceAnchor lookAtPoint to a screen point?

Discussion in 'ARKit' started by dustin_red, Dec 6, 2018.

  1. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    In Unity, I'm trying to take the ARFaceAnchor lookAtPoint on the ARFaceAnchorUpdatedEvent and translate it into a look point on the screen. Like this. Anyone else had luck with this?

    Trying to figure out the values:
    X is left to right, 1 to -1
    Y is top to bottom, -1 to 1
    Z is focal length? 0 to -2?

    Another solution might be to use the leftEyePose/rightEyePose to calculate the middle point relative to the screen?

    UPDATE: I've found other iOS developers doing the same thing. Just need to convert to C# with a similar approach of using the middle point between the eyes look direction that intersects with the camera.

    https://github.com/virakri/eye-tracking-ios-prototype/blob/master/Eyes Tracking/ViewController.swift#L189
    https://github.com/andrewzimmer906/HeatMapEyeTracking/blob/master/HeatMap/ViewController.swift#L173
     
    Last edited: Dec 7, 2018
  2. joseines

    joseines

    Joined:
    Dec 28, 2018
    Posts:
    1
    Hi Dustin, did you managed to convert the point? . Did you tried the project point method ?
     
  3. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    The project I was working on for this was put on hold, so I stopped working on it for now. But here is some info from @jimmya that might help others:

    Talking about the lookAtPoint:
    And for converting the lookAtPoint from face coords to world position:
    Hope this helps.
     
  4. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    As for the project point method, here is basically where I was headed with that. Use eye positions to cast a midpoint towards the camera. This wasn't working too well because its very different results based on your distance to the device:
    Code (CSharp):
    1.  
    2.         Vector3 midPoint = Vector3.Lerp(LeftEyeAnchor.transform.position, RightEyeAnchor.transform.position, 0.5f);
    3.         Vector3 direction = Vector3.Lerp(-LeftEyeAnchor.transform.forward, -RightEyeAnchor.transform.forward, 0.5f);
    4.         Ray ray = new Ray(midPoint, direction);
    5.         float distance = Vector3.Distance(midPoint, ARCamera.transform.position);
    6.         distance -= ARCamera.nearClipPlane;
    7.         Vector lookAtWorldPos = ray.GetPoint(distance);
    8.         // convert from world pos to UI
    9.  
     
  5. jalemanyf

    jalemanyf

    Joined:
    Jun 16, 2013
    Posts:
    16
    Hi,

    I have the same problem. Any improve ? I need to be as accurate as possible.

    Thanks!