Search Unity

How to convert ARFaceAnchor lookAtPoint to a screen point?

Discussion in 'ARKit' started by dustin_red, Dec 6, 2018.

  1. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    In Unity, I'm trying to take the ARFaceAnchor lookAtPoint on the ARFaceAnchorUpdatedEvent and translate it into a look point on the screen. Like this. Anyone else had luck with this?

    Trying to figure out the values:
    X is left to right, 1 to -1
    Y is top to bottom, -1 to 1
    Z is focal length? 0 to -2?

    Another solution might be to use the leftEyePose/rightEyePose to calculate the middle point relative to the screen?

    UPDATE: I've found other iOS developers doing the same thing. Just need to convert to C# with a similar approach of using the middle point between the eyes look direction that intersects with the camera.

    https://github.com/virakri/eye-tracking-ios-prototype/blob/master/Eyes Tracking/ViewController.swift#L189
    https://github.com/andrewzimmer906/HeatMapEyeTracking/blob/master/HeatMap/ViewController.swift#L173
     
    Last edited: Dec 7, 2018
  2. joseines

    joseines

    Joined:
    Dec 28, 2018
    Posts:
    1
    Hi Dustin, did you managed to convert the point? . Did you tried the project point method ?
     
  3. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    The project I was working on for this was put on hold, so I stopped working on it for now. But here is some info from @jimmya that might help others:

    Talking about the lookAtPoint:
    And for converting the lookAtPoint from face coords to world position:
    Hope this helps.
     
  4. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    As for the project point method, here is basically where I was headed with that. Use eye positions to cast a midpoint towards the camera. This wasn't working too well because its very different results based on your distance to the device:
    Code (CSharp):
    1.  
    2.         Vector3 midPoint = Vector3.Lerp(LeftEyeAnchor.transform.position, RightEyeAnchor.transform.position, 0.5f);
    3.         Vector3 direction = Vector3.Lerp(-LeftEyeAnchor.transform.forward, -RightEyeAnchor.transform.forward, 0.5f);
    4.         Ray ray = new Ray(midPoint, direction);
    5.         float distance = Vector3.Distance(midPoint, ARCamera.transform.position);
    6.         distance -= ARCamera.nearClipPlane;
    7.         Vector lookAtWorldPos = ray.GetPoint(distance);
    8.         // convert from world pos to UI
    9.  
     
  5. jalemanyf

    jalemanyf

    Joined:
    Jun 16, 2013
    Posts:
    16
    Hi,

    I have the same problem. Any improve ? I need to be as accurate as possible.

    Thanks!