How to convert ARFaceAnchor lookAtPoint to a screen point?

In Unity, I'm trying to take the ARFaceAnchor lookAtPoint on the ARFaceAnchorUpdatedEvent and translate it into a look point on the screen. Like this. Anyone else had luck with this?

Trying to figure out the values:
X is left to right, 1 to -1
Y is top to bottom, -1 to 1
Z is focal length? 0 to -2?

Another solution might be to use the leftEyePose/rightEyePose to calculate the middle point relative to the screen?

UPDATE: I've found other iOS developers doing the same thing. Just need to convert to C# with a similar approach of using the middle point between the eyes look direction that intersects with the camera.

https://github.com/virakri/eye-tracking-ios-prototype/blob/master/Eyes Tracking/ViewController.swift#L189
https://github.com/andrewzimmer906/HeatMapEyeTracking/blob/master/HeatMap/ViewController.swift#L173

 

Hi Dustin, did you managed to convert the point? . Did you tried the project point method ?

 

The project I was working on for this was put on hold, so I stopped working on it for now. But here is some info from @jimmya that might help others:

Talking about the lookAtPoint:

And for converting the lookAtPoint from face coords to world position:

Hope this helps.

 

As for the project point method, here is basically where I was headed with that. Use eye positions to cast a midpoint towards the camera. This wasn't working too well because its very different results based on your distance to the device:

Code (CSharp):

1.  
2.         Vector3 midPoint = Vector3.Lerp(LeftEyeAnchor.transform.position, RightEyeAnchor.transform.position, 0.5f);
3.         Vector3 direction = Vector3.Lerp(-LeftEyeAnchor.transform.forward, -RightEyeAnchor.transform.forward, 0.5f);
4.         Ray ray = new Ray(midPoint, direction);
5.         float distance = Vector3.Distance(midPoint, ARCamera.transform.position);
6.         distance -= ARCamera.nearClipPlane;
7.         Vector lookAtWorldPos = ray.GetPoint(distance);
8.         // convert from world pos to UI
9.  

 

Hi,

I have the same problem. Any improve ? I need to be as accurate as possible.

Thanks!

 

Hi Dustin,
If lookAtPoint is a position relative to the ArFaceAnchor, can we multiply the viewMatrix,faceAnchorTranform and LookAtpoint to get the lookAtPoint position relative to the camera?

I have been stuck with this question and would really appreciate any kind of help
Thanks