# How to convert ARFaceAnchor lookAtPoint to a screen point?

Discussion in 'AR' started by dustin_red, Dec 6, 2018.

1. ### dustin_red

Joined:
Feb 7, 2018
Posts:
46
In Unity, I'm trying to take the ARFaceAnchor lookAtPoint on the ARFaceAnchorUpdatedEvent and translate it into a look point on the screen. Like this. Anyone else had luck with this?

Trying to figure out the values:
X is left to right, 1 to -1
Y is top to bottom, -1 to 1
Z is focal length? 0 to -2?

Another solution might be to use the leftEyePose/rightEyePose to calculate the middle point relative to the screen?

UPDATE: I've found other iOS developers doing the same thing. Just need to convert to C# with a similar approach of using the middle point between the eyes look direction that intersects with the camera.

https://github.com/virakri/eye-tracking-ios-prototype/blob/master/Eyes Tracking/ViewController.swift#L189
https://github.com/andrewzimmer906/HeatMapEyeTracking/blob/master/HeatMap/ViewController.swift#L173

Last edited: Dec 7, 2018
2. ### joseines

Joined:
Dec 28, 2018
Posts:
1
Hi Dustin, did you managed to convert the point? . Did you tried the project point method ?

3. ### dustin_red

Joined:
Feb 7, 2018
Posts:
46
The project I was working on for this was put on hold, so I stopped working on it for now. But here is some info from @jimmya that might help others:

And for converting the lookAtPoint from face coords to world position:
Hope this helps.

4. ### dustin_red

Joined:
Feb 7, 2018
Posts:
46
As for the project point method, here is basically where I was headed with that. Use eye positions to cast a midpoint towards the camera. This wasn't working too well because its very different results based on your distance to the device:
Code (CSharp):
1.
2.         Vector3 midPoint = Vector3.Lerp(LeftEyeAnchor.transform.position, RightEyeAnchor.transform.position, 0.5f);
3.         Vector3 direction = Vector3.Lerp(-LeftEyeAnchor.transform.forward, -RightEyeAnchor.transform.forward, 0.5f);
4.         Ray ray = new Ray(midPoint, direction);
5.         float distance = Vector3.Distance(midPoint, ARCamera.transform.position);
6.         distance -= ARCamera.nearClipPlane;
7.         Vector lookAtWorldPos = ray.GetPoint(distance);
8.         // convert from world pos to UI
9.

5. ### jalemanyf

Joined:
Jun 16, 2013
Posts:
22
Hi,

I have the same problem. Any improve ? I need to be as accurate as possible.

Thanks!

6. ### pranavrajiv

Joined:
Feb 10, 2020
Posts:
1
Hi Dustin,
If lookAtPoint is a position relative to the ArFaceAnchor, can we multiply the viewMatrix,faceAnchorTranform and LookAtpoint to get the lookAtPoint position relative to the camera?

I have been stuck with this question and would really appreciate any kind of help
Thanks

unityunity