Face parts tracking?

Is there any way in ARFoundation to get the transform of eyes/nose/mouth specifically? A cursory look in the documentation didn't reveal anything.

 

With ARCore subsystem you can get nosetip and forehead (left and right).
With ARKit there was a way to get at least the eyes but seems they depreciated that plugin, as I am asking for the same:
https://forum.unity.com/threads/arkit-eye-tracking-with-arfoundation.691330/

If they enabled eyes would be great, for the nose what I do is check the mesh vertices and find the one that is the most forward, which should be the nose tip.

 

ARCore and ARKit expose this information differently, so you'd need to downcast the XRFaceSubsystem to the ARCore or ARKit implementation to access this platform-specific functionality.

For ARCore's "region poses", see https://docs.unity3d.com/Packages/com.unity.xr.arcore@2.1/manual/index.html#face-tracking. There's a sample called ARCoreFaceRegions.
For ARKit's "blendshapes", see the sample called ARKitFaceBlendShapes.

 

Hey tdmowrer,

How about eye transforms in ARFoundation? Do we have access to those?

In the Unity ARKit Plugin, we had leftEyePose and rightEyePose on the ARFaceAnchor.

Is there an equivalent in ARFoundation or its subsystems?

 

@tdmowrer Thanks for the infos! I was also wondering about the face anchors - is there a way to access the position of eye brow, ear, etc on ARKit via ARFoundation?

 

Yes, it's possible via ARKit Blendshapes. Please see this example scene:
https://github.com/Unity-Technologi...cenes/FaceTracking/ARKitFaceBlendShapes.unity