Search Unity

Face parts tracking?

Discussion in 'Handheld AR' started by w0nche0l, Jun 9, 2019.

  1. w0nche0l

    w0nche0l

    Joined:
    Feb 21, 2015
    Posts:
    4
    Is there any way in ARFoundation to get the transform of eyes/nose/mouth specifically? A cursory look in the documentation didn't reveal anything.
     
  2. Mephesto_Khaan

    Mephesto_Khaan

    Joined:
    Jun 2, 2013
    Posts:
    25
    With ARCore subsystem you can get nosetip and forehead (left and right).
    With ARKit there was a way to get at least the eyes but seems they depreciated that plugin, as I am asking for the same:
    https://forum.unity.com/threads/arkit-eye-tracking-with-arfoundation.691330/

    If they enabled eyes would be great, for the nose what I do is check the mesh vertices and find the one that is the most forward, which should be the nose tip.
     
  3. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    518
    ARCore and ARKit expose this information differently, so you'd need to downcast the XRFaceSubsystem to the ARCore or ARKit implementation to access this platform-specific functionality.

    For ARCore's "region poses", see https://docs.unity3d.com/Packages/com.unity.xr.arcore@2.1/manual/index.html#face-tracking. There's a sample called ARCoreFaceRegions.
    For ARKit's "blendshapes", see the sample called ARKitFaceBlendShapes.
     
  4. JBMS

    JBMS

    Joined:
    Sep 30, 2017
    Posts:
    6
    Hey tdmowrer,

    How about eye transforms in ARFoundation? Do we have access to those?

    In the Unity ARKit Plugin, we had leftEyePose and rightEyePose on the ARFaceAnchor.

    Is there an equivalent in ARFoundation or its subsystems?