Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question [UNITY ARFOUNDATION + IOS 14.2] FACE MESH IS DETECTED BUT NO EYE DATA/TRANSFORM IS HELP

Discussion in 'AR' started by thomasrafty, Sep 1, 2021.

  1. thomasrafty

    thomasrafty

    Joined:
    May 28, 2020
    Posts:
    6
    Hey guys I bought a macbook air to get my eye rotation but for some reason its not picking up my eyes. When I try to call ARFace.leftEye it just returns null even when a face mesh is detected. How do I turn on Eye tracking? Im using Unity 2020.3.16f1
    I've been following tutorials such as this one and by all means it shows its possible to receive the eye transform at least in the past. Has this been depreciated or something?
     
  2. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,122
    I looked at the face tracking descriptor of ARKitFaceSubsystem:
    Code (CSharp):
    1. [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)]
    2. static void RegisterDescriptor()
    3. {
    4.     if (!UnityARKit_FaceProvider_IsSupported())
    5.         return;
    6.  
    7.     UnityARKit_FaceProvider_OnRegisterDescriptor();
    8.  
    9.     var descriptorParams = new FaceSubsystemParams
    10.     {
    11.         supportsFacePose = true,
    12.         supportsFaceMeshVerticesAndIndices = true,
    13.         supportsFaceMeshUVs = true,
    14.         supportsEyeTracking = UnityARKit_FaceProvider_IsEyeTrackingSupported(),
    15.         id = "ARKit-Face",
    16.         providerType = typeof(ARKitFaceSubsystem.ARKitProvider),
    17.         subsystemTypeOverride = typeof(ARKitFaceSubsystem)
    18.     };
    19.  
    20.     XRFaceSubsystemDescriptor.Create(descriptorParams);
    21. }
    As we see from the code above, the supportsEyeTracking is determined at runtime, so it can be false under some conditions. Unfortunately, UnityARKit_FaceProvider_IsEyeTrackingSupported is a call to a native library, so it's not possible to see why the code may return false. A comment from the AR Foundation team would be very helpful.

    ARKit's documentation also is not helpful, the only documentation I found on face tracking support is this one and it mentions eye tracking only in the context of iOS12:
    @thomasrafty Could you please tell what iOS device do you have and its OS version?
     
  3. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,810
  4. swishchee

    swishchee

    Joined:
    Jul 23, 2020
    Posts:
    61
    I'm getting the same issue, where .leftEye and .rightEye are sporadically null, even when (face manager).description.supportsEyeTracking = true; I've looked at the code samples in AR Foundation Samples and all of the samples assume that the ARFace is a known component on the same GameObject. However, that's precisely the problem I'm having. _arFaceManager.trackables has a total of 3 entries of type ARFace (at indices 0, 1, 2) and only one of them has the .leftEye and .rightEye transforms. Why are there 3? The maximum face count for the face manager is set to 1.

    At other times, I've seen that _faceManager has 2 ARFace entries.

    I do not know how the face manager is determining when to spawn a detected face.
     
    Last edited: Sep 27, 2023