Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Question Accessing EyeTracking Device on Vive Pro Eye

Discussion in 'VR' started by torantie, Jun 2, 2021.

  1. torantie

    torantie

    Joined:
    Mar 28, 2017
    Posts:
    4
    Similar to this thread https://forum.unity.com/threads/accessing-eyetracking-device-on-hololens-2.924350/, I am trying to use the eye tracking API to get the fixation point. When i list the devices, like in the previously mentioned Thread, I get "name:Head Tracking - OpenXR, role:Generic, manufacturer:OpenXR, caracteristics:HeadMounted, TrackedDevice".

    The thing is that i can access the individual eye position and rotation by using:

    Code (CSharp):
    1.         if (TryGetPosition(UnityEngine.XR.CommonUsages.leftEyePosition, XRNode.LeftEye, out var leftEyePosition)
    2.             && TryGetRotation(UnityEngine.XR.CommonUsages.leftEyeRotation, XRNode.LeftEye, out var leftEyeRotation)
    3.             && TryGetPosition(UnityEngine.XR.CommonUsages.rightEyePosition, XRNode.RightEye, out var rightEyePosition)
    4.             && TryGetRotation(UnityEngine.XR.CommonUsages.rightEyeRotation, XRNode.RightEye, out var rightEyeRotation))
    5.         {
    6.             Debug.Log(string.Format("Left Eye Rotation2:{0} , Position2:{1}", leftEyeRotation, leftEyePosition));
    7.             Debug.Log(string.Format("Right Eye Rotation2:{0} , Position2:{1}", rightEyeRotation, rightEyePosition));
    8.         }
    9.  
    10. bool TryGetPosition(InputFeatureUsage<Vector3> inputFeatureUsage, XRNode node, out Vector3 position)
    11.     {
    12.         UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(node);
    13.         if (device.isValid)
    14.         {
    15.             if (device.TryGetFeatureValue(inputFeatureUsage, out position))
    16.                 return true;
    17.         }
    18.         position = default;
    19.         return false;
    20.     }
    21.  
    22.     bool TryGetRotation(InputFeatureUsage<Quaternion> inputFeatureUsage, XRNode node, out Quaternion rotation)
    23.     {
    24.         UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(node);
    25.         if (device.isValid)
    26.         {
    27.             if (device.TryGetFeatureValue(inputFeatureUsage, out rotation))
    28.                 return true;
    29.         }
    30.         rotation = default;
    31.         return false;
    32.     }
    That is why i am wondering why i can still not access the Eyes Struct since TryGetFeatureValue in :
    Code (CSharp):
    1. bool TryGetEyes(out Eyes eyes)
    2.     {
    3.         UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(XRNode.CenterEye);
    4.         if (device.isValid)
    5.         {
    6.             Debug.Log(string.Format("name:{0}, role:{1}, manufacturer:{2}, characteristics:{3}, serialNumber:{4},",
    7.                 device.name, device.role, device.manufacturer, device.characteristics, device.serialNumber));
    8.             if (device.TryGetFeatureValue(UnityEngine.XR.CommonUsages.eyesData, out eyes))
    9.                 return true;
    10.         }
    11.         eyes = default;
    12.         return false;
    13.     }
    always returns false.
    Since there isn't any indication in the documentation (see https://docs.unity3d.com/Manual/xr_input.html under "Accessing eye-tracking data") that this shouldn't work I am kind of confused.

    Is the Eyes Struct currently just not accessible and do I have to work with the individual eye position or am I perhaps missing something else to access the Eyes struct.
     
  2. the_real_apoxol

    the_real_apoxol

    Unity Technologies

    Joined:
    Dec 18, 2020
    Posts:
    467
    Thank you for the detailed information, we will take a look.
     
    torantie likes this.
  3. the_real_apoxol

    the_real_apoxol

    Unity Technologies

    Joined:
    Dec 18, 2020
    Posts:
    467
    Looked into this a bit closer and with OpenXR eye tracking does not work using the CenterEye XRNode as far as I can tell. Eye tracking for openxr requires that the EyeGazeInteraction be added to the interaction profiles and then you can make an action that binds to
    <EyeGaze>/pose
    . This of course only works if the runtime and device you are using supports tracking eyes.
     
  4. torantie

    torantie

    Joined:
    Mar 28, 2017
    Posts:
    4
    Thank you for the suggestion and quick reply. I tried out the binding path but unfortunately with no success. I get left and right eye position, but after thinking about it again those probably have nothing to do with eye tracking.
    Settings wise i activated openXR:
    upload_2021-6-4_20-32-9.png
    Selected these interaction Profiles:
    upload_2021-6-4_20-32-31.png
    I also tried the full position (<EyeGaze>/pose/position) and rotation (<EyeGaze>/pose/rotation) paths:
    upload_2021-6-4_20-41-57.png

    Steam VR uses the SRanipalRuntime so i am currently running that. Since I calibrated the Eye Tracking on my Device I am assuming that my Vive Pro Eye has working Eye Tracking capabilities. I also checked with the device of a colleague and it didn't work on his either. So a device problem is probably unlikely. Not sure what else i can test to be honest. Any other suggestions would be highly appreciated.

    Just to give more context these are the values i get in the XR Interaction Debugger for the OpenXR Head Tracking device (where i am assuming the gaze values should show up).
    upload_2021-6-4_20-51-48.png
     
  5. the_real_apoxol

    the_real_apoxol

    Unity Technologies

    Joined:
    Dec 18, 2020
    Posts:
    467
    If you check your logs you will be able to see which OpenXR extensions were enabled. If the eye gaze extension is not there then it wont work unfortunately. If you want to paste your editor.log (assuming you are running it in the editor) we can take a look. I do not know if the Vive Pro Eye has the proper extensions implemented or not in the steam runtime.
     
  6. the_real_apoxol

    the_real_apoxol

    Unity Technologies

    Joined:
    Dec 18, 2020
    Posts:
    467
    I missed this part, the eye gaze values would show up in their own Eye Gaze device if everything is working correctly, they are not part of the Head device.
     
  7. torantie

    torantie

    Joined:
    Mar 28, 2017
    Posts:
    4
    Here you go. But since it doesn't show up as a separate device like you said it is probably just not supported yet for the Vive
     

    Attached Files:

  8. the_real_apoxol

    the_real_apoxol

    Unity Technologies

    Joined:
    Dec 18, 2020
    Posts:
    467
    I checked with valve and it unfortunately is not. No ETA as of yet as to when it will be.
     
  9. torantie

    torantie

    Joined:
    Mar 28, 2017
    Posts:
    4
    ah what a shame. Thank you for your time though.
     
  10. wilcoboode

    wilcoboode

    Joined:
    Nov 16, 2012
    Posts:
    6
    Good afternoon,

    I stumbled across this threat, and was wondering if there is an update on this topic.
    We are running into issues combining OpenXR & the Tobii SDK usign the HTC Vive Pro Eye, and wondered whether the Vive Pro Eye & OpenXR Gaze-Interactions are already supported.

    We are using Unity 2022 & OpenXR 1.4.x, however we do not get any data, and would like to see if its a problem on our end, or if the device is still not supported.

    Kind Regards
     
    tsepton and vrl4demo like this.
  11. tsepton

    tsepton

    Joined:
    May 12, 2022
    Posts:
    2
    Hi everyone,

    We are also using Unity 2022 with OpenXR, and it seems like eye tracking does not work.
    Any update on the topic ?

    Thanks
     
  12. AxelGogoris

    AxelGogoris

    Joined:
    Jan 8, 2020
    Posts:
    5
    Hej :)

    I know this was 6 months ago but...
    Any update on this?
    Thanks !
     
  13. Virtual_Ross

    Virtual_Ross

    Joined:
    Jan 24, 2023
    Posts:
    1
    Hey everyone, we are also trying to access eye tracking data from the VIVE Pro. We are using the OpenXR plugin and the XR Interaction Toolkit 2.3.x. Also we have been able to calibrate the eye tracking in the Vive dashboard.

    I tried to access the position of the eyes through the EyePosePosition action reference with the binding "pose/rotation [Eye Gaze (OpenXR)]" which is preselected for the EyePosePosition action reference inside the action asset "XRI Default Input Actions" (coming from the Starter Assets from XR Interaction Toolkit) and then simply using the actionReference.action.ReadValue<Vector3>() method.
    Although this concept works when applied to e.g. the controller position, this does not work for the eye tracking data for me.
    Is maybe the Vive OpenXR Plugin required to access the eyetracking? Or is the OpenXR Plugin enough?
    Maybe someone can help us out here.
     
  14. Tanya_Li

    Tanya_Li

    Unity Technologies

    Joined:
    Jun 29, 2020
    Posts:
    109
    One thing to check is if the testing device actually supports eye tracking feature.
    I would recommend checking the player.log first and looking for
    Available Runtime Extensions: 
    and see if XR_EXT_eye_gaze_interaction is listed there.
    So far I know Hololens, Varjo and HTC vive cosmo supported eye gaze extension.
     
    AxelGogoris likes this.
  15. AxelGogoris

    AxelGogoris

    Joined:
    Jan 8, 2020
    Posts:
    5
    Hey!
    Vive Pro Eye does have eye tracking feature (listed below)
    "[XR] [15772] [10:02:06.758][Info ] Available Runtime Extensions: (27)
    [XR] [15772] [10:02:06.758][Info ] XR_EXT_hand_tracking: Version=4
    [XR] [15772] [10:02:06.758][Info ] XR_EXT_eye_gaze_interaction: Version=1"

    Do you know what we should do next? Just like @Virtual_Ross, I've tried a bit of everything but can't make the OpenXR Eye-tracking work :(

    If anyone pass by, feel free to provide any tips!
     
  16. Tanya_Li

    Tanya_Li

    Unity Technologies

    Joined:
    Jun 29, 2020
    Posts:
    109
    if could try enabled Eye Gaze Interaction Profile in the OpenXR setting UI, like this:
    upload_2023-3-17_13-32-59.png

    and then you can make an action that binds to <EyeGaze>/pose.
    Previous post with couple screenshots was the right way to use it.
     
  17. AxelGogoris

    AxelGogoris

    Joined:
    Jan 8, 2020
    Posts:
    5
    Hey!
    You're right, got it working, thanks :)
    But since it's not the gaze that is directly returned (no eyes struct)... I think I'll stick with Vive solutions for now!

    Thanks for your help !
     
  18. Tech-Labs

    Tech-Labs

    Joined:
    Feb 5, 2014
    Posts:
    106
    @AxelGogoris I'm at the same point but still stuck. Could you please explain what you did (and did not?) install in Unity (2022) to get your Pro Eye working with the OpenXR and the Eye Gaze profile?
    The GazeInputManager does not detect eye tracking on my Pro Eye (connected, eye tracking enable, calibrated).
    I tried installing the Vive OpenXR extension, but that did not make any difference.
    I noticed that OpenXR with Eye Gaze Profile does not automatically start SRanipal (it does start SteamVR). Not sure if we still need that though! SRanipal icon stays orange, so it seems that interface is not working without it being called upon fro Unity (using the legacy Vive Unity SDK).

    When I look in my Editor Log I see the XR_EXT_eye_gaze_interaction 'is not supported by runtime'. So something doesn't work....
     
    Last edited: Feb 8, 2024