Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug hand tracking inconsistent

Discussion in 'VR' started by AnKO-Farsight, Sep 15, 2023.

  1. AnKO-Farsight

    AnKO-Farsight

    Joined:
    Mar 17, 2022
    Posts:
    4
    Hello,

    I'm experiencing issues with hand tracking in editor. I would like to know if someone else had the same issue and solved it.

    The issue is that sometimes, hand tracking using quest and cable on the same project (and sometime during the same editor session) will work and other time won't. I can't figure out why. I did not achieve to find a particular sequence that would trigger the issue.

    The basic sequence I'm doing to use hand tracking :
    1. plug cable and turn quest 'On'
    2. start unity and project through the Hub
    3. turn link mode 'on' in quest
    4. enter play mode

    This resulting in sometime working and some other time not working.
    I tried different orders like started Link before opening project etc.

    I'm using Unity 2021.3.25, I tried using Unity 2022.3.9 and had the issue.
    I'm not using oculus Integration and don't want to. I use directly the XRHand package from Unity.
    Unity is in Stand Alone platform
    OpenXR is the selected Plug-in provider.
    Hand Tracking Subsystem is ticked in the OpenXR section under XR plug-in Management in Project Settings
    I do not have warning message when I enter Play Mode.

    Any idea on what could cause the issue ?

    Additional code info :

    I initialize the script that will sync my hand SkinnedMeshRenderer with the information from XRHandTracking subsystem with this function (called each update) :

    Code (CSharp):
    1.  
    2.         private bool TryEnsureInitialized()
    3.         {
    4.             if (m_Subsystem != null) return true;
    5.  
    6.             m_Subsystem = XRGeneralSettings.Instance?.Manager?.activeLoader?.GetLoadedSubsystem<XRHandSubsystem>();
    7.             if (m_Subsystem == null) return false;
    8.  
    9.             var jointIdNames = new string[XRHandJointID.EndMarker.ToIndex()];
    10.             for (int jointIndex = XRHandJointID.BeginMarker.ToIndex(); jointIndex < XRHandJointID.EndMarker.ToIndex(); ++jointIndex) jointIdNames[jointIndex] = XRHandJointIDUtility.FromIndex(jointIndex).ToString();
    11.  
    12.             m_LeftHandSynchronization = new HandSynchronization(m_LeftHandMesh, m_Origin.transform,jointIdNames);
    13.             m_RightHandSynchronization = new HandSynchronization(m_RightHandMesh, m_Origin.transform, jointIdNames);
    14.  
    15.             m_Subsystem.trackingAcquired += OnTrackingAcquired;
    16.             m_Subsystem.trackingLost += OnTrackingLost;
    17.             m_Subsystem.updatedHands += OnHandsUpdated;
    18.             return true;
    19.         }
    Function that update the skinnedMeshRenderer :

    Code (CSharp):
    1.  
    2.         private void OnHandsUpdated(XRHandSubsystem subSystem, XRHandSubsystem.UpdateSuccessFlags updateSuccessFlags, XRHandSubsystem.UpdateType updateType)
    3.         {
    4.  
    5. #if UNITY_EDITOR
    6.             if (UnityEditor.EditorApplication.isPaused) return;
    7. #endif
    8.  
    9.             bool MatchesFlag(XRHandSubsystem.UpdateSuccessFlags match)
    10.             {
    11.                 return (updateSuccessFlags & match) != XRHandSubsystem.UpdateSuccessFlags.None;
    12.             }
    13.  
    14.             // we have no game logic depending on the Transforms, so early out here
    15.             // (add game logic before this return here, directly querying from
    16.             // m_Subsystem.leftHand and m_Subsystem.rightHand using GetJoint on each hand)
    17.             if (updateType == XRHandSubsystem.UpdateType.Dynamic) return;
    18.  
    19.             //set position
    20.             if (MatchesFlag(XRHandSubsystem.UpdateSuccessFlags.RightHandRootPose)) m_RightHandSynchronization.UpdateRootPose(m_Subsystem.rightHand);
    21.             if (MatchesFlag(XRHandSubsystem.UpdateSuccessFlags.LeftHandRootPose)) m_LeftHandSynchronization.UpdateRootPose(m_Subsystem.leftHand);
    22.  
    23.             if (AreHandsFrozen) return;
    24.  
    25.             //set right hand pose
    26.             if (MatchesFlag(XRHandSubsystem.UpdateSuccessFlags.RightHandJoints))
    27.             {
    28.                 m_RightHandSynchronization.UpdateJoints(m_Subsystem.rightHand);
    29.                 if (m_RightHandMesh.HandSkeleton.HasOverridePose == false) m_RightHandSynchronization.Synchronize();
    30.             }          
    31.  
    32.             //set left hand pose
    33.             if (MatchesFlag(XRHandSubsystem.UpdateSuccessFlags.LeftHandJoints))
    34.             {
    35.                 m_LeftHandSynchronization.UpdateJoints(m_Subsystem.leftHand);
    36.                 if (m_LeftHandMesh.HandSkeleton.HasOverridePose == false) m_LeftHandSynchronization.Synchronize();
    37.             }
    38.            
    39.         }
    UpdateJoints and Synchronize just uses data from XRHand to update the hand SkinnedMeshRenderer

    Any idea ?