Search Unity

Bug (Case 1407649) OpenXR - Device disconnected events not firing?

Discussion in 'VR' started by ROBYER1, Mar 1, 2022.

  1. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    This seems to have been an issue for over 2 years now, when using Unity OpenXR package 1.3.1 currently, when using InputDevices.deviceDisconnected, nothing fires when XR devices are disconnected during play mode. When using InputDevices.deviceConnected, despite the connected events all firing at start of play mode for all known devices, the connected events fire perfectly in play mode when devices are connected thereafter. Similarly, in Window > Analysis > Input Debugger, you can see the OpenXR controller devices stay active there too and don't disconnect properly.

    To clarify, I am using SteamVR for OpenXR and testing with a lighthouse setup of Vive Cosmos + Faceplate + Vive Controller + Base Station

    Tiny repro project: https://drive.google.com/file/d/1MvNI6q0sJupliWSFgLZwZlEvKSrylxUl/view?usp=sharing

    Sample code to easily see this:

    Code (CSharp):
    1. using System.Collections.Generic;
    2. using UnityEngine;
    3. using UnityEngine.XR;
    4.  
    5. public class Test : MonoBehaviour
    6. {
    7.     void OnEnable()
    8.     {
    9.         List<InputDevice> allDevices = new List<InputDevice>();
    10.         InputDevices.GetDevices(allDevices);
    11.         foreach (InputDevice device in allDevices)
    12.         {
    13.             InputDevices_deviceConnected(device);
    14.         }
    15.  
    16.         InputDevices.deviceConnected += InputDevices_deviceConnected;
    17.         InputDevices.deviceDisconnected += InputDevices_deviceDisconnected;
    18.     }
    19.     private void InputDevices_deviceConnected(InputDevice device)
    20.     {
    21.         Debug.Log("Device Connected: " + device.name);
    22.     }
    23.     private void OnDisable()
    24.     {
    25.         InputDevices.deviceConnected -= InputDevices_deviceConnected;
    26.         InputDevices.deviceDisconnected -= InputDevices_deviceDisconnected;
    27.     }
    28.     private void InputDevices_deviceDisconnected(InputDevice device)
    29.     {
    30.         Debug.Log("Device disconnected: " + device.name);
    31.     }
    32.  
    33. }
    34.  
    I made a vid of it here:
     
    Last edited: Mar 1, 2022
  2. thep3000

    thep3000

    Unity Technologies

    Joined:
    Aug 9, 2013
    Posts:
    400
    My understanding of the openxr spec is that there are no device connected / disconnected events. What we have unity's device connected/disconnected events hooked up to is when actions are suggested for a device, we consider it connected. When the session is destroyed, it's disconnected.

    I think the only thing you can check that is somewhat equivalent is if the path "isActive".. which corresponds to unity bindings `isTracked` and `trackingState`.

    https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#_reading_input_action_state
     
  3. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Makes sense, I made a rudimentary setup using those already and waiting a few seconds to register a device as disconnected myself.
     
  4. thep3000

    thep3000

    Unity Technologies

    Joined:
    Aug 9, 2013
    Posts:
    400
    sorry it took so long to respond to this one - I lumped it with the vive tracker issue and I wasn't able to repro that until I got the right hardware to pair 2 controllers + vive tracker.
     
    ROBYER1 likes this.
  5. Sadeiko

    Sadeiko

    Joined:
    Dec 11, 2020
    Posts:
    2
    Same behaviour in my projects when having OpenXR checked inside the "XR Plugin Management" section. Yet, I think it's still a bit confusing because talking about e.g. the Unity VarjoOpenXR SDK, the connected/disconnected state works as mentioned (when turning the controller on/off). I know this is not the right forum to address vendor behaviour but I wanted to add it for completion sake.

    Anyway: Is there possibly another way to check the "turning on/off" event of Input Devices without the need to use "isActive" as it would require a constant check in the Update() method?
     
    ROBYER1 likes this.
  6. thep3000

    thep3000

    Unity Technologies

    Joined:
    Aug 9, 2013
    Posts:
    400
    There is ambiguity in the OpenXR spec and some vendors are acting differently than other vendors wrt controller disconnect. I started a thread with the OpenXR working group a few weeks ago to make a decision one way or the other - discussions are in progress and when decision is made vendors will fix their runtimes (and ideally we'll add some sort of conformance test so runtimes don't differ again in the future). For now, unfortunately - this behavior differs between runtimes.
     
    Sadeiko and ROBYER1 like this.
  7. Sadeiko

    Sadeiko

    Joined:
    Dec 11, 2020
    Posts:
    2
    Sounds promising - thank you for sticking with this topic!
     
  8. thep3000

    thep3000

    Unity Technologies

    Joined:
    Aug 9, 2013
    Posts:
    400
    The OpenXR working group is wondering potential use cases for controller disconnect? They're wondering if it would be better if the runtime handles it - ie: pauses your app automatically and presents its own instructions on what happened. Are there cases where you'd prefer your app to have control?
     
  9. colinleet

    colinleet

    Joined:
    Nov 20, 2019
    Posts:
    189
    I'm not sure to what extent this would involve short interruptions, like controllers being lost from view, but I would always want to get a consistent notification of what happened when tracking is lost at least.

    One thing I decided early on was that my game's inputs (runtime provided info) should always visually match-up with the game view. So that player can always knows what the runtime (or hardware) is doing/seeing/or is failing to do depending on the context; basically always providing some kind of correct visual matchup for the player's proprioception. Knowing when the controllers drop out, or are not tracked, especially for very temporary occurrence (like when the controller was held back too far and wasn't visible from headset-based cameras) is a key part of that for me. To me it seemed less jarring to have the controller disappear then reappear rather than suddenly be held still at it's last known location (half a second ago).

    Pausing the app isn't always a possibility, especially with multiplayer. I'm not sure I would want an externally mandated game-pause to suddenly stop the core action of my game, like to the point of no longer feeding the still tracked telemetry -- e.g. stopping all of my player's movements from being mirrored out to their networked avatars.
     
  10. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    In my use case this is for the use of Vive trackers, if one disconnects during a session where someone uses 6+ vive trackers, it is a bit of a problem. Also, for general VR controllers, if we don't know the controller has disconnected, we can't pause the game or show a warning.
     
  11. thep3000

    thep3000

    Unity Technologies

    Joined:
    Aug 9, 2013
    Posts:
    400
    The Vive tracker extension has explicit language about controller connect / disconnect - it looks like this is hooked up properly on the disconnect side. Are you hitting issues not knowing when vive trackers are disconnected?

    For regular controllers, multiplayer is a good use-case I can bring up with the group. Also I can imagine some games having fall-back controls to PC mouse / keyboard, and you wouldn't want your game to pause if the tracked controller was disconnected.
     
  12. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Unless anything changed in the OpenXR Package in Packman or the Vive Tracker extension has been added to it in Unity which it hasn't, I still don't get any controller disconnect events for the Vive trackers through OpenXR - I even released an Asset Store asset I've been working on for ages which really could use it - https://assetstore.unity.com/packages/tools/animation/openxr-vr-motion-capture-r1tools-214497
     
  13. Dr-Nick

    Dr-Nick

    Joined:
    Sep 9, 2013
    Posts:
    25
    As mentioned by @thep3000 using the Unity InputSystem InputActions isTrackedProperty for each controller has worked for me.

    upload_2022-7-27_16-3-54.png

    Then assign function calls to the input actions performed and canceled events respectively to handle controllers being tracked and untracked. If you still need to get controller information, you can still run
    InputDevices.GetDeviceAtXRNode to poll the controller in the left and right hands respectively.

    Code (CSharp):
    1.        
    2.  
    3.         [SerializeField] InputAction m_leftControllerTracked = new InputAction();
    4.         [SerializeField] InputAction m_rightControllerTracked = new InputAction();
    5.  
    6.         private void Start()
    7.         {
    8.             m_leftControllerTracked.performed += LeftControllerStateUpdate;
    9.             m_leftControllerTracked.canceled += LeftControllerStateUpdate;
    10.  
    11.             m_rightControllerTracked.performed += RightControllerStateUpdate;
    12.             m_rightControllerTracked.canceled += RightControllerStateUpdate;
    13.  
    14.             m_leftControllerTracked.Enable();
    15.             m_rightControllerTracked.Enable();
    16.         }
     
    ROBYER1 likes this.