XR Interaction Toolkit - Hover event on UI Elements with XRRayInteractor

I'm trying to make a VR game on Oculus Quest with the XR Interaction Toolkit and I would like to configure an haptic event when the user is hovering a UI element. The problem is that the haptic event works when I'm hovering a XRGrabInteractable or a TeleportationArea with the XXRayInteractor but not on a canvas. Even the simple hover event (OnHoverUI on the picture) isn't working on the canvas. It's really weird because I can interact with the UI elements (buttons, sliders, ...).

Here are the parameters of my XRRayInteractor :


I already found a solution working without those parameters but it's really annoying. First, I needed to access the controllers (right and left hands). Then, I added a "Event Trigger" component to every single UI element on which I want a haptic feedback. Those triggers have a "Pointer Enter" event calling the "OnHoverUI" function.

Code (CSharp):

1. void Start()
2. {
3.    var inputDevices = new List<InputDevice>();
4.    InputDevices.GetDevices(inputDevices);
5.    foreach (var device in inputDevices)
6.    {
7.        if (device.role.ToString() == "LeftHanded")
8.        {
9.             leftHand = device;
10.        }
11.        else if (device.role.ToString() == "RightHanded")
12.        {
13.            rightHand = device;
14.        }
15.    }
16. }
17.  
18. public void OnHoverUI()
19. {
20.    if (leftMenuMode) leftHand.SendHapticImpulse(0, 1f, 0.01f);
21.    else if (rightMenuMode) rightHand.SendHapticImpulse(0, 1f, 0.01f);
22. }

I really would like to use the parameters of the XRRayInteractor but they don't work with the UI.

Does someone have an idea why ?

 

I'm having exactly the same problem. Did you find a solution?

 

Nop, the problem is still here but I updated the code I used. "device.role" is deprecated so you have to use characteristics :

Code (CSharp):

1. InputDevice leftHand;
2. InputDevice rightHand;
3.  
4. void Start()
5. {
6.     InputDeviceCharacteristics leftHandCharacteristics = InputDeviceCharacteristics.Left | InputDeviceCharacteristics.HeldInHand | InputDeviceCharacteristics.Controller | InputDeviceCharacteristics.TrackedDevice;
7.     InputDeviceCharacteristics rightHandCharacteristics = InputDeviceCharacteristics.Right | InputDeviceCharacteristics.HeldInHand | InputDeviceCharacteristics.Controller | InputDeviceCharacteristics.TrackedDevice;
8.  
9.     var leftControllers = new List<InputDevice>();
10.     InputDevices.GetDevicesWithCharacteristics(leftHandCharacteristics, leftControllers);
11.     if (leftControllers.Count > 0)
12.     {
13.         if (enableLog) print("Left hand found");
14.         leftHand = leftControllers[0];
15.         leftHand.SendHapticImpulse(0, 0.5f, 1.0f);
16.     }
17.  
18.     var rightControllers = new List<InputDevice>();
19.     InputDevices.GetDevicesWithCharacteristics(rightHandCharacteristics, rightControllers);
20.     if (rightControllers.Count > 0)
21.     {
22.         if (enableLog) print("Right hand found");
23.         rightHand = rightControllers[0];
24.         rightHand.SendHapticImpulse(0, 0.5f, 1.0f);
25.     }
26. }

 

Ok thanks I will give it a try

 

I created a simple script that you can attach to a canvas that can automatically add the event to all child components of the types you tell it to. It also pulls the settings from XRRayInteractor so the vibration is the same. It determines which controller to do the vibration on depending on which XRRayInteractor is active, if you have both active then it might do the vibration on the wrong controller.

Code (CSharp):

1. using System.Collections.Generic;
2. using System.Linq;
3. using UnityEngine;
4. using UnityEngine.EventSystems;
5. using UnityEngine.UI;
6. using UnityEngine.XR.Interaction.Toolkit;
7.  
8. public class AddHapticUI : MonoBehaviour {
9.     private XRController controller1;
10.     private XRController controller2;
11.     private XRRayInteractor interactor1;
12.     private XRRayInteractor interactor2;
13.  
14.     private void Awake() {
15.         List<GameObject> gameObjects = new List<GameObject>();
16.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Button>().Select(x => x.gameObject));
17.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Slider>().Select(x => x.gameObject));
18.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Dropdown>().Select(x => x.gameObject));
19.  
20.         foreach (var item in gameObjects) {
21.             var trigger = item.AddComponent<EventTrigger>();
22.             var e = new EventTrigger.Entry { eventID = EventTriggerType.PointerEnter };
23.             e.callback.AddListener(Hover);
24.             trigger.triggers.Add(e);
25.         }
26.     }
27.  
28.     private void GetControllers() {
29.         if (controller1 == null || controller2 == null) {
30.             var controllers = FindObjectsOfType<XRController>();
31.             if (controllers.Length > 0) {
32.                 controller1 = controllers[0];
33.                 interactor1 = controller1.gameObject.GetComponent<XRRayInteractor>();
34.             }
35.             if (controllers.Length > 1) {
36.                 controller2 = controllers[1];
37.                 interactor2 = controller2.gameObject.GetComponent<XRRayInteractor>();
38.             }
39.         }
40.     }
41.  
42.     private void Hover(BaseEventData arg0) {
43.         GetControllers();
44.  
45.         if (interactor1.enabled) {
46.             controller1.inputDevice.SendHapticImpulse(0, interactor1.hapticHoverEnterIntensity, interactor1.hapticHoverEnterDuration);
47.         } else if (interactor2.enabled) {
48.             controller2.inputDevice.SendHapticImpulse(0, interactor2.hapticHoverEnterIntensity, interactor2.hapticHoverEnterDuration);
49.         }
50.     }
51. }

 

I've (what I think) have improved on the above script:
(1) Minimized the circumstances where the wrong hand vibrates (by following PointerExit events as well and vibrate only on transition from Exit to Enter).
(2) In my case the original script wouldn't vibrate since the Enter event happened before the ray was updated. Added a very short delay between receiving the event and checking the ray. Might be a better solution out there, but this is what I've found.

Code (CSharp):

1.  
2. using System.Collections;
3. using System.Collections.Generic;
4. using System.Linq;
5. using UnityEngine;
6. using UnityEngine.EventSystems;
7. using UnityEngine.UI;
8. using UnityEngine.XR.Interaction.Toolkit;
9. public class AddHapticUI : MonoBehaviour {
10.     private XRController controller1;
11.     private XRController controller2;
12.     private XRRayInteractor interactor1;
13.     private XRRayInteractor interactor2;
14.     private bool interactor1Active = false;
15.     private bool interactor2Active = false;
16.     private void Awake() {
17.         List<GameObject> gameObjects = new List<GameObject>();
18.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Button>().Select(x => x.gameObject));
19.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Slider>().Select(x => x.gameObject));
20.         gameObjects.AddRange(gameObject.GetComponentsInChildren<Dropdown>().Select(x => x.gameObject));
21.         foreach (var item in gameObjects) {
22.             var trigger = item.AddComponent<EventTrigger>();
23.             var e = new EventTrigger.Entry { eventID = EventTriggerType.PointerEnter };
24.             e.callback.AddListener(Hover);
25.             trigger.triggers.Add(e);
26.  
27.             var e2 = new EventTrigger.Entry { eventID = EventTriggerType.PointerExit };
28.             e2.callback.AddListener(Exit);
29.             trigger.triggers.Add(e2);
30.         }
31.     }
32.  
33.     private void GetControllers() {
34.         if (controller1 == null || controller2 == null) {
35.             var controllers = FindObjectsOfType<XRController>();
36.             if (controllers.Length > 0) {
37.                 controller1 = controllers[0];
38.                 interactor1 = controller1.gameObject.GetComponent<XRRayInteractor>();
39.             }
40.             if (controllers.Length > 1) {
41.                 controller2 = controllers[1];
42.                 interactor2 = controller2.gameObject.GetComponent<XRRayInteractor>();
43.             }
44.         }
45.     }
46.     private void Hover(BaseEventData arg0) {
47.         StartCoroutine(OnHoverDelayed());
48.     }
49.  
50.     private IEnumerator OnHoverDelayed() {
51.         yield return new WaitForSeconds(0.02f);
52.  
53.         GetControllers();
54.         if (interactor1.TryGetHitInfo(out Vector3 v1, out Vector3 v2, out int i1, out bool isValid) && isValid && !interactor1Active) {
55.             interactor1Active = true;
56.             controller1.inputDevice.SendHapticImpulse(0, interactor1.hapticHoverEnterIntensity, interactor1.hapticHoverEnterDuration);
57.         }
58.         if (interactor2.TryGetHitInfo(out Vector3 v3, out Vector3 v4, out int i2, out bool isValid2) && isValid2 && !interactor2Active) {
59.             interactor2Active = true;
60.             controller2.inputDevice.SendHapticImpulse(0, interactor2.hapticHoverEnterIntensity, interactor2.hapticHoverEnterDuration);
61.         }
62.     }
63.  
64.     private void Exit(BaseEventData arg0) {
65.         StartCoroutine(OnExitDelayed());
66.     }
67.  
68.     private IEnumerator OnExitDelayed() {
69.         yield return new WaitForSeconds(0.02f);
70.  
71.         GetControllers();
72.         if (!interactor1.TryGetHitInfo(out Vector3 v1, out Vector3 v2, out int i1, out bool isValid) || !isValid) {
73.             interactor1Active = false;
74.         }
75.         if (!interactor2.TryGetHitInfo(out Vector3 v3, out Vector3 v4, out int i2, out bool isValid2) || !isValid2) {
76.             interactor2Active = false;
77.         }
78.     }
79. }
80.  

 

@yoavamit_unity When using your script, I am getting this error - Assets/Scripts/AddHapticUI.cs(62,43): error CS1620: Argument 1 must be passed with the 'ref' keyword in both Coroutine - OnHoverDelayed() and OnExitDelayed(). Could you please help?

 

@komalsharma21294 I believe this is due to the change to action based vs device based xr rigs, with his code working with the prior device based system.

To fix with the action based system, I did the following:
1. change the class

Code (CSharp):

1. private ActionBasedController controller1;
2. private ActionBasedController controller2;

Code (CSharp):

1. var controllers = FindObjectsOfType<ActionBasedController>();

2. change the SendHapticImpulse function calls, it is now called on interactor1 and 2 directly and only takes two arguments

Code (CSharp):

1. controller1.SendHapticImpulse(interactor1.hapticHoverEnterIntensity, interactor1.hapticHoverEnterDuration);

Hope this helps

 

Did you not need the onHoverUi anymore after you made this update? Or is it just assumed that we need it in your updated code ? Thanks in advance man.