Interacting with the UI using the new XR Input System?

In previous versions if you wanted to interact with the UI while in VR you'd have to either import one of the 3rd party toolkits that extended the event system or add colliders to everything and physics raycast to determine UI targets.

How would one interact with worldspace UI components using the new XR Input solution that unity rolled out in 2019.2?

Edit: Specifically the "interim/Legacy/New, but being replaced" version? that uses InputDevices.GetDevices and not the XR Manager version.

 

We just created a new subclass of BaseInput and hooked up our XR pointer to act like a mouse. From there, all of UGUI "just works". It takes surprisingly little time to implement once you get into it.

 

Could you extrapolate a bit more as I've managed to get the canvas to respond to events using functionality from within standalone input module, but canvas children such as buttons aren't being returned by the RayCastAll.

I've added a camera to my laserPointer (controller) which is mapped to the current active worldspace UI as needed.

Code (CSharp):

1.  
2. public Camera laserEvtCamera;
3. private PointerEventData ped;
4. private List<RaycastResult> results;
5.  
6. //Called per frame after controller position&Rotation has been updated.
7. public HitScanData PerformHitScan() {
8.     //Set active UI to respond to events from this event camera
9.     if (UIManager.Instance.currentUI != null) {
10.         UIManager.Instance.currentUI.SetWorldCamera(laserEvtCamera);
11.     }
12.  
13.     if (ped == null) {
14.         ped = new PointerEventData(EventSystem.current);
15.     }
16.  
17.     //Fake a pointer always being at the center of the camera
18.     ped.position = new Vector3(laserEvtCamera.pixelWidth / 2f, laserEvtCamera.pixelHeight / 2f);
19.     ped.delta = Vector2.zero;
20.  
21.     results = new List<RaycastResult>();
22.     EventSystem.current.RaycastAll(ped, results);
23.  
24.     ped.pointerCurrentRaycast = FindClosestResult(results);
25.  
26.     results.Clear();
27.     return new HitScanData(ped, gameObject);
28. }
29.  
30. //processes the result of the hitscan call
31. protected virtual void ProcessHitscanResult(HitScanData result){
32.     pEvtdata = result.pointerEventData;
33.     GameObject newFocus = result.pointerEventData.pointerCurrentRaycast.gameObject;
34.  
35.     // if no target / pointerEnter deleted, send exit events to anything we are tracking then exit
36.     if (newFocus == null || pEvtdata.pointerEnter == null) {
37.         for (var i = 0; i < pEvtdata.hovered.Count; ++i) {
38.             Broadcast_Focus(pEvtdata.hovered[i], pEvtdata, false);
39.         }
40.  
41.         pEvtdata.hovered.Clear();
42.  
43.         if (newFocus == null) {//not sure why pointer press gets a click event if there's no new focus
44.             Broadcast_Click(pEvtdata.pointerPress, pEvtdata);
45.             pEvtdata.pointerEnter = null;
46.             return;
47.         }
48.     }
49.  
50.     //If we have not changed hover target
51.     if (pEvtdata.pointerEnter == newFocus && newFocus) {
52.         return;
53.     }
54.  
55.     GameObject commonRoot = FindCommonRoot(pEvtdata.pointerEnter, newFocus);
56.  
57.     //if we already an entered object from last cycle
58.     if (pEvtdata.pointerEnter != null) {
59.  
60.         //Send exit handler call to all elements in the chain until we reach the new target, or null!
61.         GameObject enteredGo = pEvtdata.pointerEnter;
62.         while (enteredGo != null){
63.             //If we reach the common root break out!
64.             if (commonRoot != null && commonRoot.transform == enteredGo.transform) {
65.                 break;
66.             }
67.  
68.             Broadcast_Focus(enteredGo, pEvtdata, false);
69.  
70.             pEvtdata.hovered.Remove(enteredGo);
71.             enteredGo = enteredGo.transform.parent.gameObject;
72.         }
73.     }
74.  
75.     //Issue the enter call up to but not including the common root
76.     pEvtdata.pointerEnter = newFocus;
77.     if (newFocus != null) {
78.         GameObject focusedGo = newFocus;
79.  
80.         while (focusedGo != null && focusedGo != commonRoot) {
81.             Broadcast_Focus(focusedGo, pEvtdata, true);
82.            
83.             pEvtdata.hovered.Add(focusedGo);
84.             focusedGo = focusedGo.transform.parent == null ? null : focusedGo.transform.parent.gameObject;
85.         }
86.     }
87. }
88.  
89.  

 

I don't have time to reason about the problems with the code you posted at the moment, but that looks very complicated. Unless you have other requirements driving it, you don't need a Camera for this, nor your own custom uses of raycasting. Just implementing a subclass of BaseInput and then driving some kind of reticle in your world (via normal Xr input), then mapping that reticle's position on the "screen" to what BaseInput thinks the "mouse" position is, makes everything else just work. It's not much code.

 

No problem, As it turned out i was 90% of the way there, just didn't fully/correctly configure the parameters of the pointerEventData object. I ended up porting functionality from pointerInputModule, and BaseInputModule into my ProcessHitscanResult method to give me the desired functionality.

Admittedly my need this time was a tad more complex (convoluted?) than usual which led me down this path, but it's functioning nicely now. with multiple methods/devices of input.