Resolved VR UI Slider Issues

Hi @BernieRoehl, can you share the case # so I can make sure it's in our queue? Thanks!

 

So, Vive support is actually directly handled by HTC. Please share this on their developer forums and their teams will help.

 

If you are using the `XRUIInputModule` you may need to crank up the `Tracked Device Drag Threshold Multiplier`
By default, the UI system considers something a drag and not a tap when it moves 10 pixels in UI space for 2D input (mouse & touch), and 20 pixels for tracked controllers. Since UI can be far away, and controller input can be jittery, it feels like a much tighter threshold for tracked devices than for 2D. Once it's a drag and not a tap, I suspect the scroll view is taking ownership of the event and consuming it.

The Event System has a Drag Threshold value that sets the initial 2D value, and then for tracked devices it's multiplied by the Tracked Device Drag Threshold Multiplier. Cranking that number up is a compromise though, you'll get more reliable clicks, but you may not get as responsive drags, so you may need to find a personal sweet spot that fits your game design or rig setup.

Could also be something else, but I feel like this is at least a thread to pull on.

 

@Leoss
You are right, I was only responding to the person above, and not the main reported issue in this thread.

As for sliders, I started looking at it today, and I know why it behaves that way, but I don't know if I have a fast fix for you at the moment.

TL;DR: It's always been a bug, but there are some situations where it works correctly (UI backing such that the pointer doesn't fade off to infinity between UI elements), so you may have seen it working well in other demos. Please file a bug and ideally attach this thread/info in case it's not me that ends up looking at it.

The reason sliders don't behave as well in VR has to do with where pointers start from vs. screen-based input. Mouse and Touch input always originates at the screen, and so when you raycast into the world, the position on the screen is the same 1m, 5m, 100m away from the origin. The slider knows where in screen-space, the cursor is so it can figure out how far along the slider is. E.g. in this first image, it's really easy to figure out where the scroll cursor should be even though it's not on the actual slider:


For VR, it's different. The origin of the pointer is totally separated from the view. This means that the screen-space location of the cursor is dependent on how far along the ray we are checking. For example in this image, where the red line is the pointer ray:

The screen space position of the pointer could be anywhere along that red line, depending on how deep in the world you want to check. That's just not the case with mouse and touch.

Normally, in VR, I solved needing screen space in UI events by finding the hit point of the XR pointer and getting it's screen space. This is backwards from mouse and touch where we have the screen space position and use that to find the world space hit point. This means the slider will work well so long as there is some other UI object being hit on the same plane as the slider itself, keeping a consistent depth and therefore screen space position. But when you hit gaps that pass through to infinity or other long distance UI, it starts to misbehave.

I don't have an immediate fix for this, sadly. The XR UI system was setup to try to integrate as seamlessly with the existing UI as possible so we didn't need XR Slider, XR Dropdown, etc..., but in this situation, I may need to modify the slider to find a good way to adjust how screen space is determined for pointers, though I think this second option would break UI elsewhere.

 

100% with you. Theory is nice, but even if we have bugs, you've still got to ship.

So, I attached an XR Slider. Mostly copy-paste, hacking out some Internal calls, and modifying a single function.

That function is UpdateDrag:

Code (CSharp):

1.         void UpdateDrag(PointerEventData eventData, Camera cam)
2.         {
3.             RectTransform clickRect = m_HandleContainerRect ?? m_FillContainerRect;
4.             if (clickRect != null && clickRect.rect.size[(int)axis] > 0)
5.             {
6.                 Vector2 position = Vector2.zero;
7.                 if (eventData is TrackedDeviceEventData trackedDeviceEventData)
8.                 {
9.                     clickRect.GetWorldCorners(s_Corners);
10.                     var plane = new Plane(s_Corners[0], s_Corners[1], s_Corners[2]);
11.                    
12.                     var rayPoints = trackedDeviceEventData.rayPoints;
13.                     for (var i = 1; i < rayPoints.Count; i++)
14.                     {
15.                         var from = rayPoints[i - 1];
16.                         var to = rayPoints[i];
17.                        
18.                         var rayDistance = Vector3.Distance(to, from);
19.                         var ray = new Ray(from, (to - from));
20.                        
21.                         if(plane.Raycast(ray, out var distance))
22.                         {
23.                             if (distance < rayDistance)
24.                             {
25.                                 var worldPoint = ray.origin + (ray.direction * distance);
26.                                 position = cam.WorldToScreenPoint(worldPoint);
27.                                
28.                                 Debug.DrawLine(from, worldPoint);
29.                                 break;
30.                             }
31.                         }
32.                     }
33.                 }
34.                 else
35.                 {
36.                     position = eventData.position;
37.                 }
38.                
39.                 Vector2 localCursor;
40.                 if (!RectTransformUtility.ScreenPointToLocalPointInRectangle(clickRect, position, cam, out localCursor))
41.                     return;
42.                 localCursor -= clickRect.rect.position;
43.  
44.                 float val = Mathf.Clamp01((localCursor - m_Offset)[(int)axis] / clickRect.rect.size[(int)axis]);
45.                 normalizedValue = (reverseValue ? 1f - val : val);
46.             }
47.         }

I cut out multi-monitor support (used Internal calls), and everything inside of `if (eventData is TrackedDeviceEventData trackedDeviceEventData)` is custom. What I do is create create a plane struct that represents the slider button, and then figure out where the pointer ray intersects that virtual plane. I use that to determine my screen position, and then fall through all the same logic as the normal slider. The scrolling area should be very similar, do a search for eventData.position, and replace that value with the raycast against plane code above.

However, this only fixes half the issue. The positioning is now correct, but it doesn't have the same behaviour as a mouse where it still calculates the position of the scroller even if I'm not pointing at anything. It just sort of stops whenever I point into open space.
I traced this down to the UIInputModule. There is some code in the `internal void ProcessTrackedDevice(ref TrackedDeviceModel deviceState, bool force = false)` that is broken on drag. If you look at this blob:

Code (CSharp):

1.                 Vector2 screenPosition;
2.                 if (eventData.pointerCurrentRaycast.isValid)
3.                 {
4.                     screenPosition = camera.WorldToScreenPoint(eventData.pointerCurrentRaycast.worldPosition);
5.                 }
6.                 else
7.                 {
8.                     var endPosition = eventData.rayPoints.Count > 0 ? eventData.rayPoints[eventData.rayPoints.Count - 1] : Vector3.zero;
9.                     screenPosition = camera.WorldToScreenPoint(endPosition);
10.                     eventData.position = screenPosition; // <------- This line right here needs to be deleted
11.                 }
12.  
13.                 var thisFrameDelta = screenPosition - eventData.position;
14.                 eventData.position = screenPosition;
15.                 eventData.delta = thisFrameDelta;

I put a comment on the line that needs to be removed. It's shorting out the calculation of eventData.delta directly below it. This brings back the ability to update drag while pointing at nothing. This one is a little awkward for you to patch in as you'll also need to bring in a custom XRUIInputModule and I believe a lot of XRI is internal. You may be able to clone the package and put it locally in your packages folder if you just want to make custom modifications directly.

Please report both these as bugs. I know these systems, and want to help, but I'm working on other things, and doubtful I'll be the one tackling these issues directly.

Hope this helps!

 

Hey @RogueStargun,

This is still an issue when the pointer is not hitting a UI element (firing off to infinity). There are a couple of workarounds that will work until a proper fix is released.

Option 1: Create a Hit Background (Image) child object on the Slider and scale that to accommodate some buffer. You could stylize this image or make it transparent if you want the 'floating slider' effect.

Option 2: The default Background child object has Raycast Padding in the Image component that can be adjusted to achieve the same effect as option 1. I had to crank these to about 200 left/right and 50 top/bottom to get it feeling good, but it will depend the size of your slider and the scale of your UI elements.

Both of these have the added benefit of reacting to clicks near the slider bar when parented to an existing panel or not (floating), which really helps lower user frustration.

Let us know if these are workable solutions for you.

 

Hey all,
We did find a minor issue with dragging Sliders and ScrollRects in our XRUIInputModule that will be fixed in the upcoming version (2.0.2). We saw a similar complaint and partial fix in another thread which led us to investigate. Historically, there was a bug fix in the InputSystem version that didn't get applied to the XRI one. Thanks again @AzurySimon for calling this out.