Official XR Interaction Toolkit 0.10 preview is available

We have just published a new preview of the XR Interaction Toolkit (XRI) that brings a few new features and a lot of bug fixes. For those who want to experiment with XRI, the best way to start is with our samples available at https://github.com/Unity-Technologies/XR-Interaction-Toolkit-Examples. As always, you can refer to our documentation for more information.

What’s new:

• Smooth locomotion - move around the virtual environment at a constant rate, using the trackpad or thumbstick of a VR controller
• Support for Unity Input System - use the input system to drive VR interactions
• Keyboard/mouse simulation - simulate input without having to deploy to device
• Improved layout of properties in the Inspector window
• New samples, including use of the Universal Render Pipeline

Bug fixes:

• Fixed some behaviors not supporting multi-object editing in the Inspector
• Fixed PrimaryAxis2D input from mouse not moving the scrollbars on UI as expected. (1278162)
• Fixed issue where Bezier Curve did not take into account controller tilt. (1245614)
• Fixed issue where a socket's hover mesh was offset. (1285693)
• Fixed issue where disabling parent before XRGrabInteractable child was causing an error in OnSelectCanceling().
• Fixed Tracked Device Graphic Raycaster not respecting the Raycast Target property of UGUI Graphic when unchecked. (1221300)
• Fixed XR Ray Interactor flooding the console with assertion errors when sphere cast is used. (1259554) (1266781)
• Fixed foldouts in the Inspector to expand or collapse when clicking the label, not just the icon. (1259683)
• Fixed created objects having a duplicate name of a sibling (1259702)
• Fixed created objects not being selected automatically (1259682)
• Fixed XRUI Input Module component being duplicated in EventSystem GameObject after creating it from UI Canvas menu option (1218216)
• Fixed missing AudioListener on created XR Rig Camera (1241970)
• Fixed several issues related to creating objects from the GameObject menu, such as broken undo/redo and proper use of context object.
• Fixed issue where GameObjects parented under an XRGrabInteractable did not retain their local position and rotation when drawn as a Socket Interactor Hover Mesh (1256693)
• Fixed issue where Interaction callbacks (OnSelectEnter, OnSelectExit, OnHoverEnter, and OnHoverExit) are triggered before interactor and interactable objects are updated (1231662, 1228907, 1231482)
• Fixed compilation issue when AR Foundation package is also installed
• Fixed the Interactor Line Visual lagging behind the controller (1264748)
• Fixed Socket Interactor not creating default hover materials, and backwards usage of the materials (1225734)
• Fixed Tint Interactable Visual to allow it to work with objects that have multiple materials

Known issues:

• Teleportation is not functional when the Continuous Move Provider sets Gravity Application Mode to Immediately
• Teleportation position is overridden by continuous movement when both occur on the same frame
• Custom reticles get displayed on objects without a custom reticle (1252565)
• Socket Interactor can apply the wrong rotation to an interactable and cause the interactable to skew in scale when the interactable has a parent with a non-uniform scale (1228990)
• Socket Interactor does not take the enabled state of the Renderer into account when drawing the hover mesh
• Adding an interactable to a parent GameObject of an interactable that is being destroyed will cause the colliders to not be properly associated with the new interactable (1231482)
• Controller connection interruptions disable interactors when using the Controller Manager script from the examples project (1241245)
• Anchor manipulation in Ray Interactor is inconsistently applied based on the controller class, and deadzone should be configured in the action for Action-based controllers
• Layer of Grab Interactable does not inherit layer of Interactor when selected, which can cause Continuous Move locomotion of the rig to be pushed away in the wrong direction when the object overlaps with the Character Controller
• In the example VR project, the Interactor Line Visual only appears in the left eye when using Windows Mixed Reality

Roadmap
We now have a public roadmap available for users to see our latest plans, upvote existing feature requests, and/or submit new feature requests. We are currently working towards a public 1.0 release next year (Unity 2021.2). Most of our focus and development efforts now are on bug fixes, UX improvements, and polished documentation & samples. The feature set for public release will primarily reflect what exists today.

Sharing feedback
This forum is the best place to open discussions and ask questions. As mentioned above, please use the public roadmap to submit feature requests. If you encounter a bug, please use the Unity Bug Reporter in the Unity Editor, accessible via Help > Report a Bug. Include “XR Interaction Toolkit” in the title to help our team triage things appropriately!

 

If you're using the Action-based version of the behaviors, and you're referencing an Input Action contained in an asset on the XR Controller to use for position and rotation tracking, you'll need to ensure the Action is enabled. The XR Controller will only automatically enable Input Actions that are directly defined on the component, and will require you to manage enabling or disabling the externally defined Input Actions. You can add a GameObject to your scene and add the Input Action Manager behavior to it, and then add the Input Action Asset that the Input Actions are defined in to the Action Assets list in the Inspector. That behavior will then enable all the Input Actions in that asset during its own OnEnable.

You'll also want to make sure Active Input Handling in Edit > Project Settings > Player is set to either Input System Package (New) or Both. While in Play mode, you'll need to make sure the Game view has focus for input to be read properly. You can set an option to lock input to the Game view so you don't have to remember to do that, see https://docs.unity3d.com/Packages/c...tml#using-actions-with-action-based-behaviors.

 

Hi @freso, yes our focus is on stability and usability. This includes a high priority on bug fixes, polishing our samples and documentation, and UX improvements to ensure the onboarding experience of using this toolkit is optimal.

 

There's currently nothing built-in to do this out-of-the-box with XRI. One approach to do something like this would be to create a behavior that would be responsible for converting gestures with the controllers into translation and rotation amounts to either move the XR Rig or the object you are manipulating. You could serialize multiple Input System Actions that would bind to a controller's position, rotation, and the grip button. The behavior would listen for the Actions that represents the grip button, and when both are pressed, sample and store the position and rotation of each controller, and likely take the average. Then each frame, the behavior would convert the difference between the initial sampled pose with the current pose, and convert that into a desired change of position and rotation. All of that would be achievable with using the Input System package.

Depending on how your scene is structured, you could then use the position and rotation deltas you computed with that behavior into either updating the Transform of the object you want to manipulate, or the XR Rig with it pivoting around the object you are manipulating. If you're updating the XR Rig, you can write a new behavior that derives from

LocomotionProvider

to push those changes to the XR Rig through the

LocomotionSystem

.

You may be able to search online for examples to hopefully help out with this locomotion method. Some refer to this style as "move the environment".

 

I created a bug issue (1291475) to get the documentation button fixed.

Some of the renaming done this version, like changing some properties from PascalCase to camelCase, was to follow Unity style conventions for a package of this type. Most of the properties in the package were already following this convention, but a few didn't match and were updated for consistency.

The other methods you mentioned were actually split into multiple methods. For example,

OnSelectEnter

was split into

OnSelectEntering

and

OnSelectEntered

. This was done to allow the different phases of an interaction state change, like "select", to be overridden and to change when the public event is invoked for the Interactor and Interactable. Before this change, the public event on the Interactor would be invoked before the Interactable had a chance to process the change, which could lead to undesired behavior. In this new version of the package, both the Interactor and Interactable will finish processing the change before the events are invoked. These public events were also renamed to match the function, for example

onSelectEnter

to

onSelectEntered

. This timing of these methods and events is visualized in the documentation, see Extending the XR Interaction Toolkit.

 

This is a bug, I created an issue (1291523) for you to track.

We'll likely copy the world position and rotation of attachTransform instead of local position to support more hierarchies.

 

No specific ETA as we work through our backlog; we'll keep you updated via the Issue Tracker link @chris-massie shared.

 

It should work fine. If you think there is a bug, follow the steps under "Sharing feedback" in the original post.

I'm not sure what's going wrong since the the more generic XR HMD should work for all supported devices. Make sure Tracking Type is set to Rotation And Position and make sure the Game view has focus by clicking in it with your mouse while playing. To help with debugging, open Window > Analysis > Input Debugger, expand the Actions foldout, and it should have at least two actions, "Main Camera - TPD - Position" and "Main Camera - TPD - Rotation". If you can expand those foldouts, it will show the resolved controls.

Since you mentioned using Steam, when I test with my Oculus Quest when I use the deprecated OpenVR Desktop package instead of the XR Plug-in Management packages, the two Actions resolve to

/OculusQuest/centereyeposition

and

/OculusQuest/centereyerotation

when the Tracked Pose Driver (New Input System) is using

<XRHMD>/centerEyePosition

and

<XRHMD>/centerEyeRotation

for the Actions. (Note that the T button will toggle between the UI selector and the raw string.)

 

Using [Any] (see wildcards in https://docs.unity3d.com/Packages/c...manual/ActionBindings.html#binding-resolution) will mean that it will be possible to bind to any device layout for centerEyeRotation rather than only those that derive from the XRHMD class. I don't foresee any problems with using that instead, especially if you're not registering any custom device layouts with a centerEyeRotation that you wouldn't want to use.

 

Valve's OpenVR Unity plug-in requires their SteamVR plug-in for input. It does not support Unity input, and thus doesn't work with the XR Interaction Toolkit. Please refer to their documentation for more information regarding their requirements. That will change as Valve transitions to OpenXR, and Unity adds initial support for OpenXR. With OpenXR, all input from conformant OpenXR runtimes (including Valve’s runtime) will be routed through Unity and can be accessed through Unity's Input System and the XR Interaction Toolkit.

 

By default, the player is able to look through walls. You can achieve a simplistic solution that may be sufficient for your game/experience with the current version of the package, though it does involve setting up the XR Rig in a certain way. You can use a Character Controller component on the XR Rig to add a single capsule-shaped collider to represent the body and head of the player. Since it's just one single capsule collider, that does mean that you won't be able to, for example, lean your head out and over the railing of a balcony to look down since your body will be constrained.

If you add the Character Controller Driver behavior to the rig and set it to listen to the Continuous Move Provider, that behavior will drive the center position and height of the Character Controller based on the camera position whenever that Continuous Move Provider does a locomotion event. If you set Gravity Application Mode to Immediately on the Continuous Move Provider, that will make the events fire every frame instead of just upon thumbstick input. With this configuration, as you move your head/HMD, the Character Controller Driver will try to move the Character Controller to that position. If the wall has a Collider and you try to move your head through it, the rig will be pushed away due to the collision.

You can use layers and set the Interaction Layer Mask on the Grab Interactable and the Socket Interactor to define which objects are compatible. See https://docs.unity3d.com/Packages/c...t@0.10/manual/index.html#interactionlayermask for more.

 

I'm sorry you had frustrations while using it. We're working on better warnings and potentially removing the need to give the Game view focus, but it isn't finalized yet.

That is currently not possible in the package at the moment. Follow the link in the OP to submit feature requests.

The functions you listed all operate on an Interactable, and those typically require that they have a Collider. UI interaction with a World Space Canvas with the Ray Interactor is achieved with the XRUI Input Module on the Event System, which then executes events by calling methods implemented by behaviors on those objects, similar to how Mouse clicks are handled in a non-VR game/experience.

If you want to add the ability to hover to select for UI, there are a few ways you could go about implementing it. You could add a custom behavior to the buttons in the UI that implements

IPointerEnterHandler

and

IPointerExitHandler

, and trigger a press after hovering for long enough. Another way would be to override

XRUIInputModule

or write a custom Input Module to do the hover to select logic. To send haptic feedback, you would need to call

XRBaseController.SendHapticImpulse

.

 

You should be able to call

XRInteractionManager.SelectEnter

to initiate the select through script after you spawn the Interactable object.

The functions are protected internal and virtual so you're able to override and call them in a derived class. Some also have an associated event accessible through script or in the Interactable Events foldout in the Inspector window. You should be able to override some methods in

XRGrabInteractable

to do what you're describing. We'll also look into making more functions in that class like

Drop

virtual.

That is something we are aware of and is on our roadmap.

We'll look into making this public.

 

You can create a child GameObject of the Ray Interactor and set its position/rotation that you want, and then set the Ray Interactor's Attach Transform to that object. However, there's currently a bug (1291523) mentioned in post #9 that resets the pose of the

originalAttachTransform

that is used for the cast. A fix is currently in progress and will be included in the next patch. As a workaround until then, you can subclass

XRRayInteractor

and override

Awake

to set the position/rotation of

originalAttachTransform

to the pose you want.

Very cool gif, nice job!

There's no difference, it just calls the other function.

 

I don't know of any examples that I can point you to. If you would like to submit a feature request, use the link in the OP to our public roadmap.

If you would like to implement a solution yourself, adding an in-game keyboard will be fairly complex, but it should be doable with the packages as they exist now. I'm going on the assumption that you are wanting to create a world-space canvas with a keyboard layout, use Ray Interactors to point at key buttons, and use it to type into Input Fields in Unity UI.

If your use case is limited in scope, you could create a world-space canvas with the keyboard buttons you want to support, use the pointer down and up events of the buttons to notify a custom behavior that the key has been pressed/released, and use it to construct the input characters that you can then make use of. You could then copy it to whatever you need through available scripting APIs. You can look at the examples repo linked in the OP for the scene setup you'll need to support using Ray Interactors to click world-space canvas buttons. You may want to also make considerations for different keyboard layouts for localization.

If you want to support Input Fields more generically, and have a floating in-game virtual keyboard to type into it, you'll likely want to copy the

XRUIInputModule

class included in the XR Interaction Toolkit and customize it for adding keyboard simulation. The Input Module is responsible for executing events and also to set which UI object is the currently selected GameObject in the EventSystem. Since clicking the in-game keyboard buttons would steal focus from the Input Field, you'll need to modify that input module to control whether to skip the call to

SetSelectedGameObject

so the Input Field remains the selected object even when you pull the trigger on your virtual keyboard. Then you'll need to add to the Process loop of the input module to push a queue of Event objects to the selected Input Field. Your custom behavior that listens to the press/release events of the keyboard buttons would use Event.KeyboardEvent to construct these Event objects. The input module would check if the

EventSystem.currentSelectedGameObject

has an InputField/TMP_InputField component, and then call the

ProcessEvent

method on it so it can handle the key press. Finally, you'll need to add in logic somewhere to display the virtual in-game keyboard when the Input Field receives focus.

There's a lot of details I'm probably missing, but hopefully this sends you down the right path if you want to try writing your own.

 

The Ray Interactor doesn't interact with UI elements with the Collider-based select and hover interaction states, it does so through the XRUIInputModule on the EventSystem GameObject. The

isSelectActive

property won't play a role with UI interaction. Instead the

XRBaseControllerInteractor.isUISelectActive

property is used which reads the value from

XRBaseController.uiPressInteractionState

which is typically driven by the controller trigger.

If you want to manually cause a UI press through script instead of from the actual controller, you'll need to override the

XRBaseController.UpdateInput

method to set the

XRBaseController.uiPressInteractionState

properties based on when you want to activate and deactivate that state for press and release.

 

Please report this bug following the steps in the Sharing feedback section in the OP and we'll get that fixed.

 

We have an upcoming release that has an additional fix so that mouse input will work with UI as expected when using the XRUIInputModule when Active Input Handling of the project is set to Input System Package (rather than Input Manager (Old) or Both).

I'm not sure that answers your question though because I don't fully understand what you mean. Can you explain what you mean by using the right thumbstick and what is pointing at the UI scroller?

 

Ah, I understand now, thanks. That was a feature of the Oculus Integration asset package, however we currently don't have the ability to drive

IScrollHandler

objects, such as Scroll Rect, implemented in the input module included with the XR Interaction Toolkit. Use the public roadmap linked in the OP to submit a feature request for this.

There are a couple approaches you could take if you want to implement a solution yourself. You could extend

XRUIInputModule

and override the

DoProcess

method to add the step of setting

PointerEventData.scrollDelta

and execute the event in the hierarchy, similarly to how the mouse wheel is handled when processing mouse input. Unfortunately however, at this time most of the code in the input module you would need access to is private or internal, so you would likely need to make copy of that class and its base class. Another approach you could take would be to add a behavior to the UI element itself that would listen for the pointer enter event, and read from an Input System Action bound to the thumbstick on the controller to get the input amount, and use that to scroll the element in the UI through script.