Official XR Interaction Toolkit Preview Release (0.9)

Does this affect the common input usages XRController bindings already written in the examples? I'm working on migrating a project over to using this but if the input coding changes again it could be an extra load of work!

 

Yes, you're right, I totally missed it - XR Direct Interactor! thank you

@andybak - you're also right, thanks

 

BUG? @mfuad @Matt_D_work @StayTalm_Unity
When you grab objects they're pulled from their position in the Hierarchy and set as root objects. I confirmed that this happens in the demo scenes. Is this supposed to be the default behavior?

It creates issues, am I missing a way to set this to not be the default behavior? I have a menu with grabbable objects. When the objects are moved and the menu is closed the objects are now no longer in the menu, and as such remain in the scene. I can rebuild the project to assume everything interactable has to be a root object but I would rather not if I don't have to.

 

How do the XR Controller classes behave with 3dof Headset controllers (rotation only) - I was originally using the Tracked Pose driver with Arm Model script provided by the XR Legacy Input Package, however that is now quite difficult to implement with the XR Ray Interaction script, and also without the XR Controller script, I can't seem to easily set this up?

 

Did you report the issue where both the curve beam and pointer beam show simultaneously? I am having this issue too on Oculus Quest and Oculus Go!

I have identified this is a case caused by a Nullreference in the ControllerStates.ClearAll function ran by OnEnable in the ControllerManager Script. I can recreate it by loading to another scene or reloading the scene at runtime

Reported at Case 1208482

You can fix it in the meantime by commenting out the line for left and right controllers in the RegisterDevices function that is m_LeftControllerState.ClearAll(); :

Code (CSharp):

1.  void RegisterDevices(InputDevice connectedDevice)
2.     {
3.         //  Debug.Log("Registering device: " + connectedDevice);
4.         if (connectedDevice.isValid)
5.         {
6. #if UNITY_2019_3_OR_NEWER
7.             if ((connectedDevice.characteristics & InputDeviceCharacteristics.Left) != 0)
8.  
9. #else
10.             if (connectedDevice.role == InputDeviceRole.LeftHanded)
11. #endif
12.  
13.             {
14.                 m_LeftController = connectedDevice;
15.                 // m_LeftControllerState.ClearAll();
16.                 m_LeftControllerState.SetState(ControllerStates.Select);
17.             }

 

Is there a way to set the XRRig tracking space type via script? Or does it follow with the return of XRDevice.GetTrackingSpaceType automatically?

We currently have the issue where XRDevice.GetTrackingSpaceType returns Stationary always for Oculus Link and build running on Oculus Quest regardless of what Guardian tracking mode we have set (Roomscale or Stationary) - reported at Case 1208329

Also, we would be able to detect the device by name and change what we want via script but XRDevice.Model returns nothing when using XR Management Subsystems Package. Reported at Case 1208324

We are working with the Oculus Go and Quest, hence wanting to switch out the rig, the XRRig by default in the examples is not suitable for the Go

XR Interaction Toolkit - On Oculus Go, ControllerManager errors due to one controller active - Case 1208341

If you are able to look into any of this it would be really helpful, all 3 of those issue cases are stopping us from having a means of changing the rig at runtime on the Oculus Quest or Go to support roomscale or stationary or even just changing what controllers we have active for each platform in the scene.

 

WOW! Nice work ... thanks.

I didn't report it because I didn't actually know how (outside of making a forum post).
Where did you go to open a case on the issue?

 

While your project is open in the Unity Editor - Help> Report a Bug

More info about bug reporting here: https://unity3d.com/unity/qa/bug-reporting

 

So for my own understanding - should be looking at this as a replacement for VRTK? Or perhaps a replacement for both VRTK *AND* the oculus assets pack. I'm partway through the Oculus VR course https://learn.unity.com/course/oculus-vr which focuses pretty heavily on VRTK as the bridge.

Questions to anyone who has experience working in both this and VRTK/Oculus. How do you find this compares to VRTK? Would I bother using this without the Oculus pack? Is it possible to create a VR experience that doesn't pull in anything external outside of this? Is there even any value in doing that?

Especially considering hand-tracking on the quest ... which is super cool I'm going to make my way through finishing the course then try to take a fresh look at the elements from the perspective of quality, ease of integration and, dependency elimination.

... and welcome back from vacation @StayTalm_Unity and the rest of the team!

 

Thankyou so much for your detailed and thoughtful responses, I was worried I had overshot on feedback here but it was difficult to sum it all up in the feedback survey when I was finding the issues over the course of a few development sessions.

Alignment with the legacy input systems Arm model and improved cross-platform support for 3dof and 6dof headsets with 1 or 2 controllers will be fantastic for ease of cross-platform use.
This functionality is all almost there for it though and you've all done a great job at unifying the necessary XR functionality here

- If the Controller Manager script could potentially have an example of registering/unregistering 1/2 controllers (for example a 3dof with 1 controller) and separately a 6dof with 2 to switch out the controller states (for example the 3dof could have the grabbable interactor that shoots a ray at grabbables and a teleporter on touchpad press but the 6dof 2 controller setup would work better with a direct grabbable on one hand and the teleporter on another on touchpad press, that would be really neat). However, I could probably switch this out myself with the Input Device API otherwise if that's more advisable.

Looking forward to the fixes and thanks again, your work is a huge help for my projects!

 

I would use this as a replacement for VRTK, in my opinion VRTK was a lazy quick-start and any customisation of underlying functionality seemed to be complex due to the way it was written. I've been using just the Unity tracked pose drivers for my project from the XR Legacy Input helpers package and other things like teleportation/locomotion are more custom to your project. This pack does add UI interaction which in the past has involved hacks/workarounds. Support for VRTK has been on/off. For future-proofing I would use the Unity stuff like XR Interaction package as it has that Unity team support.

My thread here is a good starting point, I should update it to point people to this package really, lots of good advice there.

https://forum.unity.com/threads/vrt...-project-starting-development-in-2019.620998/

Feel free to direct message me if you have any questions about migrating to this package - I don't want to distract from feedback and discussion here.

 

Thanks for this feedback.

I'd have to agree that some parts of VRTK seem heavier than they might need to be. I was doing the "locomotion" unit last night in the course and found that, while I liked & understood the need for interface abstraction there were some parts that seemed to be a little heavier than they needed to be.

Specifically it struck me that there was a lot of hooking things up in the editor manually that I thought could lead to situations that were hard to debug.

I'll definitely reach out if I can't figure out the XRTK after the course, I'm hoping the stuff is intuitive enough to pick up and run with.

It's probably valuable feedback for the Unity team to talk about other frameworks and toolkits if it leads to constructive criticism of the XRTK. Like what do the other frameworks do well that Unity's native setup doesn't do?

Now that they're back from vaction - I'd definitely like to reiterate my desire to have an XR toolkit scene switcher built in to the example scene so I could try out a number of the different interactions in a single build. There's just so much work in getting something built to the quest it'd be nice to have that streamlined.

 

That would be super easy to make with a simple script using a function to call LoadScene and a worldspace UI button to call it, or test them out in editor by changing scenes and hitting play What headset are you using?

 

Also cross-platform Fixed-Foveated Rendering, which is available through (the as of yet unreleased for Android VR) Vulkan. Is there any news on that you are able to share or even just a hint that the XR teams are aware? FFR is almost essential for android VR and currently is only actually achievable on Oculus Quest/Go with the Oculus Integration package through the OVRPlugin.dll - this is a bit of a pain for us in terms of cross platform support.

There was a talk at Oculus Connect about the Vulkan FFR and how it would simplify cross-platform VR development and help with optimization for different platforms

 

One thing I would like is the option to grab from 2 points rather than just the singular attach point, would help a lot for FPS games.

Also the option for it not to just snap to a position in your hands, picking up things can be weird if it just teleports to a position.

 

Any word on how to use this with OpenVR headsets such as the Vive/Index? Currently XR Subsystems Management doesn't have a Loader for OpenVR

https://forum.unity.com/threads/xr-plugins-and-subsystems.693463/page-3#post-5336661

 

My thoughts after playing around with the toolkit a couple of times:

Bug - For the record, the "all pointers active all the time" bug that Robyer1 reported also happens on PC builds.

Manipulating "Kinematic" or "Velocity driven" objects is pretty unpleasant due to severe juddering. On my Rift S it is very noticeable. On my Index, the juddering is less noticeable as you increase the framerate. At 144hz it looks as if there is a motion blur effect being applied to the object being held. What's weird is that the juddering still seems to apply after being thrown. In the scene with apples, if you throw one, you'll notice it's still juddering until it hits the floor, then will roll smoothly as the physics kicks in. Not actually sure what the point of these grab types are? They all seem to have the same behaviour when you throw them.

Things I"m surprised by:

Needing to pull trigger to complete a teleport
No smooth locomotion
No swapping objects between hands

Other things I would like to see:

Animated hands example
2 hand grab options
More teleport options
More snap placement options.

It would be great if one of these could be done as an example of how you intend people to extend the toolkit themselves.

Long term, I really hope unity continues support this fully. I'd really like the new input system and ui elements to be made compatible as soon as possible. Recently, someone has made MRTK compatible with oculus quest hand tracking , it would great if this toolkit ended up being that versatile. On the whole though, a good start .

 

Yup exactly what I was thinking, having that be an option would be fantastic because teleportation / snapping is very useful for some objects, but things like boxes for example would be better with the grabbing any part system. I honestly think with those 2 options people would get a LOT more use out of the XR kit

 

RE: The judder that Shizola talked about, I do also get this and it can make the experience pretty unpleasant. I do also agree with adding in teleport options, smooth locomotion / turning and swapping options if possible as those would be very nice as well.

Also a reset world position option would be really good too

 

I'm on a quest - with a laptop that doesn't support quest-to-link so the run-it-in-editor option is off the table, but the worldspace button is what I'm thinking of... or some sort of UI interface that pops up on left-hand menu button press. I'm not too worried about adding it myself to the project for local testing and development, but unless someone on @StayTalm_Unity 's side includes it, we'd have to re-add the menu item thing for every new release.

 

@StayTalm_Unity Regarding the XRRig Tracking Space Type, when using the Oculus Quest with Oculus Link in editor XRDevice.GetTrackingSpaceType returns Stationary always for Oculus Link and build running on Oculus Quest also, I have this reported at Case 1208329, is this expected behaviour?

 

Is the Physics tracker from the Unity SuperScience pack any help for this? I'm sure your teams have looked into this internally but I found this pack super useful for smooth holding of physics objects in another project

https://github.com/Unity-Technologies/SuperScience

Also, the worldspace UI canvas can only have one camera assigned to it for raycasting still, is there a workaround for this you would suggest? (In the case of ever having two XR Rigs or more going on)

 

Great to hear, and also I understand the reason for only one camera being active for UI interaction there. Haven't found a use case for having 2 active otherwise as you suggested do it via networking.

We are struggling with working on OpenVR headsets with the new XR Management Subsystem current as the OpenVR XR Plugin isn't out yet.

Also UniversalRP through SRP does not render UI or Transparent objects on Oculus Go/Quest when we tried out the samples with it unless when you disable MSAA or disable opaqueTexture in URP pipeline setting - it is likely related to this issue: https://github.com/Unity-Technologies/ScriptableRenderPipeline/pull/5060

Hopefully that gets fixed as well as the Fixed-Foveated Rendering not working with UniversalRP either as most of this interaction package and fixes for the above issues should be coming in Unity 2020 releases we assume.

 

I might be missing something, is there a way to hot-swap controller models during runtime?

The XRController script checks a private bool (m_PerformSetup) to decide if it should setup the controller model. Without me going in and making this public, and then tripping it after setting the modelPrefab to my next desired prefab transform, is there a way to hot-swap?

or.. should I just have all my controllers loaded at start and turn their respective game objects off and on as needed? **1/16/20 EDIT I ended up doing this GO swap solution since it works and I didn't get any better suggestions**

 

I am trying to get teleport and ray activated by Thumb touch on Rift but I can't get touch to work. Updated XR Managment, XR legacy input helpers, Oculus XR Plugin to the newest version. Does touch on oculus touch work with XRI?

Edit:
Of course, I realized 3 min after posting this that activation might be hardcoded somewhere so I now got it to work.
@StayTalm_Unity
1)What is "activation" in XRController for? I thought it would invoke activation of teleport or base state respectively from ControllerManager but activating teleport state is hardcoded. Would it be possible to let the user choose which button/touch switches controller into teleport state?

2)Could unity generate HandleInteractionAction method in XRController so that it also accepts boolean values? Thumbstick click (which right now is probably the most common way to teleport on oculus) is type of bool and doesn't work with this method in XRI 0.9.2.

 

Hi, I've just been reading up and playing with it for a day now but it seems really promising.

I picked up one issue when using the Oculus Quest. If I'm in the scene and then leave the guardian and come back inside the guardian it fails to load the scene back in/switch from passthrough it just goes black and the 3 loading dots appear. The scene is was testing was the WorldInteractionDemo.

 

Hi everyone. I've been trying to get the WorldInteractionDemo to work on the Oculus Quest but without much joy. Have a heap big load of console errors (screen grab for taster). Before I start mucking about with it, does anyone have any idea why I'm getting this? Is it a Metadata issue?

Unity 2019.3.0f3

 

Yes will do. Just not sure of it was a bug or user error!

 

Hi @StayTalm_Unity and the whole community,

I use the package for quite a few days now and experienced a bug (I think so) when deleting placed Objects in AR.
It seems that Objects that are placed with an SelectionInteractable on it are registered once placed at the InteractionManager. When destroying the object it seems that the InteractionManager still has a reference to the already deleted object. I tried that out with all InteractableScripts - Only the SelectionInteractable seems to cause this error.
So once a object containing that script is deleted there is no chance to place objects or select objects anymore.

Can somebody copy that behavior or do I miss something elementary?

Greets d4n3x

 

When using the supplied AR demo scene on IOS, I'm noticing that if you have different horizontal planes of different heights, the supplied cube game object will translate to the highest plane. So if you scan a plane on the floor, then scan one on a table, then place the cube on the floor plane and hold your finger down on the cube and move it to translate, the cube will fly upwards to the table plane height and towards your finger.

Being new to AR foundation, I'm not sure if this is a bug, or the expected behavior.

 

This would be invaluable, often I have an issue with needing to call something like move the rig to 0,0,0 at start or awake, then recenter, but player movement code on our character controller also tries to move the character at the same time or between them (Which I have stopped by delaying it). But the ideal state would be for the player rig to be set to 0,0,0 as an operation on the XRRig before anything else runs to give people peace of mind that their rig will start at the 0,0,0 of its starting position in the scene.

Something like 'Recenter Rig on Awake' - "XRRig will be positioned at its position in the scene on start regardless of the user's position"

 

Hi Guys,

It's not clear to me how to disable AR interactions when you need to. I've tried disabling the XRInteractionManager and ARPlacementInteractable or looking into the scripts for an appropriate state but no luck.

EDIT: I just ended up disabling the ARGestureInteractor.

 

Anything on ETA for supporting OpenVR? It's something that some people have asked but I haven't seen an answer.
Thanks!

 

In response to Case 1183256 where I reported Oculus Quest defaults to stationary not room-scale tracking mode regardless of the guardian being set to 'Stationary' or 'RoomScale', QA have told me that behaviour is intended.

This messes with the point of having a stationary rig for 3dof where the headset is positioned at a set height (due to no positional tracking) and 6dof headsets which should use Room-scale as they have full positional control.

What do you guys think of this? I'm baffled as the XRRig has a Stationary and Roomscale setting which seems to be more for 6dof/3dof but the Quest always reporting 'stationary' messes with this for us @StayTalm_Unity

 

Any chance of a link? I can't see a way to look up issues via ID.

 

https://fogbugz.unity3d.com/default.asp?1183256_0tvhssoas6dmtruk

 

not sure why people are saying OpenVR isn't supported. I downloaded the sample project from the Unity github and ran it on a Vive. Just take a look at the packages manifest on line 15: https://github.com/Unity-Technologi...1ccf0eca3a6e0682bc0/VR/Packages/manifest.json