Official Unity support for OpenXR in preview

At the moment I've run into yet another issue (Unity 2020.3, XR 4.0.5, OpenXR 1.0.3) using my Vive Pro Eye and using the eye tracking functionality.
When I call the eye tracking calibration (runs in SteamVR) the calibration runs fine, but when done it returns to the default "Waiting for application" screen of SteamVR. This looks like a new issue as I've never run into this before. Other projects (pre 2020 and OpenXR and XR plugins) all ran fine. The "Waiting for application-name" only occurred when initially starting the application. Switching to the calibration utility never posed any problems on returning to the Unity app.
It also seems to happen more often using a build version of the project. Some instances work fine in the Editor which do not work (ie showing the "Waiting for application" pop-up) in the build version.

What I do is:
- At the start of my scene I check if the XRDevice is running, if not I Initalize and start the Subsystem.
- So the scene starts fine in the Vive
- Then I wait for a key press (new input system) and call the SRanipal.Eye.LaunchCalibration method
- The calibration program runs fine
- After calibration (the calibration program quits automatically) I end up in the SteamVR limbo environment (in my case a blueish sky) with the "Waiting for <apllication name>" pop-up in view. The Unity scene is actually active on the background (I can hear the audio).

That is in the Editor. If I build this (64 bit ofc) and run it directly the whole application hangs at the start with the pop-up "Waiting for <application name>". So the scene doesn't even start properly.

Any ideas?

Marco

 

I'm using the latest OpenXR plugin (1.20), Action Based (which is working fine with Oculus XR plugin), and Oculus Runtime. HMD is tracking, but controllers will not track or move.(Rift S)

Could not recreate input device 'Oculus Oculus Touch Controller OpenXR (XRInputV1)' with layout 'OculusTouchController' and variants 'Default' after domain reload

Cannot find parent 'devicepose' of control 'devicepose/istracked' in layout 'XRInputV1::Oculus::OculusTouchControllerOpenXR'

In the previous version of OpenXR the controllers worked, the position and rotation was just off but explained by Unity's explanation in the post above. I checked using the SteamVR runtime and same thing happens.

 

So with OpenXR on Quest, how do we set the framerate?

edit: 1.2 has really wrong offsets for the Oculus Touch Controllers. They were spot on with 1.1 so will have to rollback until you have fixed them.

 

Are OpenXR and Quest / 2 hand tracking working today? We're planning to move to OpenXR as soon as possible if theses features are available in the next few weeks

 

I'm having a slight controller position discrepancy with Oculus Quest controllers on Oculus Link when running on Oculus vs Steam as the default OpenXR provider. They seem identical in position/rotation between the two, except they are slightly off on the z axis position by a finger joint or two.

How can I resolve this, or do I need to wait on some OpenXR update to fix it?

 

Also, I don't have any controller input in a standalone Windows 64 build. In editor it works fine. This is for an Oculus Quest 1 over Oculus Link. Anyone have any ideas on why that would be?

 

It can't be just me but I couldn't find anyone else reporting the audio output issue apart from another post in this forum back in Feb with 0 response.

The audio output is channelled to the computer rather than the headset in both Oculus & SteamVR runtimes, which suggest that the problem might be in the OpenXR plugin itself. The only workaround is to set the audio output to the headset in the windows mixer, which isn't really nice to ask your players to do every time...

Are you guys experiencing the same? Or am I missing something?

Also another less annoying issue is that the VR doesn't seem to start at all when leaving it at default. I had to use the OXRS workaround to set runtime json path to reliably get VR to start properly.

 

I found that there is a different default position (not 0,0,0) on Oculus Touch Controller when using OpenXR with SteamVR and SteamVR beta. Is that intentionally?
Is there any way to get the used SteamVR Version so I can apply different offsets via code when I use OpenXR?

 

I created a new project with the URP sample scene, added OpenXR plugin and a loop sound in the scene, nothing else. And experienced exactly the same issue. Audio is not heard unless changing the audio output to the headset from the windows audio mixer. And the warning sound that reports my controller battery low is heard without problem, only the game audio is not there. Both Oculus & SteamVR runtime have the same issue.

To look at it positively, it's good to know that it's not a problem with my project...

But I still can't believe such an obvious issue not getting picked up within a few months... Or is it some thing wrong with my system then? The above is tested with Oculus Quest 2 on Air Link. Unity 2021.1.5f1 + OpenXR plugin 1.1.1

Please can someone from Unity repeat what I did and confirm if this is also happening to you?

 

Since when is "- In Window -> Analysis -> Input Debugger, is "Lock input to game view" enabled?" required?
I added VR support to our project 1 week ago and everything worked.
Then today i tried it again and this error appeared:
"Lock Input to Game View in order for tracked pose driver to work in editor playmode."
With no explanation what that means. It is not mentioned in the documentation.
If I had not fount this post I would have never found it.

 

Can a dev please provide an update on this rediculous offset sitatuion with OpenXR.
Pre - 1.1 Oculus controllers badly offset.
1.1 - Oculus controllers now spot on
1.2 - Oculus controllers offset again

I don't want to fix these offsets (which I will have to do on every controller type) for the devs to just changing everything again and redo them.

 

Did you read this post? #291

 

I did read that, however it doesn't changed the fact that what was once fixed, is now broken again. If it's going to stay like this, then fine I'll add the offsets to fix them myself however ifit would be good to get warning if they are going to mess with this futher. It's just annoying that it was spot on and now it's off by some arbitrary amount.

 

Have you checked if SteamVR beta is causing this? When I went back to the normal SteamVR branch, this issue was solved (for now)

 

I'm having this same issue. SteamVR just updated to a new version, 1.17.12, and I'm not on the beta. Now my controller positions are screwed up when they were correct before with being similar to Oculus positions. Now it's a rotation offset that is different from the normal. My understanding is OpenXR is supposed to report the positions in a similar way to get rid of all this weird offset issues.

 

Yep, I went around it by making a custom offset when Oculus Controllers are connected. It's not 100% in the center but at least it works... for now...
With SteamVR 1.16.x there wasn't any issues. Funny enough, that in the SteamVR 1.17 release notes it reads "Match Oculus grip and aim poses for touch controllers".

 

I bet they fixed it for OpenVR, which broke it for OpenXR.

 

Well, then why does work fine when I use OpenXR though Oculus insteadt of OpenVR?

 

My guess is there's logic in SteamVR that they fixed a rotation offset that affected OpenVR, but unintentionally it affected the OpenXR implementation when ran through SteamVR. This is just a guess on my part, I am not affiliated with Steam, I just hope it gets fixed soon.

 

This is with the Oculus OpenXR runtime.

EDIT: There is an offset different between Oculus Controllers on the SteamVR runtime and the OpenXR runtime....

 

I've just switched to OpenXR, but it seems like the haptics still aren't working. I can't get them work with this workaround either. I'm on 2020.3, using the OpenXR 1.1.1 package. Can anyone weigh in who got the haptics to work?

 

Updated to 1.2.2, tried on both the Knucles and the Quest 2's Touch controllers, with both device.SendHapticImpulse() and SendHapticImpulseCommand.Create() + device.ExecuteCommand().

 

Code (csharp):

1. var command = UnityEngine.InputSystem.XR.Haptics.SendHapticImpulseCommand.Create(0, Amplitude, Duration);
2. InputSystem.GetDevice(CommonUsages.RightHand)?.ExecuteCommand(ref command);

Code (csharp):

1. controller.SendHapticImpulse(Amplitude, Duration);

 

Yes, it is called. I've tried both at 1, I've even tried the amplitude up to 50 in case it's not a normalized scale (I've not found any mention of what these functions actually expect in the documentation).

 

When I try to run that sample, all I get is a long list of null reference exceptions related to the input actions not being able to bind properly. Not entirely sure what is going on.

 

2020.3. Those first two errors I also get in my own scenes (always 17 iterations). They seem to be of no real consequence, since the controllers do track fine.

 

Just tested it in a clean project, and there it does work, without any of the errors. Could it be somehow related that my project was updated from 2019.4?

 

Seems that

Code (CSharp):

1. UnityEngine.XR.OpenXR.OpenXRRuntime.name

always returns an empty string on OpenXR 1.2.2 & Unity 2021.1.10f1.

 

Unity must be in a good mood today, because I'm no longer getting the errors within my project, and it's not crashing every five minutes. The haptics in the OpenXR sample now worked, and using that code (XRControllerWithRumble.SendImpulse()), I have my haptics back!

 

We are also suffering from the ~45deg up-rotated poses. We are using Oculus Touch controllers, and observing with OpenXR 1.2.2 and xr interaction toolkit 1.0.0pre4:

- using Oculus OpenXR runtime: all good, ray is aligned with aiming of controller

Debug View shows, that pointer Rotation and device Rotation are different:


The OpenXR ControllerSample shows that aim and grip poses are different (how it should be, off by that good 45degrees):


- using SteamVR OpenXR runtime (SteamVR 1.17.12): no good, laser's are rotated ~45deg up (if based on pointer direction)

Debug View shows, that pointer rotation and device rotation are equal:


The ControllerSample shows, that aim and grip pose are identical (no nice offset of 45deg):


Is this only us or are others having the same?

 

That's SteamVR 1.17.12, i'll add it to my post

 

Re-tested again and the issue is gone, sorry for the trouble. Turns out I had not enabled OpenXR as I was switching between branches.

 

I have noticed lots of issues with oculus touch controllers and different behaviours depending on which device, We have Quest 1 & 2 via oculus link, then different behaviour on a rift and different on a rift S. This is on the latest SteamVR beta and OpenXR plugin 1.2.2

Quest 1 & 2 seem fine.
Rift S we see the right hand is fine, the left hand is incorrectly positions and the grip/aim pose are the same
Original Rift, we see both hands are incorrect (same issue as Rift S left hand)

Additionally there seems to be different offsets on different runtimes. Oculus vs SteamVR runtime are different. We are gonna publish to steam and enforce SteamVR runtime, which is a bit of a workaround, but isn't the idea that it will work on any runtime in the same way. This is supposed to be production ready right? I don't see how anybody could release a game using this tech stack right now.

I realise a lot of these issues lie with SteamVr. Is there a decent channel to report these issues directly to them and track their progress? The steam bug report system through their forums is the only way I know of but that feels more user facing, rather than developer/api facing.

 

Great, that sounds good. @DominiqueSandoz. It's not only you having this issue.

We tweaked all our controller offsets for each device to try and provide a consistent experience across devices. As many others will have done, and as many other reported that in a recent beta, those poses changed. They decided that this new pose is now correct. It appears they only "Fixed" it for specific devices.

For clarify and thoroughness, I tested all the oculus devices (Quest 1&2, Rift & Rift S) using the latest beta (1.17.16 edit: I noticed they just released a new one, although patch notes do not indicate any further fixes for this) and OpenXR plugin 1.2.2.

All screenshots are taken with hands in a relaxed natural position (aiming forward to the horizon, my head looks at them from ~45 degrees yaw)

Oculus Rift S
Left: Left hand, grip pose and aim pose are identical and incorrect. Fix has not been applied
Right: The fix has been applied


Oculus Rift
Both Hands: Grip pose and aim pose are identical and incorrect. Fix has not been applied



Oculus Quest 1 (via Oculus Link)
Both Hands: Grip pose and aim pose are identical and incorrect. Fix has not been applied



Oculus Quest 2 (via Oculus Link)
Both Hands: Grip pose and aim pose are identical and incorrect. Fix has not been applied

 

I'm experiencing some discrepancies between the Oculus and SteamVR OpenXR runtimes that are causing a fair amount of headaches in the process of porting an application to the OpenXR plugin.

The first of these is that when using the Oculus OpenXR runtime it seems like all cameras in Unity stop rendering unless the user is considered as present (i.e. having the HMD on the user's head). The game loop still runs fine in the background though, its just rendering that seems to stop. With the SteamVR OpenXR runtime, this never happens and everything seems normal when the HMD is idle. The render pausing is rather problematic as we have cases where we want the user to interact with UI on the desktop for a short period of time.

A similar issue happens whenever we try to make use of InputDevices.GetDeviceAtXRNode(XRNode.Head). The isValid property of the output is always true in the SteamVR OpenXR runtime while on the Oculus one, it is only true if the user is considered present.

Code (CSharp):

1. var head = InputDevices.GetDeviceAtXRNode(XRNode.Head);
2. Debug.Log(head.isValid); // Always true with SteamVR OpenXR runtime, Only true in Oculus OpenXR runtime if HMD considers user to be present

I've been testing this with an Oculus Rift S so I dont know if it works any different for other HMDs, but some of these differences are pretty fundamental (especially the rendering pausing bits!). Has anyone experienced similar issues or found workarounds for preventing rendering from stopping when there is no user presence?

 

The issue already starts in the editor - until the headset is properly initialized, the game view is just entirely black, even in a scene with no cameras rendering to the headset.

 

It would be great if Unity could stop messing with all the offsets, it's honestly driving me insane.

I updated the latest LTS and OpenXR plugin (1.2.2) and now in OpenXR Oculus Runtime the controllers are completely offset both positionally and rotationally. I tried an old build from previous LTS w/ OpenXr (1.2) just to make sure it wasn't an Oculus update and nope, everything is as it should be. Hop back into the updated build and controllers completely offset. I tried updating to the latest OpenXR but it made no difference, now in Unity Oculus controllers are completely offset even in the Oculus OpenXR runtime.

OpenXR needs to be put back into preview till it doesn't break every single time something change.

 

I use Unity 2020.3.11, Oculus XR Plugin 1.9.1, OpenXR Plugin 1.2.2, XR Interaction Toolkit 1.0.0-pre.4 and XR Plugin Management 4.0.6.
I get the following error messages when testing with Oculus Quest:

Could not recreate input device 'Oculus Oculus Touch Controller OpenXR (XRInputV1)' with layout 'OculusTouchController' and variants 'Default' after domain reload
UnityEngine.InputSystem.LowLevel.NativeInputRuntime/<>c__DisplayClass7_0:<set_onUpdate>b__0 (UnityEngineInternal.Input.NativeInputUpdateType,UnityEngineInternal.Input.NativeInputEventBuffer*)
UnityEngineInternal.Input.NativeInputSystem:NotifyUpdate (UnityEngineInternal.Input.NativeInputUpdateType,intptr)
UnityEditor.EditorAssembliesrocessInitializeOnLoadAttributes (System.Type[])

and

InvalidOperationException: Cannot find parent 'devicepose' of control 'devicepose/istracked' in layout 'XRInputV1::Oculus::OculusTouchControllerOpenXR'
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.InsertChildControl (UnityEngine.InputSystem.Layouts.InputControlLayout layout, UnityEngine.InputSystem.Utilities.InternedString variant, UnityEngine.InputSystem.InputControl parent, System.Boolean& haveChildrenUsingStateFromOtherControls, UnityEngine.InputSystem.Layouts.InputControlLayout+ControlItem& controlItem) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:557)
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.AddChildControlIfMissing (UnityEngine.InputSystem.Layouts.InputControlLayout layout, UnityEngine.InputSystem.Utilities.InternedString variants, UnityEngine.InputSystem.InputControl parent, System.Boolean& haveChildrenUsingStateFromOtherControls, UnityEngine.InputSystem.Layouts.InputControlLayout+ControlItem& controlItem) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:508)
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.AddChildControls (UnityEngine.InputSystem.Layouts.InputControlLayout layout, UnityEngine.InputSystem.Utilities.InternedString variants, UnityEngine.InputSystem.InputControl parent, System.Boolean& haveChildrenUsingStateFromOtherControls) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:337)
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.InstantiateLayout (UnityEngine.InputSystem.Layouts.InputControlLayout layout, UnityEngine.InputSystem.Utilities.InternedString variants, UnityEngine.InputSystem.Utilities.InternedString name, UnityEngine.InputSystem.InputControl parent) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:193)
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.InstantiateLayout (UnityEngine.InputSystem.Utilities.InternedString layout, UnityEngine.InputSystem.Utilities.InternedString variants, UnityEngine.InputSystem.Utilities.InternedString name, UnityEngine.InputSystem.InputControl parent) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:102)
UnityEngine.InputSystem.Layouts.InputDeviceBuilder.Setup (UnityEngine.InputSystem.Utilities.InternedString layout, UnityEngine.InputSystem.Utilities.InternedString variants, UnityEngine.InputSystem.Layouts.InputDeviceDescription deviceDescription) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDeviceBuilder.cs:51)
UnityEngine.InputSystem.InputDevice.Build[TDevice] (System.String layoutName, System.String layoutVariants, UnityEngine.InputSystem.Layouts.InputDeviceDescription deviceDescription) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/Devices/InputDevice.cs:678)
UnityEngine.InputSystem.InputManager.AddDevice (UnityEngine.InputSystem.Utilities.InternedString layout, System.Int32 deviceId, System.String deviceName, UnityEngine.InputSystem.Layouts.InputDeviceDescription deviceDescription, UnityEngine.InputSystem.InputDevice+DeviceFlags deviceFlags, UnityEngine.InputSystem.Utilities.InternedString variants) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/InputManager.cs:1033)
UnityEngine.InputSystem.InputManager.RestoreDeviceFromSavedState (UnityEngine.InputSystem.InputManager+DeviceState& deviceState, UnityEngine.InputSystem.Utilities.InternedString layout) (at Library/PackageCache/com.unity.inputsystem@1.0.2/InputSystem/InputManager.cs:3420)
UnityEngine.InputSystem.LowLevel.<>c__DisplayClass7_0:<set_onUpdate>b__0(NativeInputUpdateType, NativeInputEventBuffer*)
UnityEngineInternal.Input.NativeInputSystem:NotifyUpdate(NativeInputUpdateType, IntPtr)
UnityEngineInternal.Input.NativeInputSystem:Update(NativeInputUpdateType)
UnityEditor.EditorAssembliesrocessInitializeOnLoadAttributes(Type[])


Is there a fix for this?