Question Possible to make the Oculus Passthrough API work with the OpenXR plugin?

Does anybody know if it is possible to make the Oculus Passthrough API work with the OpenXR plugin? It seems to work when ONLY the Oculus plugin is enabled in XR plugin management, but when either OpenXR or both is enabled, the background simply turns black.

I am trying to use the hand-tracking feature from XR Toolkit 2.3.0, which requires the OpenXR plugin to be enabled, and ideally, I would combine this with the Oculus Passthrough API.

 

Any Luck?

 

Hi, I can't seem to get it working.

I'm using the Oculus Integration v50 linked by you. I have the OVR Manager attached to the 'Complete XR Origin Hands Set Up' prefab on the XR Origin along with a OVR Passthrough Layer attached. The Main Camera environment set to solid colour (0,0,0,0) . I get the following error: Failed To Initialize Insight Passthrough. Passthrough will be unavailable. Error Failure_OperationFailed. Accompanied by black screen in both the playtime in the editor and on build. So i try to go through and fix all outstanding problems listed in the Oculus tab in Project Settings, The Outstanding issues: Oculus must be added to the XR Plugin active loaders. and Recommended items: Unity's OpenXR Plugin is not recommended when using the OculusSDK, please use Oculus XR Plug-in instead.

If i go through and fix all issues it removes OpenXR out of the plug in management and the hand tracking doesn't work.. I then get to a point where Unity completely crashes because its using both OpenXR and Oculus backends.

This has been hell to try and get working and I'm being forced to choose one or the other (I cannot use full Oculus Integration OVR Rig due to scripting conflicts)

Any help would be much appreciated.

 

Can confirm does not seem to work, just fails with Error Failure_OperationFailed (either in my own attempts, or the sample project).

 

Any Updates would be greatly appreciated!

 

Hi, Lucasperlee and I solved this issue using Unity 2022.2.11f1, V50 Integration asset, OpenXR 1.7.0, Unity XR Hands 1.1.0 and Unity XR Interaction toolkit 2.3.0 (XRI). Here is a tutorial on how we did we did it:

1. Download and install V50 Integration asset, the Open XR, Unity XR Hands and XRI, as shown in the image below (V50 integration won’t appear in package manager):
   
2. Go to the Oculus tab in the project settings and apply all issues except these two (there’s no need to install the oculus XR plug in in package);
3. If your project already has a custom android manifest you need to make a backup and exclude it from the project. In oculus project setup tool fix the issue related to android manifest to generate a new one. Now manually merge your backup with the generated one.
4. In your new merged manifest add “android:exported=”true”” in the manifest activity line
5. Check OpenXR in XR plugin Management.
6. Use these configurations in the Open XR tab in XR Plug-in Management.
7. Check the project validation tab in XR plugin management and fix everything that appears there.
   
   
8. If the project validation tab didn’t add “USE_INPUT_SYSTEM_POSE_CONTROL” in the scripting define symbols, add it.
   
9. Open a new scene and add the Complete XR Origin Hands Set Up prefab from the XRI package samples(You can unpack and create your own prefab if you’d like).
   
   
10. In the Main camera inside the Complete XR Origin Hands Set Up Hierarchy, change clear flags to solid color and background to transparent black (0,0,0,0).
    
    
11. In XR Origin in the Complete XR Origin Hands Set Up hierarchy add OVR manager with these configurations:
    
    
12. Still in XR Origin add OVR Passthrough Layer with these configurations:
    
    
13. You are ready to build the project

 

I am Lucasperlee and I approve this message

 

@ericprovencher , Interested in that!

 

I just built v51 and it worked fine with the exact same configuration as mentionned above.

 

v53 and same configuration here. If I enable passthrough, I can't see hands

 

I tried this method to use passthrough and that works, but for some reason it is breaking switching between controllers and hands. You can see the hands if you put your controllers down, but if you try to use controllers the rig won't switch to show controllers. Is anyone else running into this issue? Is there a way to solve this?

 

Hello, I've followed the steps laid out by androanHeinen, and passthrough works. However my controllers aren't tracking, they're not even showing up in the connected devices on the input debugger. The head is tracking fine. Does anyone know why that may be?

 

Were you able to resolve this? I'm also having this issue after following the same steps. HMD is tracking via openxr and input system, and I'm also getting passthrough, but the controllers are not tracking anymore.

 

The solution posted above seems to be working for me in Builds. In Play Mode, I have not had any success yet even though I have set up passthrough over link. Can anyone confirm that the above works in Play Mode? I get the following:

[OVRPlugin] [Insight MR] Initializing passthrough failed with error -1004. (arvr\projects\integrations\OVRPlugin\Src\Util\InsightMrManager.cpp:803)

Specs

• Unity 2021.3.23f1
• XRI 2.3.2
• XR Plug In Management 4.3.3
• XR Hands 1.1.0
• Oculus XR Plugin 3.3.0
• Oculus Integration 54.1

 

Let me put an end to this discussion!

OpenXR itself does not support being used in conjunction with Oculus Interaction. To address the need of "wanting to use OpenXR for passthrough on Quest", Unity and Meta have jointly launched the Meta OpenXR feature plugin! (Based on things done with AR Foundation)
(https://blog.unity.com/engine-platform/get-started-developing-for-quest-3-with-unity)

Although it's still in its early stages, it's already in a usable state! Remove Oculus Interaction, everyone no longer needs to find a way to resolve the conflict between OpenXR and Oculus.

 

hey folks not trying to hijack the thread, but if you just wanted passthrough and not hand tracking, just controller tracking, is there perhaps a method posted for that? or maybe some tips on how to alter the directions posted in that screencap tutorial earlier? also i'm the only member of the team with a Quest 1, and i'm wondering if any of the integrations support it at all? i'm on V55. i like the previous post on the Meta OpenXR plugin but i'm willing to bet there will be no support for Quest 1 with that. i'm planning on getting a Quest 3 when it comes out but i need to limp along for now. any help appreciated!

 

+1 to what @HelckOne mentioned above. My current understand is that's the future of how Unity is planning to support this. This forum post might be helpful for anyone looking at this in the future: https://forum.unity.com/threads/introducing-meta-quest-support-in-ar-foundation.1450966/

 

Isnt a way to do it without depending on que Oculus Plugin? ie. OVR and so, so it works on other Passthrough-handtracking capable headsets that are not from Meta? we try to use the as least platform exclusive code as possible

 

Yes you can @Extrys , using OpenXR + Unity XR Interaction toolkit AND Meta or Pico OpenXR plugins (not sure for HTC's VIVE XR Elite though)

• If you wanna use Meta passthrough with OpenXR :

Add Meta OpenXR package (com.unity.xr.meta-openxr) and check the capabilities in OpenXR settings but also in the submenu Meta Quest Support options.



• If you wanna use Pico 4 passthrough with OpenXR :

Add from disk the "just released" package PICO Unity OpenXR 1.1.0
from Developers portal and make sure to follow these few steps to enable the right capabilities at start or whenever you want with a script.

WARNING : I had many troubles using this new OpenXR plugin.
Like the broken ray interactor in MR (which works in the last package with Meta thanks to XRI's team), but also the hands renderer and the XR Origin Hands.

You can still use PICO Unity Integration SDK which works with XRI (passthrough + controllers & hands).



I got to say Unity's team is making a great work with OpenXR and HMDs brands to standardize XR developement.

 

@Yann_de_Couessin what version is that? In the OpenXR Feature Group, for Meta Quest, I only have "AR Foundation: Meta Quest Features" and "Meta Quest Support". I'm on package Meta OpenXR Feature version 0.1.2

 

@ericprovencher have you managed to make Passthrough work in Unity via Link cable/Air link? Or does the game need to be exported to the headtset for the Passthrough to work?

Edit : I see now that it's mentioned on GitHub that Link doesn't currently work, thank you!

 

Thank you, this is very relevant to my use case! Just to clarify for my own sanity:

Assuming I want to develop something that works for both the Quest and Pico, I can create an app that works for both just using OpenXR by checking the "Meta Quest Support" checkbox and downloading and installing the "PICO Unity OpenXR SDK". This just supports basic OpenXR features like controller tracking and button input.

Previously, to access Quest-specific features like passthrough, you would need to install the "Oculus Integration SDK" which was built on OpenXR, but not compatible with Unity OpenXR features. However now, on newer versions of Unity, it seems like there's a package you can install from the Unity Registry titled "Unity OpenXR Meta" that allows you to use Quest-specific features using Unity's OpenXR. Is that all correct to say?

 

yes @m0nkeybl1tz exactly. Just one thing : the passthrough won't work from the editor for now. You need to build the project on the Quest.

 

@Yann_de_Couessin great info so far. So for further clarification, is passthrough method supported for Pico without the full pico integration as well. I am interested if this method works for hands and overlays too.