Search Unity

Official XR Plugins and Subsystems

Discussion in 'AR/VR (XR) Discussion' started by matthewr_unity, Jun 12, 2019.

  1. matthewr_unity

    matthewr_unity

    Unity Technologies

    Joined:
    Dec 7, 2017
    Posts:
    2
    ***Platforms Supported Thus Far***
    • Oculus (Released June 11th 2019)
    • Windows MR VR (Released June 19th 2019)
    • Windows MR AR (Released July 2nd 2019)
    • Magic Leap (Released July 16th 2019)
    • Google VR (Released July 26th 2019)
    • PSVR (Released August 6th 2019: You can find it via the Unity forum on Sony's Developer site.)
    What’s New
    The Unity XR team has been working hard to improve our multi-platform offering, which includes a new plugin architecture. As a result, this changes how settings are stored for AR/VR build targets, and how the SDKs for each supported platform are loaded.

    The core part of our new plugin architecture is formed by subsystems. These subsystems implement groups of related functionality (e.g., camera, depth, display, input, etc.) and expose them as low-level managed C# APIs to developers, resulting in the following:
    • More stable, multi-platform APIs
    • Faster updates via the Unity Package Manager (e.g., the “Oculus XR Plugin” package is able to be updated outside of major Unity editor release cycles)
    • Simpler multi-platform development
    AR Foundation, the recommended approach for building AR applications on handheld devices, was also built upon this new architecture. We will be adding support for other devices that support AR features in the future.

    Starting with Unity 2019.2, there will be new preview packages for our supported AR/VR platforms that use the new plugin architecture, with the goal of being production-ready by Unity 2019.3. We plan to deprecate the integrated AR/VR packages in Unity 2019.3.

    Developing for AR/VR Platforms on Unity 2019.2

    Using The New Plugin Architecture
    The new plugin architecture has changed how Unity loads and manages SDKs of your target platforms. To use the new packages, do the following:
    1. These package are preview packages, so you will need to enable the ability to see preview packages first. In the package manager menu, there is an "Advanced" drop-down by the search bar. Open it, and you will see "Show Preview Packages". This should make the XR Management package appear. If you still don't see it, you may need to switch to "All Packages", in the drop-down below the Packages tab.
    2. Download the XR Management package from Package Manager
    3. Download the Loader for each SDK/platform you want to build for (e.g., the Oculus Loader)
    4. Select the target SDK/platform from Available
    Instead of going to Player Settings as a first step, the settings for XR and SDKs will now appear under Project Settings. The XR Management package will now serve as the main entry point for loading the right package for each target SDK/platform and managing respective settings. XR Management is also needed to make the XR Settings show up in Project Settings. Once downloaded, the XR Management package will take you to Project Settings, where loading and management of supported XR platforms takes place (see screenshot below).


    Download the loader.


    Install the downloaded loader.

    The Tracked Pose Driver is now recommended for camera tracking:
    • Add a Tracked Pose Driver component to the main camera game object in your project.
    • Set the Tracked Pose Driver to use the “Center Eye” as the pose source, and untick the “Use Relative Transform” option.


    Using Integrated Platforms
    We want to note that the integrated XR platforms are still available in Package Manager. These packages will remain as is, but we encourage you to download the XR Management package and try the new XR plugins. You may not be able to use Legacy VR if you have XR Plugins installed, as some plugins may disable Legacy VR to prevent users from using two at the same time.

    Roadmap
    Over the next few weeks, we plan to release more implementations for Unity’s other supported XR platforms under the plugin architecture as preview packages. These include the Windows MR, Magic Leap, Google Daydream, and Google Cardboard XR Plugins. During this time, we’ll be looking for early testers and feedback to get these packages in production-ready state.

    We plan to have XR plugins for most of our supported XR platforms in production-ready state by the release of Unity 2019.3 later this year (Windows MR and Magic Leap XR Plugins will likely remain in preview).

    Let Us Know What You Think
    The main goal of this preview release is to gather feedback on how the new plugin architecture is working for you so we can fine-tune the subsystems and workflows to fit your needs. Please provide input via this quick survey or ask questions in this forum.

    Developer Notes:
    D3D11 for Desktop, and OpenGLES3.0 for mobile are currently the only supported graphics APIs for the Oculus XR Plugin preview. Please note: the default graphics API for new mobile projects is Vulkan (which will be supported in the near future), so be sure to revert to GLES3.0 before deploying to your device.
     
    Last edited: Aug 7, 2019
    Corysia, ROBYER1 and EyePD like this.
  2. matthewr_unity

    matthewr_unity

    Unity Technologies

    Joined:
    Dec 7, 2017
    Posts:
    2
    Known Issues:
    • The Oculus Integration utilities from the Asset Store are not currently fully supported when using the Oculus XR Plugin.
    • Quest and Rift S controllers report a "Thumbrest" usage, which they do not have.
    • The Oculus XR Plugin is failing some publishing validation tests:
      • VRC validator test
      • TestSubmitFramesWhenNotVisible
      • TestResponseToRecenterRequest (fails on Integrated XR as well)
      • TestAppShouldQuit
    • The camera is in a different position when entering play mode compared to Legacy XR when using the Tracked Pose Driver.
      • It is possible to work around this by:
        • Create an empty game object
        • Set it at 0, 1, -10 (the default camera location)
        • Make the camera a child of this game object, set camera transform to 0, 0, 0.
        • Enter playmode and camera is now in the expected location
    • Changing the render viewport scale and opening the Oculus dash while depth and dash support are enabled causes unexpected rendering artifacts
    • XRDevice API values are not populated (Documentation)
    • Rift S is not fully supported.
    • The view is not landscape locked when deploying to GearVR in developer mode.
    • The Lightweight Render Pipeline has shadow issues when using Single Pass Instancing on desktop.
    • Occlusion meshes aren’t enabled yet.
    • In certain situations, eye textures may not destroy correctly, potentially resulting in a crash. This usually only happens if eye textures are created and destroyed repeatedly.
     
    Corysia and ROBYER1 like this.
  3. TimeWalk-org

    TimeWalk-org

    Joined:
    Nov 3, 2014
    Posts:
    38
    I installed Unity 2019.2.0b6 (under Windows), but see no "XR Management" package inside the Package Manager. Nor do I see the new "XR" category under Project Settings. I do see that there is a "Built-in package" titled "XR", but this does not seem related to the "XR Management" tool cited above. Do I need a different (alpha?) build of 2019.2 for this? I tried 2019.3.0a5 as well. No sign of the XR Management package.
     
    Last edited: Jun 18, 2019
  4. wirmachenbunt

    wirmachenbunt

    Joined:
    Jun 28, 2013
    Posts:
    7
    same question, where to start when trying to build a VR project. there is no lightweight VR template nor is the documentation leading anywhere, see question concerning the XR Management
     
    DanjelRicci and ROBYER1 like this.
  5. ThatDarnCat

    ThatDarnCat

    Unity Technologies

    Joined:
    Jan 4, 2017
    Posts:
    23
    The package is a preview package, so you will need to enable the ability to see preview packages first. In the package manager menu, there is an "Advanced" drop-down by the search bar. Open it, and you will see "Show Preview Packages". This should make the XR Management package appear. If you still don't see it, you may need to switch to "All Packages", in the drop-down below the Packages tab.

    upload_2019-6-18_10-4-35.png

    upload_2019-6-18_10-4-45.png
     
  6. joelybahh

    joelybahh

    Joined:
    Mar 18, 2015
    Posts:
    66
    Does the normal Oculus (Android) package support native virtual reality for the Quest?' Or will the support for the Quest with native VR only be available through this workflow? :)
     
    BrainSlugs83 and ROBYER1 like this.
  7. jackpr

    jackpr

    Unity Technologies

    Joined:
    Jan 18, 2018
    Posts:
    50
    The "Oculus (Android)" package does support Quest. That package will be deprecated in the future in favor of this new workflow.
     
    BrainSlugs83, ROBYER1 and joelybahh like this.
  8. RobertBuckley

    RobertBuckley

    Joined:
    Aug 23, 2018
    Posts:
    4
    Was able to get the package installed and the loader downloaded/selected, and the tracked pose driver set up, but when i attempt to play I get 'Unable to start XR SDK Oculus" and some failed messages. Also tried doing a build and the app just crashes.

    Using 2019.3.0a6 in a new project.
    Oculus@0.8.4-preview


    Is there some setup I'm missing?
     

    Attached Files:

    alexchesser and TashaSkyUpchurch like this.
  9. jackpr

    jackpr

    Unity Technologies

    Joined:
    Jan 18, 2018
    Posts:
    50
    Hi Robert, I'm curious: is the Oculus runtime app open when you try this?

     
  10. RobertBuckley

    RobertBuckley

    Joined:
    Aug 23, 2018
    Posts:
    4
    I don't even have it installed on my desktop. Working on deploying to the GO.


     
  11. RobertBuckley

    RobertBuckley

    Joined:
    Aug 23, 2018
    Posts:
    4
    So asking about the desktop prompted me to check my XR plugin management settings for desktop, i had the Oculus Loader in there and in the android. i removed the desktop one, and the errors dont show up anymore, but deploying to the GO the app still crashes before anything shows. click app -> black screen -> app to app selection
     
    alexchesser likes this.
  12. RobertBuckley

    RobertBuckley

    Joined:
    Aug 23, 2018
    Posts:
    4
    Sanity checked myself, brand new project, install xr package manager, download oculus loader, add tracked pose driver, build, install. app started, head is tracking, but there was no splash logos.
     
  13. WisockiJr

    WisockiJr

    Joined:
    Jul 5, 2014
    Posts:
    30
    should we get both ARCore and Google Cardboard VR support for 6dof on mobile devices?
     
  14. MrMatthias

    MrMatthias

    Joined:
    Sep 18, 2012
    Posts:
    191
    How will hand gestures be handled for the different platforms be handled by the new InputSystem?
     
    ROBYER1 likes this.
  15. holo-krzysztof

    holo-krzysztof

    Joined:
    Apr 5, 2017
    Posts:
    77
    Is it possible to create our own XRSubsystem for e.g. custom hardware prototyping?

    Edit: I've done some research and it doesn't yet seem to be fully possible. I've taken a look at the WMR implementation because that's the platform I'm most familiar with, and some of the things (e.g. XRDisplaySubsystem) seem to be implemented in native code with no documented public interface.

    Now the question is, what's the roadmap for this? Are you planning to expose this to developers so we can create e.g. our own display subsystem to take advantage of the stereo rendering stack and other Unity XR features?
     
    Last edited: Jul 3, 2019
  16. purnapatadia

    purnapatadia

    Joined:
    Jul 20, 2017
    Posts:
    7
    I created an empty project of 2019.2.0b7 on Mac OS. Installed XR Management. Downloaded Oculus Loader and it is showing an error :"Legacy XR is currently disabled. Unity has detected that you have one or more XR SDK Provider packages installed. Legacy XR is incompatible with XR SDK. Remove all XR SDK Packages from your project to re-enable legacy XR".

    "Virtual Reality Supported" checkbox under Player->XR Settings has been disabled due to the same error...
     
  17. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    Just to get this right: this package doesn't support Steam VR yet?
     
  18. ThatDarnCat

    ThatDarnCat

    Unity Technologies

    Joined:
    Jan 4, 2017
    Posts:
    23
    Correct, OpenVR is not available as an XR Plugin at this time.
     
    ROBYER1 likes this.
  19. pm32847

    pm32847

    Joined:
    Mar 12, 2019
    Posts:
    2
    External developers are currently still not able to create their own plugins for custom hardware that would plug into Unity's XR system, right?

    Related to @holo-krzysztof 's question.
     
    mikewarren and VirtualPierogi like this.
  20. mfuad

    mfuad

    Unity Technologies

    Joined:
    Jun 12, 2018
    Posts:
    335
    Hey @holo-krzysztof and @pm32847, right now it's not possible to create your own subsystem. We'll comment further if that changes in the future.
     
  21. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    202
    I'd be interested in understanding the pain points in configuration that you're hitting.
     
  22. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Sure, hopefully this feedback is of use and any suggestions would be welcomed - difference in inputs between controllers with/without touch pad - like the Windows mixed reality controllers have touch pad and analog sticks whereas the vive controllers have touchpads only which lead to issues with our teleportation/movement mechanisms as WMR controls usually rely on you snapping the analog stick forward to see the teleportation arc then you let go of the analog stick so it snaps back and you teleport.

    With WMR and Quest analog sticks, they are very sensitive to input, VRTK v4 teleportation by default has a rotational ability where once you have held the stick forward, you can rotate the way to face once teleported before letting the analog stick snap back to default position by letting go of it. The issue with the Quest is the analog stick snaps back and slightly moves over to the opposite side so you often end up facing 180 degrees off your intended direction(!). Further to this, the WMR analog sticks are just as sensitive and prone to this also. We are thinking of just removing VRTK v4 and writing something custom or adding a deadzone to all analog stick input vectors.

    Sometimes the tracked pose driver for left and right controllers just picks up the left and right controller by default when left as a 'Generic XR' device with the Vive, on some other headsets like the Quest or WMR Lenovo Explorer, it often assigns both devices to be the left controller, or none at all so nothing works. The workaround for this seems to be to set both left and right trackables to be the Left and right hand control devices respectively.

    When we used to start unity projects, a 'VR LWRP' preset existed with a nice little VR setup, in 2019 that seems to be gone - will this be replaced by some sort of VR template? Unreal Engine has an easy VR setup with some very basic preset teleportation and analog stick free move defaults which would be very handy to have for quick set up of VR projects. Whereas any time we make a new project we start from scratch, have to use something like VRTK or other community made solutions that could have support or documentation dropped/unfinished (like VRTK v4)
     
  23. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    202
    lets move this off to a different thread as this isn't strictly related to the xr plugin changes. we can def help in some of these. :)
     
    ROBYER1 likes this.
  24. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
  25. iSinner

    iSinner

    Joined:
    Dec 5, 2013
    Posts:
    201
    Being unable to test AR projects in the editor by using a web cam(Vuforia supports this) or the camera from the phone, i emphasize, directly in the editor, is a deal breaker for me in using AR Foundation.

    Will this be addressed? building to a device every time i need to check something is not practical.
     
    dimassavostianov likes this.
  26. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Interested in seeing OpenVR support!
     
  27. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Edit: Logged this as Case 1173593

    Getting 4 errors about 'The type or namespace name 'XRSessionSubsystem' could not be found (are you missing a using directive or an assembly reference?)' when importing Oculus Loader in 2019.3.0.a10 - submitting a but report for it now but is there a fix?
     
    Last edited: Aug 1, 2019
  28. ThatDarnCat

    ThatDarnCat

    Unity Technologies

    Joined:
    Jan 4, 2017
    Posts:
    23
    The current Oculus Plugin preview package is for 2019.2. We'll have an updated package that is compatible with 2019.3 when 2019.3.0a12 is available.
     
    ROBYER1 likes this.
  29. Cosmonaut72

    Cosmonaut72

    Joined:
    Nov 16, 2017
    Posts:
    9
    Is there any word as to when the Rift S issues will be resolved? Presently, it's our headset of of choice enterprise production
     
    soleron likes this.
  30. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    904
    So under this new package-based approach, how do we do something like enable fixed foveated rendering on Oculus Quest now?
     
  31. wechat_os_Qy01wWCkBnBAo_8MjUOB1DB4M

    wechat_os_Qy01wWCkBnBAo_8MjUOB1DB4M

    Joined:
    Jul 28, 2019
    Posts:
    1
    I can not find XR Button in eidt->project settings, after installed XR management, can anyone help?
     
  32. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    517
    I'm new to VR and wanted to try out development with this new system. I have a Oculus CV1 w/ touch.

    I start a brand new LWRP 2019.2.1 project. I download the XR Management package (2.0.0-preview26), and upgrade the XR Legacy Input Handlers to 2.0.6. I go into the ProjectSettings for XR, download the Oculus loader, add the Oculus plugin. Add a pose tracker to the main camera in the sample scene. Set it to be center eye. Hit play.
    Get a bunch of errors about IsMatrixValid, MinMaxAABB errors in my log. I only see out of my left eye.
    I change to single pass, and I can see stereo now, but everything is upside down and I still receive those errors.
    I remove the post process script from the camera. Everything is right-side up, but obviously the shadows are messed up.

    I realize I'm being verbose, but these are the hurdles that you're expecting your users to jump through to just get a basic camera attached to the HMD right now. I haven't even started with hand presence. How are you expecting users to interact with this system for the first time to get started? What happened to the VR template? How do I get rid of the assertions and errors? I don't mind some problems doing something more obscure, but this is basic first user setup and I've already hit some pretty obvious roadblocks. Please document your first setup and make sure it works without errors/problems.
     
    Last edited: Aug 17, 2019
  33. ju4nl

    ju4nl

    Joined:
    Sep 19, 2013
    Posts:
    9
    Same case for me on Windows. Just followed the instructions and had the same error. It's crazy...
     
    abelsang9683 and BiggerInsideVR like this.
  34. ju4nl

    ju4nl

    Joined:
    Sep 19, 2013
    Posts:
    9
    Absolutely agree... I just started a project trying to build for Oculus Go, followed the guidelines and didn't manage to build so far. As basic as a Camera and a Cube in scene... It's insane. Did someone in Unity tested this before testing?
     
    DanjelRicci and ROBYER1 like this.
  35. prawn-star

    prawn-star

    Joined:
    Nov 21, 2012
    Posts:
    77
    Hello
    So I'm just trying to build a simple cardboard app for Android (sphere & solid coloured camera) with this new system. To no avail.
    I get these errors repeating in the logcat
    E/GvrShimUnity: getControllerConnectionStatus device 0 api status:
    E/GvrShimUnity: getControllerConnectionStatus device 1 api status:
    App hangs at Unity splash screen

    Switch to Android build
    I installed XR Management package
    Downloaded the Loader for Google VR Cardboard
    Selected it from Available
    Hi create new to create Google VR Settings (do I need to do this)
    Added Tracked Pose Driver to the main Camera

    Am I missing something?
     
  36. badradionz

    badradionz

    Joined:
    Feb 2, 2018
    Posts:
    21
    This does not work for Google Cardboard.
     
  37. Vinayak-VC

    Vinayak-VC

    Joined:
    Apr 16, 2019
    Posts:
    60
    I am having problem creating IL2CPP APK build with ARCoreFoundation in it, MONO works fine but ILCPP gives an error

    I have attached screenshots.
    Please let me know if anyone has found the solution.
     

    Attached Files:

  38. jan_bajana

    jan_bajana

    Joined:
    Sep 15, 2015
    Posts:
    22
    I have one question here.
    What is Unity plan for OpenXR integration? Is Unity planning to integrate OpenXR runtime parallel with UnityXR? So C# XR Subsystems could use OpenXR instead of UnityXR to load device plugin?
    This would help future device providers to do only one implementation and support all graphic engines which has OpenXR integration.
    So for me as a new device provider, I would prefer to implement OpenXR from the begin.

    Thank you for an answer.
     
    Last edited: Sep 11, 2019
    mikewarren likes this.
  39. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    I'm also interested by OpenXR integration. It's already available in UE4 since 4.22 and works pretty well.
    What are the limitation to integrate it into Unity?
     
    ROBYER1 likes this.
  40. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    This issue does not happen on the latest 2019.3 beta, please upgrade to it (I wasted time trying to see if this issue happened on 2019.3 beta as well so you owe me)
     
  41. Vinayak-VC

    Vinayak-VC

    Joined:
    Apr 16, 2019
    Posts:
    60
    Actually the problem was in preview packages if you use stable packages then the issue won't happen. (So I don't owe you ;-) )
     
    ROBYER1 likes this.
  42. Skeketor23

    Skeketor23

    Joined:
    Apr 6, 2019
    Posts:
    49
    Can somebody help me to understand how cross platform development works with this new approach?
    So let's say I want to write an App that works with an Oculus Rift as well as for Windows Mixed Reality.

    Which packages can I use in this case?
    Once I use the "Oculus XR Plugin" ... then I'm stuck with Oculus?
    So what packages are cross platform? And what is the best way to set this up for cross platform?
     
  43. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    569
    > Once I use the "Oculus XR Plugin" ... then I'm stuck with Oculus?

    Oculus AND OpenVR are both cross-platform across PC VR (with a few caveats).

    However Oculus also works on the Quest (and the Go if you care about that) whilst OpenVR doesn't
     
    BaaWolf likes this.
  44. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    I too would be interested to know how to make sure that separate plugins are only built out per platform where needed
     
  45. Skeketor23

    Skeketor23

    Joined:
    Apr 6, 2019
    Posts:
    49

    Ok thanks. So when I use the new Oculus XR Plugin, I will also be able to run it on Windows MR?
    Currently I'm using only the build in UnityEngine.XR stuff, and that works also for quest. But did not do any extensive tests so far. Just basics.

    I think it would be great to have an overview, about what plugin is compatible with what headset and also some example about how to setup a pure cross-platform project, even if it is missing some functionality.
     
    Last edited: Oct 1, 2019
  46. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    @matthewr_unity I have messaged you about this directly also but it may be useful to respond back here to help others - I can see that you also mention here the Oculus Integration package is not fully supported with this XR Subsystem workflow, the issue there being that we can only set the Fixed Foveated Rendering level through the Oculus Integration package through the OVRPlugin. If we could have some documentation or advice on how to set it through the XR Subsystem Oculus Loader or any other native XR platform features through this integration, it would save us having to import those heavy example SDK packs like Oculus Integration.

    Please let us know if or how we can do this?

    @Matt_D_work
     
    Last edited: Oct 2, 2019
  47. Skeketor23

    Skeketor23

    Joined:
    Apr 6, 2019
    Posts:
    49
    Can you give more details on that? Right now I can use the native unity xr support and I can build to OpenVR and run the application on steam.
    This will not be possible anymore with the new approach?
    Will there be a "native unity xr plugin" in the future?

    Or can you give an example, about how to setup a project that uses the new cross platform stuff shown in this oculus talk:

    upload_2019-10-1_10-53-10.png

    How do I setup a project, that uses this new XR Interaction Package and that runs on Windows MR, Oculus Rift, Oculus Quest and any other possible platform?
    Will this be possible?
     
    a436t4ataf and alexchesser like this.
  48. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    202
    that is the intention :)

    right now its using the feature API from 19.1 (https://docs.unity3d.com/ScriptReference/XR.InputDevice.html TryGetFeatureValue) to map input. we're going to be doing a bunch more work to make input even better post launch.
     
  49. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,454
    Hi Matt, do you have any information on my question above Skeketor23's question? What we are asking about isn't documented at all and we are looking to activate Fixed Foveated Rendering on the Oculus Quest/Go without using the Oculus Integration off the asset store, as is recommended in this thread where it is stated that Oculus Integration doesn't work with the Oculus Loader.

    You response would be really appreciated
     
    fherbst likes this.
  50. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    569
    @Matt_D_work Any any when the interactors/interactables stuff shown here
    is going to be available? We're currently stuck trying to decide whether to reimplement some OpenVR stuff on the Oculus SDK and a peek at this would be really helpful in knowing whether to hold off for now.
     
    jbat100 likes this.