Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

XR Plugins and Subsystems

Discussion in 'AR/VR (XR) Discussion' started by matthewr_unity, Jun 12, 2019.

  1. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    75
    We're aiming to release at the same time as 2019.3 RC.

    @ROBYER1 i'm not the right person re: FFR will see if I can find more info.
     
  2. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    75
    yep, this is something we're cooking up.

    basically, there are different plugins per hardware API, much in the same way there used to be different options under the player settings / XR menu. you can assume that each plugin will support the same spread of devices as the previous internal integration. the order of loaders is also similar to the way the ordering worked in the player settings / xr menu.
     
  3. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    222
    So December-ish?

    Will there be a preview of any of the interactable/interactor stuff available earlier than that? I'd love to try it out.
     
  4. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Thankyou, I have requested a response from the original Matt who started this thread as hopefully he should know.

    To be concise for when he returns to the thread to reply - we need to know if we can activate Fixed Foveated Rendering on UniversalRP (or built-in while URP isn't compatible with FFR) using the Oculus Plugin used in this XR Subsystem package. We don't want to import the Oculus Integration package from the asset store which contains its own Oculus Plugin, which according to this thread also isn't compatible with the XR Subsystem integration of Oculus for Android anyway.

    Which means we need to know if we can access or activate it programmatically within our app through the XR Subsystem integration of the Oculus Plugin. Is there a way to do this?
     
  5. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    75
    The package should be out as a preview package before December-ish!*

    *with a very high % certainty but a very low % of commitment to an actual date ;) We'll do our best to keep you updated for the actual date we go live :)
     
    andybak likes this.
  6. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Right, things are pretty confusing overall to keep track of with many conflicting pieces of information. Thanks for update.
     
    appymedia, ROBYER1 and Matt_D_work like this.
  7. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    We are still waiting to hear from anyone at Unity to see if we can activate FFR on Oculus Quest without the Oculus Integration imported in the project. It even says here back on this post that the Oculus Integration is not fully supported with XR Subsystems and I also am sure I read somewhere else that there is a clash between the OVR Plugin dll and the XR Subsystems copy of the OVR Plugin?

    If we navigate to XR>Settings>Oculus Settings.asset and open it with notepad, we can see the following settings, could it be possible to set the FFR mode in there too or via script accessing the OVR Plugin it hooks to?


    Code (CSharp):
    1.   m_Name: Oculus Settings
    2.   m_EditorClassIdentifier:
    3.   m_StereoRenderingModeDesktop: 1
    4.   m_StereoRenderingModeAndroid: 2
    5.   SharedDepthBuffer: 1
    6.   DashSupport: 1
    7.   V2Signing: 1
    8.  
     
  8. areavisuale

    areavisuale

    Joined:
    Jul 23, 2015
    Posts:
    44
    I don't understand very well how XR Management works. Before it appeared in the Project Settings I downloaded the ARCore and the ARKit plugin packages, and this was enough.
    Now, with XR Management I had to create 1 loader for each plugin and link them to the corresponding platflorm (ARKit -> iOS ARCore ->Android). Correct?

    Now I tried to disable "Initialize on startup" for start AR in a second time (https://docs.unity3d.com/Packages/com.unity.xr.management@2.0/manual/EndUser.html), but if I have the ARSession and ARSessionOrigin gameobjects in scene, AR starts automatically anyway.
     
  9. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    I find the legacy naming scheme for something essential you need right now, is a little bit crazy. You would do well to call it Interim Input System, not Legacy. Legacy is incorrect use of language for this intention.
     
    appymedia, fherbst and ROBYER1 like this.
  10. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    75
    totally agree that using "legacy" was a bad idea, unfortunately changing it at this point would likely cause more confusion. The plan is to release a package simply called 'com.unity.xr.inputhelpers' for 2020.1 (it'll work back to 19.1) which will have a newer version of the TPD.

    Hopefully that'll clear up some of the confusion there.
     
    fherbst and hippocoder like this.
  11. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Brilliant thanks, still waiting for a response about activating the Fixed Foveated Rendering through the new Oculus Loader in XR Subsystems without importing the Oculus Integration asset pack and the OVRPlugin.dll which clashes with the Oculus Loader itself.
     
    Claytonious and appymedia like this.
  12. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    590
    @matthewr_unity?
     
    ROBYER1 likes this.
  13. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Unity staff knows about this fully but a response might not be up to the XR team, as it seems to be oculus side / pipeline side too, seems like one of those things we are best being patient with.
     
  14. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    It would be wonderful to be able to activate it through the Unity XR APIs without pulling in loads of DLLs like the Oculus Integration Package that also collide with built in Unity support for those platforms and also different storefronts that do not allow other competitors DLLs in apps.
     
  15. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Yep you will not need to do anything like that. XR will cover it all. I am patiently digging through the API. I do not use nor will use any Oculus components from the asset store. Oculus doesn't want us to either.
     
    ROBYER1 likes this.
  16. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Please let me know if you find anything in the API, I already searched the docs - I can see on the Oculus Native docs for Go/Quest that FFR is a system property you can set, wondering if this is possible through a script

    Code (CSharp):
    1. First, you may wish to check if the device supports foveated rendering. Check the system property VRAPI_SYS_PROP_FOVEATION_AVAILABLE, which will return VRAPI_TRUE if foveated rendering is supported.
    2.  
    3. Then, to use FFR, call the following to set the degree of foveation -
    4.  
    5. vrapi_SetPropertyInt( &Java, VRAPI_FOVEATION_LEVEL, level );
    6. Where level can be 0, 1, 2, or 3.
    7.  
    8. 0 disables multi-resolution
    9. 1 low FFR setting
    10. 2 medium FFR setting
    11. 3 high FFR setting

    https://developer.oculus.com/documentation/mobilesdk/latest/concepts/mobile-ffr/
     
    hippocoder likes this.
  17. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Yeah it doesn't make sense for us to be using any of the oculus API calls here, it should all go through the XR API if I'm not mistaken. I would love Unity to correct me though. A moderator can't be giving out bad advice after all ;)

    I have a fair shake of suspicion Unity wants to sort out Vulkan first so they don't do work twice, but in the meantime we have to guess and point and wonder :p

    It's all to do with the MSAA/FFR/Resolve nonsense that's going on isn't it? Stuff that unmentionable competitors have sorted out, but appears to be pretty tricky at the moment here.
     
  18. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    I am awaiting a response from the XR Graphics Lead on it, the people need answers! And yes oh my god there are many issues with Opaque textures, UI not rendering, depth issues, post processing for VR is being fixed up and who knows what the state of Vulkan is but Unreal has had that down for ages now. I never chose to use Unity to be stuck waiting for fixes and making up excuses for the engine developers :(
     
    hippocoder likes this.
  19. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Does seem like it's more on the SRP side than it is on the XR side. Maybe a bit of both. But Both is precisely what the hottest selling Mobile VR device needs.

    We're talking about a literal game changer, and horrifyingly, other engines have it completely nailed :/ Horrifying because I'd much rather be in Unity land...
     
    appymedia, fherbst and ROBYER1 like this.
  20. appymedia

    appymedia

    Joined:
    May 22, 2013
    Posts:
    85
    I love Unity and am a Plus subscriber but reading your comments and my own experience again today with the little time I had yet again I'm running into Unity VR issues as opposed to my own problems is really getting to me. Unity guys I'm not sure you realise how confusing and a minefield it is when your a user trying to do VR stuff esp on the Quest (I thought dev for the Rift S was fun) as I'm now finding out myself ☹️.
     
  21. daves

    daves

    Unity Technologies

    Joined:
    Oct 18, 2018
    Posts:
    8
    Hi @ROBYER1, we're working with Oculus to make FFR supported better for Unity's XR rendering. Currently, FFR only works when you render directly into the XR eye textures since it's the Oculus plugin that enables the variable-rate shading that FFR uses; Unity's rendering wasn't involved in setting the necessary parameters. This is also the reason that FFR is disabled if there's a blit or resolve into an intermediate render target. We're working on getting the necessary data from their plugin so we can apply it "higher up" in the rendering process. That work's underway, but not completed yet.

    Hope that helps a little.
     
    Claytonious and hippocoder like this.
  22. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Me @fherbst and @hippocoder would be interested in the FFR being accessible through the Unity XR Apis similar to how you set Multi Pass/Single Pass and various other settings through the implementations of the XR plugins through Unity. Without the need for any platform specific dll - just something we can call via code if the device has the capability to do it like in the new XR Subsystems Management being rolled out through Package Manager.

    **edit** Have just seen this is something support cross-platform with Vulkan, this would make sense as the next course of action for FFR support rather than using it only on Oculus through the OVRPlugin. It would make sense to use the Vulkan implementation of it for cross-platform FFR and simplifying cross-platform development.

    It helps to know the work is happening and your team are aware of the blit/render target override.
     
    Last edited: Oct 15, 2019
  23. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    +massive thank-you to all the hard working staff at Unity and Oculus, let's not forget :)
     
  24. DanjelRicci

    DanjelRicci

    Joined:
    Mar 8, 2010
    Posts:
    172
    Thanks for all the updates on XR plugins. I don't have much free time to iterate for my VR experiments (I'm not doing this for work unfortunately) so I'm really looking forward to the moment all these plugins will snap into place.
    I would like to ask, especially to Hippocoder and ROBYER1, do you have any suggested combination of Unity and plugin versions to get things working quickly on Oculus Quest? I'm having all sort of issues with all the latest versions, from wrong floor height to being unable to perform any hand gesture, plus compilation errors simply by following Oculus docs.

    There's this anecdote in particular that makes me itchy about XR not yet working well in Unity: a coworker brought its Daydream in the office two days ago, showing me an Unreal demo he made in basically an hour after dinner. It was a good example of dynamic lighting so I tought to replicate it on Quest to see how it performed... Two evenings in and I have yet to compile something that works the way I expect. :(
     
    ROBYER1 likes this.
  25. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Just get the oculus stuff from asset store (search oculus) and try the example scenes and off you go. It's pretty much that simple. Everything else (where I am) is time consuming in setup and you need to code everything from scratch.
     
  26. DanjelRicci

    DanjelRicci

    Joined:
    Mar 8, 2010
    Posts:
    172
    That's what I did originally, but it just didn't work. I didn't need any of the Oculus stuff except of the Quest basics (room scale and hand gestures), but these two weren't working either.
    In the end I had to revert back to a version I was sure it was working for me (2019.1.4f1) and luckily things went smooth from there on. I believe the issue was the fact I was using a beta version of Unity, but it's a pity since in the end I couldn't try all the latest workflow in VR plugins and URP.
     
  27. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    My current project is in two states: one using built in unity 3d renderer and the 'legacy XR Input systems' package to handle the controller tracking. The other project version is the same but using the new Universal Render Pipeline - but that version is experiencing many bugs.

    Stick to 2019.2 or the more recent 2019.3 beta both work fine, use built-in Unity renderer and not the new SRP stuff like URP(LWRP). The current Oculus Integration stuff should be working fine on those just give me a message if it isn't and I can have a look into it?

    I am using the 'Legacy XR Input' package (or similarly named I am not at my PC right now) and that works perfectly for room scale tracking and head/controller tracking.

    For inputs I just nabbed all the Input mappings from the Oculus Integration, but they are the same as the table in this help page. https://docs.unity3d.com/Manual/xr_input.html

    The rest is custom to your project - I chose to not use the Oculus bits and made my own character controller (it was actually very easy after looking at the Oculus player controller scripts). I also made sure roomscale tracking was working with this code which is very important on Quest as the tracking space type is not set to roomscale by default(!!)

    Code (CSharp):
    1. UnityEngine.XR.XRDevice.SetTrackingSpaceType(UnityEngine.XR.TrackingSpaceType.Roomscale);
     
    DanjelRicci likes this.
  28. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Like you said it does take a bit of time to do things from scratch but it actually is beneficial to as JoeStrout has said elsewhere on the forums before that actual time you spend making your player controller for VR is tiny in comparison to the rest of development. Plus you have full control/understanding of what is going on in your app rather than just using prefabs from things like VRTK and Oculus Integration then wondering why they don't always behave correctly.

    Maybe I will write a tutorial sometime or share the prefab of my player controller to help others but I want to wait for the Unity 'XR Interaction package' which is pretty much the same thing I have made but with updates from Unity (hopefully).
     
  29. asa989

    asa989

    Joined:
    Dec 18, 2015
    Posts:
    31
    is Windows mixed reality controller supported at this moment?
     
  30. DanjelRicci

    DanjelRicci

    Joined:
    Mar 8, 2010
    Posts:
    172
    Thanks for your support, I settled on 2019.2 now and everything really works the way I expect (LWRP, room scale, hand gestures, Android manifest, etc). I want to use a fast SRP for performance so going with the built-in renderer is not an option. I will switch to 2019.3 when everything will work as designed, As I said, my weekly time for this project is incredibly limited so I really need things that work out of the box- as they should.

    You can easily reproduce my issues by using 2019.3 beta (though I'm on OSX), adding Oculus integration, using URP, and making a scene with a bunch of cubes, one OVRCameraRig , and two OVRCustomHand. After various tests I managed to make things mostly work but no way for hands to do any gesture... But it doesn't matter now, on 2019.2 it's all working without hassle.

    I'll take care myself for things like locomotion, interaction, and such. As you said, writing your own is the best way to keep all the code under control.
     
  31. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    LWRP is now UniversalRP, neither are really truly production ready for VR. Use them at your own risk!
     
  32. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    From the very first 4 lines of the first post in this thread

    ***Platforms Supported Thus Far***
    • Oculus (Released June 11th 2019)
    • Windows MR VR (Released June 19th 2019)
     
  33. brunocoimbra

    brunocoimbra

    Joined:
    Sep 2, 2015
    Posts:
    53
  34. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    222
    I kind of presumed that URP was just a rebranding/renaming of LWRP. It seems to share most of the same code.

    Is it really sufficiently different to consider it as going back to beta status?
     
  35. brunocoimbra

    brunocoimbra

    Joined:
    Sep 2, 2015
    Posts:
    53
    Well, URP is only available on 2019.3, which is on Beta, so I consider URP as being on Beta too until 2019.3 is officially released and Unity announces that URP is ready for production.
     
  36. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
  37. pccross

    pccross

    Joined:
    Jun 20, 2015
    Posts:
    93
    Is support for Steam 2.x on the roadmap (via OpenVR, I'm assuming)?
     
    ROBYER1 likes this.
  38. asa989

    asa989

    Joined:
    Dec 18, 2015
    Posts:
    31
    Is it possible to run two WMR VR headsets on one computer using XR plugin?? im assuming the new input system can give me all the data that I need from headsets and controllers, but my concern is if unity does rendering on 2 headsets! also, can I have 2 different platform in one project?
     
  39. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    I surely can't be the only person having a collision between XR Management and Oculus Integration - OVRManager in my project?

    "Type or namespace name 'XRDisplaySubsystem' could not be found"

    Collision between Oculus Integration and XR Management.PNG
     
  40. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    590
    I have Oculus imported and am using OVRManager and the new XR Management package and am not having this problem. Our Unity versions probably differ. I'm on 2019.2.9f1.
     
  41. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    It's interesting also that there is no more 'Mock HMD' option in this setup, considering we often use the Mock HMD for development and rapid iteration, it would be great to have it back. Also the lack of FFR for Oculus Quest due to the collision between Oculus Integration and the Oculus Loader is a big no-no for my project still, preventing me from moving to using this new feature.
     
  42. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    222
    Also curious why OpenVR is being left out of the initial launch. There's got to be a story behind that. Is Valve being uncooperative? Are there technical hurdles?

    This is going to be useless for most projects without OpenVR/SteamVR support.
     
    brunocoimbra likes this.
  43. gijoe92

    gijoe92

    Joined:
    Feb 24, 2013
    Posts:
    32
    I'm running into issues because of the lack of OpenVR/SteamVR support.

    Prior to 2019.2, I was working with OpenVR/SteamVR and MagicLeap in the same project, but now I am no longer able to do that. To support both platforms now, I need to maintain two separate Unity projects or go through the hassle of removing MagicLeap support before I can get access to the Legacy XR features.

    Is there any timeframe for when we can expect to see OpenVR/SteamVR support in the new XR Management features?
     
    brunocoimbra likes this.
  44. Colton_Kadlecik

    Colton_Kadlecik

    Joined:
    Nov 18, 2013
    Posts:
    12
    Is there any kind of event that happens when the SDK has been loaded? The old system would change the main cameras FOV so Screen Space - Camera Canvas's would fit properly, however the new one does not. I can manually change the FOV and other settings such as near plane etc. just need to know when an SDK has loaded.
    Cheers,
    Colton
     
  45. pccross

    pccross

    Joined:
    Jun 20, 2015
    Posts:
    93
    When I was at Oculus connect back in September, the Unity XR presentation they gave seemed to indicate that the controller mappings in SteamVR 2.0 were a big challenge to integrate, which is why the long delay in providing support for it. I wish someone from Unity would chime in on this thread to provide any updates.
     
    ROBYER1 likes this.
  46. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    Would be great even to just have a staff member post that information somewhere so that we all understood.
     
    pccross likes this.
  47. Ethan_VisualVocal

    Ethan_VisualVocal

    Joined:
    Mar 23, 2016
    Posts:
    157
    @daves @Matt_D_work @tdmowrer
    TLDR:
    What's the vague timeline of Unity Cardboard support for the "new stereoscopic rendering path"? Is there maybe a 2019.3 preview package we can test?

    Google VR is dead so we're pretty concerned about how to continue to support Cardboard and expand XR in our actively shipping product.

    ---
    The gritty details:
    Our specific scenario is we must support Cardboard as the baseline scenario, and are actively working on adding Hololens 2 support. However:
    So, we need updated Cardboard support, but "Google VR" is dead -- Google's Daydream VR is no longer being sold, and they have archived (made read-only) their Google VR for Unity github repo: https://github.com/googlevr/gvr-unity-sdk

    Thankfully Google is (finally) open-sourcing Cardboard. https://developers.googleblog.com/2019/11/open-sourcing-google-cardboard.html ... and at the very end of Google's blog post:
    But... When? I'm pretty terrified of the timeline here, since there's not a lot of incentive for Google to put much for resources into this.

    I'm guessing there will be no "open source" repo for the Cardboard Unity SDK, since it seems (to me) to require integration with private Unity APIs?

    To me, the ideal scenario is Unity maintain this package moving forward now that Google is divesting itself of VR-with-a-phone tech, since Unity has paying customers for this feature and Google doesn't.

    Thanks... and apologies for the wall of text!
     
    ROBYER1 likes this.
  48. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    Google does tend to drop off support for things sooner rather than later, but what stops you from staying with a known and working setup? do you need to upgrade?
     
  49. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    440
    @Matt_D_work Is anyone working on the Oculus Loader going to add back the 'Low Overhead Mode' option as it's completely missing from the options shown in the Oculus Loader menu - see the comparison screenshot below

    oculussettings.PNG
     
    hippocoder likes this.
  50. Ethan_VisualVocal

    Ethan_VisualVocal

    Joined:
    Mar 23, 2016
    Posts:
    157
    Yes -- key bit from my mega-post: "That means that currently, you cannot mix Cardboard (which uses the legacy path) and the new Hololens package (which uses the new one)."

    This isn't a "ship blocking" issue right now, since the new Hololens package is only in Preview, but it will very shortly become one.
     
    ROBYER1 likes this.