Search Unity

Feedback Follow-Up on XR Platform Blog Post - How are the various systems supposed to work together?

Discussion in 'AR/VR (XR) Discussion' started by fherbst, Jan 27, 2020.

  1. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    468
    After reading the recently released XR Platform blog post, I, as many other users, had a lot of questions that this left unanswered. I commented, and got a direct reply by Unity, that still leaves questions. So this forum thread is to follow up on that!

    Please note that I really don't want to argue which features are supported and working in theory and practice. I just want to get an understanding of how all the different New And Shiny Things are supposed to align with each other. Ideally, I'll figure out what I'm doing wrong.
    Also, please don't reply with stuff like "this and that is in preview" or "you can keep using the deprecated systems for so and so long". Previews aren't really previews if they can't even be tested, and starting new projects on officially deprecated tech isn't great either.

    1) Quest support with the new XR Plugin System
    • Currently, neither latest released 7.1.8 URP nor latest URP master (as of today) work on Vulkan at all in VR.
      I would really like to understand why Unity keeps saying it does on 2019.3 - what am I missing here?
    • This is a screenshot of Unity's sample URP scene (with postprocessing disabled, updated to latest SRP master (same on 7.1.8), updated to XR Management, using Oculus Loader, using Vulkan, stereo mode set to "multiview") on Quest:
      upload_2020-1-27_21-14-48.png
      One thing I noted is that LogCat outputs
      Unity: Multi pass stereo mode doesn't support Camera Stacking. Overlay cameras will skip rendering.

      despite rendering mode being set to "multiview" and there not being any stacked cameras in this scene.
    • On OpenGLES3 it looks slightly better - left eye renders properly. Right eye is still black.
      So my conclusion is that Quest does not work with URP on XR Management yet and I really hope I'm just forgetting a stupid toggle here.
    • Bug Reports that are blockers:
      • Case 1214807 [URP] Quest with Vulkan is broken on new XR Plugin System
      • Case 1214827 [URP] Quest with OpenGLES3 is broken on new XR Plugin System
        (note: I also tested this today with the latest SRP master branch and latest 2019.3 branch. Same issues.)
      • Case 1215369 Quest Vulkan built-in render pipeline broken
    2) Quest support with "deprecated" VR
    • Only URP version I know of that supports FFR, does not break ShaderGraph and works on Quest is my own fork, which is pretty old by now and still does not have GPU Instancing. Newer official versions always break one or another feature.
    • Bug Reports that are blockers:
    3) Input System support (this is mixed between VR / AR)
    • URP is throwing continuous errors with Input System. This bug is known since at least August:
      https://issuetracker.unity3d.com/is...input-handling-is-set-to-input-system-package and QA today told me "there's no timeline for this".
    • AR Foundation does support Input System, however, the AR Foundation Samples don't. It's not that hard to change that (it is for beginners though!) but gives me the feeling that the AR Foundation team isn't really testing with the new Input System.
    • XR Interaction Toolkit does not support Input System.
    • Touch Samples for Input System work, but as soon as you try to use them outside the sample project all the "Composite Bindings" are missing.
    • The Input System examples only work in "New Input System" exclusively, they don't work if "Both" (old & new) are active
    In an ideal world, someone at Unity would sit down and actually try to create real-world, cutting-edge projects - combining packages, really - that compile in Editor and build to a device and still have the existing samples functioning:

    A) Starting a new Handheld AR Project right now and wanting to use the latest and greatest features:
    ARFoundation + XR Management + ARCore XR Plugin + ARKit XR Plugin + Input System + XR Interaction Toolkit.
    Bonus points: URP, ShaderGraph, not using the "Both" mode for Old and New Input but solely "New"

    B) Starting a new VR Project right now for Oculus Quest
    URP + ShaderGraph + XR Management + Oculus XR Plugin + Input System + XR Interaction Toolkit

    C) Bonus Points: Starting a new VR Project right now that is cross-platform between Rift, Vive, and Quest
    (disclaimer: I know this isn't really possible right now since we're all waiting for a reaction from Valve)

    @LeonhardP it would be great if you could get Matt Fuad to see this and ideally reply here - I'd really like follow up with him about the remaining questions after he replied to my initial batch of questions. Thanks!
     
    Last edited: Jan 30, 2020
    JamesClow, gjf, zlsa and 7 others like this.
  2. cbboyer

    cbboyer

    Joined:
    Jun 2, 2014
    Posts:
    11
    That blogpost is absurd in light of these open questions - 10/10 post, getting to the core of what developers on the ground need to do to be successful with VR. A simple response from Unity about what configuration exactly should be used for Quest + URP + Vulkan would be enormously useful.
     
    ROBYER1 and zlsa like this.
  3. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    671
    Also wondering how we actually activate FFR on Vulkan on Quest as it's not documented anywhere and Matt mentioned it is out there and working?
     
    zlsa likes this.
  4. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    671
    zlsa likes this.
  5. Corysia

    Corysia

    Joined:
    Mar 14, 2018
    Posts:
    92
    Oculus Quest with URP + Multi-View + OpenGL ES3 (remove vulkan) renders fine for me. Add Vuklan in and I get the double-view in the left eye and black right eye.
     
    ROBYER1 likes this.
  6. StayTalm_Unity

    StayTalm_Unity

    Unity Technologies

    Joined:
    May 3, 2017
    Posts:
    135
    I can't speak for graphics, but I'd like to clear things up regarding the New Input System.

    All low level input XR APIs work in both old and new input systems. Under the hood it's the same data, and the only difference when setting Active Input Handling is which developer-facing system we send that data to.

    Due to the transition, old samples are using existing APIs, New Input System is using New Input System APIs, and how the New Input System throws warnings when it's set as active and the other APIs are called, some samples use one format and some use another, but setting that Active Input Handling to both should allow both to work seamlessly. If that's not the case, report a bug, and tag me, since I'll be the one fixing it.

    XR Interaction Toolkit being migrated to the New Input System is on my roadmap, and will be done soon.
    We wanted to avoid feature creep and get it out the door and available for feedback. We look forward to the greatly added abstraction that Action Maps provides for more custom actions.
     
    Gruguir, gjf and ROBYER1 like this.
  7. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    468
    Thanks @StayTalm_Unity for the Input System heads-up.

    While it's totally understandable that there's mixups during the transition, I'd argue that "supporting the systems that are not deprecated" is not feature creep but essential to get good feedback.

    Otherwise I think this (as in the case outlined above) just hinders adoption instead of enabling it - yes, I want to start new projects now (that either have pretty long or pretty short timeframes) on a set of non-deprecated packages, and not sprinkle in some deprecated and soon-obsolete stuff. I want to give good feedback on the full thing, from a developer's perspective, to see how my development experience will be for the next years - not go through each new system one-by-one and test it against the "old way".
     
    Last edited: Jan 29, 2020
  8. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    652
    Thanks so much for communicating with us - we really do appreciate it!

    Do you know who we *can* reach out to about Vulkan on URP and getting fixed foveated rendering turned on?
     
  9. StayTalm_Unity

    StayTalm_Unity

    Unity Technologies

    Joined:
    May 3, 2017
    Posts:
    135
    Report a bug please.
    I choose to come on here to keep abreast of things, but I don't want to force my colleagues to do the same unless they feel it helps them.
     
    zlsa, Corysia, ROBYER1 and 1 other person like this.
  10. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    671
    @Claytonious if you are also able to bug report your issues with Vulkan on Quest and URP and post those back here, I will try to get QA to chase them
    We have reported these cases
    • Case 1214807 [URP] Quest with Vulkan is broken on new XR Plugin System
    • Case 1214827 [URP] Quest with OpenGLES3 is broken on new XR Plugin System
      (note: I also tested this today with the latest SRP master branch and latest 2019.3 branch. Same issues.)
    • Case 1215369 Quest Vulkan built-in render pipeline broken, rendering both eyes overlapped into left eye
     
  11. Shaunyowns

    Shaunyowns

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    55
    Hello everyone, just wanted to chime in and say that we’re listening. We are actively looking into the issues that have been brought up on this post and elsewhere and we’ll respond in full when we assess the situation.

    Thank you for all of your posts!
     
  12. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    671
    @Shaunyowns thanks for the update

    To add to this feedback, user @Claytonious posted about activating the announced Fixed Foveated Rendering (FFR) on Vulkan Multiview on Oculus Quest/Go here
    https://forum.unity.com/threads/ocu...-0f6-with-vulkan-cross-eye-distortion.817878/

    From my own testing as well as Claytonious' post, there currently is no documentation on how to activate it or adjust it when using the Oculus XR Plugin on its own with XR Management, but in the android logs, it reports this, which does tell us there is a way to adjust it using the Oculus XR Plugin alone:
    "Vulkan FFR is supported, with density map size 32x32"

    Where there should be some guidance on how to activate FFR:
    https://docs.unity3d.com/Packages/com.unity.xr.oculus@1.1/manual/index.html
     
  13. TashaSkyUpchurch

    TashaSkyUpchurch

    Joined:
    Jan 25, 2018
    Posts:
    10
    I know Unity can fix this! And I hope when they do, they also manage to fix, how the system that produced the problem is broken itself... It really made me sad to see a nice release video for 2019.3 saying everything thing was working.. and then it just was not working at all. same old problems as beta 2019.3
     
unityunity