Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Official Composition Layers Experimental Package is now available

Discussion in 'AR/VR (XR) Discussion' started by triciabecker_unity, Feb 26, 2024.

  1. triciabecker_unity

    triciabecker_unity

    Unity Technologies

    Joined:
    Jan 29, 2024
    Posts:
    3
    We’re thrilled to announce the experimental launch of the Composition Layers Package! Composition Layers are designed to significantly enhance the rendering of graphics in XR applications, providing a notable improvement in the visual presentation of text, images, and UI elements such as menus, along with a marked enhancement in video quality. Greater visual fidelity is achieved without an increase in element size or overhead.


    Key Features:
    1. Cross-Platform Solution: The Composition Layers package works for all OpenXR runtimes and compatible hardware.
    2. High-Quality Rendering: By bypassing one of two sampling processes, Composition Layers reduce the visual artifacts in the render pipeline. This results in enhanced image quality, offering clearer and more engaging visuals.
    3. Improved Video Quality: Videos rendered via composition layers result in larger, clearer screens suitable for displaying video content in XR environments.

    Getting Started Tips:
    • Compatibility Check: Make sure you’re using Unity 2022.3.7 and up. Both Built-in Render Pipeline and URP are supported.
    • Prerequisites: Before working with the Composition Layers Package, make sure you’ve imported the Open XR 1.11.0-exp.1 Package into your project. Additionally, there are some graphics settings that you must adjust to ensure that the Composition Layers display correctly in the Editor and at runtime.
    • Current Limitations:
      • Composition Layer Interactive UI components currently only support Quad and Cylinder layer types, and a single canvas per Composition Layer UI.
      • Single Pass Instanced rendering is not currently supported. We are planning to add support in the future.

    Additional Notes:
    • You can install the Composition Layers Experimental Package via the ‘Add package by name’ dialog box in the Package Manager. See Installation Instructions for a step-by-step guide on how to add the package.
    • Composition Layers are emulated by default in Scene View and Game View while in Play Mode. When not in Play Mode, Composition Layers are not emulated in Game View. Emulation can be disabled in Unity Preferences.
    • Emulation for Build Mirror View Window (PCVR build target) is disabled by default. You can enable emulation in the CompositionLayersRuntimeSettings asset.
    • It is recommended to use fewer than 15 Composition Layers per scene.

    For detailed instructions on working with Composition Layers, refer to our Composition Layers documentation.

    As a reminder, Composition Layers is currently an experimental package and should be used with caution in production applications. You can find more information on using experimental packages in Unity here.

    We’re excited to see how you integrate Composition Layers into your projects and look forward to hearing your feedback. Feedback can be shared on this thread, or through our XR roadmap Compositor Layers card.
     
    Last edited: Feb 29, 2024
  2. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    917
    How about HDRP support?
     
  3. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    482
    Could this be part of the Open XR package instead of having yet another package to deal with?
     
  4. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    314
    Putting everything in the same package is not ideal for iteration velocity and it makes it more complex for QA to review changes to let things get out the door. This package has a lot in it, and it would weigh down the OpenXR package a fair bit.
     
    DevDunk, goezdemir and laurentlavigne like this.
  5. KrabbyQ

    KrabbyQ

    Joined:
    Dec 19, 2016
    Posts:
    6
    This is so very cool! Kudos to the XR team who are making a lot cool releases recently.

    Saw "Behind the scene there is a hidden camera that captures the texture of the Canvas" in the Composition Layer UI section. Is it possible to use it as a standalone feature to get the texture of UI canvas?

    Can't even count how many times I had to set up custom layer and camera for capture canvas texture in Unity. I am sure a lot of Unity devs working with UI could use this streamline process.
     
  6. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    590
    Awesome!

    I was looking on the Provider implementation guide
    https://docs.unity3d.com/Packages/com.unity.xr.compositionlayers@0.5/manual/provider-guide.html

    I want to implement WebXR Layers. For that I need a reference to the Texture or RenderTexture of the layer, to then use in WebGL calls when the WebXR API allows that.

    Just to make sure that I got it right:
    When a developer sets a CompositionLayer component with TexturesExtension component, I understand that I can get the texture reference from the TexturesExtension data.

    And it's the same when a developer sets Composition Layer for Interactive UI, the Projection Eye Rig will have a CompositionLayer component with TexturesExtension component that renders the XRI Rig.
    Then I can get the RenderTexture reference from its TexturesExtension data?

    There are no native graphics calls that Unity does for this? It's all on the provider to implement?
    If so, that's great :)
     
    Sift3D and DevDunk like this.
  7. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    482
    Installing another package is not the end of the world, but there definitely seems to be a trend within Unity in recent years, "let's just do whatever is easier for us". User ease of use doesn't seem to be a priority anymore.

    And while I'm complaining, it would be good if the documentation showed an example of the difference using Composition Layers would make for a UI to give devs an idea of whether it would be worth implementing.
     
  8. AlexandruFlorea-QC

    AlexandruFlorea-QC

    Joined:
    Mar 9, 2022
    Posts:
    1
    Thank you for providing early access to the feature for feedback! So far it looks really good. Are there plans to also expose the "Default Scene Layer" through this package? At the moment there doesn't seem to be a way to customize it and add extensions to it, such as the Color Scale and Bias extension.

    And the passthrough layer also doesn't seem to be exposed. Is that planned?
     
  9. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    Thanks for releasing this package! It's been a long time coming. I would also like to author a standalone implementation of XR_FB_passthrough, I've had a pure c# version stubbed for a long while that only required the layer submission to finish it off. Is this now possible with this package?
     
  10. triciabecker_unity

    triciabecker_unity

    Unity Technologies

    Joined:
    Jan 29, 2024
    Posts:
    3
    Good idea; showing goes a long way here and something we definitely want to do.

    The difference is super evident when you look at it side by side on device, but video capture quality makes showing the actual contrast a little more challenging. That said, we're actively working on getting captures and hoping to have more visuals to share soon.
     
    Shizola likes this.
  11. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    Yes, there is no native graphics calls, it's all on the provider.

    Let us know if you run into any issues trying to implement your WebXR provider, we will try to help unblock you.
     
    Sift3D likes this.
  12. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    This is a great feature request! We will investigate this and see if we can solve it in future releases. Thanks!
     
    AlexandruFlorea-QC likes this.
  13. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    Yes, it's possible to implement passthrough composition layer with this package. And we do have plans to give users access to a passthrough layer, FYI.
     
  14. triciabecker_unity

    triciabecker_unity

    Unity Technologies

    Joined:
    Jan 29, 2024
    Posts:
    3
    Only BiRP and URP for now since those are the recommended pipelines for XR.
     
  15. Tanya_Li

    Tanya_Li

    Unity Technologies

    Joined:
    Jun 29, 2020
    Posts:
    109
    We made composition layer package as a separate package by design as it allows other plugins (besides OpenXR) to implement their own layers support . See layer provider section in the doc for references: https://docs.unity3d.com/Packages/c...0.5/manual/provider-guide.html#layer-provider.
    Thanks.
     
  16. SamZ_XR

    SamZ_XR

    Unity Technologies

    Joined:
    Jan 31, 2023
    Posts:
    10
    Hey Mike,

    Could you provide some more details on why you're interested in authoring your own implementation of XR_FB_passthrough? Understanding your perspective would really help us prioritize this implementation effectively. Thanks!
     
  17. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    Hi Sam, sure thing.

    It's less of an issue with layers and more OpenXR in general. At the moment I feel that the point of OpenXR is failing, especially where Android is concerned; we have the standard, but each device vendor is shipping their own SDK and implementation of the same loaders/extensions that are incompatible with each other. For example, Pico's OpenXR implementation does support XR_FB_passthrough, but I can't use Meta's SDK to address that, because it's wrapped up in the OVRManager shared binaries that are specific to Quest, and vice-versa. Each vendor's SDK complains when the opposing vendor's device input profiles are present, even though they'd do no harm as they wouldn't be loaded. Ultimately, if this was a pure OpenXR system akin to something like StereoKit, then if a feature wasn't supported it would just be ignored. There's no reason we can't have a single build across OpenXR devices. This is before we get into the politics of the various Platform APIs!

    For context I'm one of the developers of Open Brush, and we currently have 5 separate Android builds for what should ideally be 1, 2 if you count China specific builds. https://github.com/icosa-foundation/open-brush/actions/runs/8160767392, all blocked by various artificial limitations of the vendors, even though the underlying spec should enable them to be a single build.

    Therefore, I'd really like to build our own suite of extension implementations that live outside the vendor SDKs, where we can use the cross platform loader and guarantee portability. I've already authored a version of XR_FB_display_refresh_rate (https://gist.github.com/mikeskydev/8e27668eade137a1085843868fbc6269) that works, and I'm wanting to see how far we can take it. This was my WIP of XR_FB_Passthrough: https://gist.github.com/mikeskydev/5f52bd25cd40f67a1b753dd6abd95ffe

    /rant! Thanks for listening :D
     
    Shizola and De-Panther like this.
  18. SamZ_XR

    SamZ_XR

    Unity Technologies

    Joined:
    Jan 31, 2023
    Posts:
    10
    Hey Mike,

    Thank you for providing detailed context and sharing your efforts in working around the limitations and inconsistencies between different vendor SDKs in OpenXR implementation.

    Have you tried Unity's OpenXR (Meta) packages instead of Meta's SDK? Our OpenXR (Meta) packages are supposedly compatible with all other OpenXR-compliant runtimes. But I totally understand your frustration, as this goes against the purpose of having a standard like OpenXR.

    Thank you once again for sharing your thoughts and experiences. We will seriously consider your request and the issues you have raised. If there's anything we can do to support your efforts, please let us know.
     
    virtualdario and Mike_Icosa like this.
  19. fastcoder

    fastcoder

    Joined:
    Sep 20, 2013
    Posts:
    16
    How can we capture a screenshot when using OpenXR layers? Unity's screenshot capture function only captures eye buffer but not other layers.
     
  20. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    You can enable emulation of the layers in standalone win, mac, linux build and then take screenshots with the system's screenshot tools: https://docs.unity3d.com/Packages/c...manual/project-settings.html#runtime-settings

    Or if that doesn't work for you, then you can also take screenshots in the runtime like the Quest, or via Sidequest for higher resolutions.
     
    Last edited: Mar 18, 2024
  21. lloydsummers

    lloydsummers

    Joined:
    May 17, 2013
    Posts:
    359
    So, just to be clear about the intended use here as it seems like this isn't the case in the docs.

    But, one of the main uses of compositor is to render "something" while transitioning scenes or doing heavy loads. However, this says disabling the object disables the composition layer, and I'm expecting the objects to be disabled on a scene load...

    https://communityforums.atmeta.com/...-in-Unity-like-when-using-SteamVR/td-p/611801
    https://forum.unity.com/threads/oculus-compositor-overlay-loading-screen.498430/
    https://developer.oculus.com/documentation/unity/unity-ovroverlay/

    Does this mean it won't actually work for changing the color on scene loading? In which case, it would make more sense to not use this, switching OpenXR for Oculus SDK, and using ovr-overlay?

    I'm just trying to understand the full use case here and maybe its because this is being called Composition Layer instead of Compositor, and being used to render stuff during the scene instead of when the headset is under a performance load and trying to stop screen tearing between scene loads.
     
    Sift3D likes this.
  22. lloydsummers

    lloydsummers

    Joined:
    May 17, 2013
    Posts:
    359
    Last edited: Apr 3, 2024
  23. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    I haven't tried it with this particular use case yet, but perhaps you could have your loading compositor layer on an additional scene that is additively loaded? whenever you need to switch main scene, you can do so in the background.
     
    Sift3D likes this.
  24. lloydsummers

    lloydsummers

    Joined:
    May 17, 2013
    Posts:
    359
    Thank you for the feedback - I did debate trying this to see if it would work. I ended up removing it for now anyway, as it was blocking Video Players from rendering to the scene, and I recognize it is in experimental phase.

    I'll definitely keep it in mind should we return to it down the road :)
     
    Sift3D likes this.
  25. goezdemir

    goezdemir

    Joined:
    Oct 8, 2021
    Posts:
    3
    Is there an ETA when this will get out of the experimental state?
     
  26. TheatreN

    TheatreN

    Joined:
    Jun 7, 2023
    Posts:
    1
    There is clearly a feature update planned for single pass rendering etc. An ETA for V1 feature completion would be very useful.
     
  27. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    Thanks for the response! I have looked into the OpenXR (Meta) packages, but it seems very heavily aimed at AR Foundation, and I notice it has a hard dependency on AR Foundation rather than it being optional. I'd rather use the standard XR setup and call Start/Stop as I feel like it. I can see how I might be able to alter this plugin to achieve what I was aiming for, however.
     
  28. EricRauschenbach

    EricRauschenbach

    Joined:
    Dec 21, 2023
    Posts:
    2
    I am trying to use Projection Eye Rig to pass several content layers to the OpenXR runtime. That seems to be working for a few seconds, but will then result in a crash of Unity Editor. Furthermore, depth submission seems to fail when using any layer on top of the default scene layer. Are these known issues? Any advice on how to improve?
     
  29. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    Update: Finally had time to look at this. I was able to get my previous implementation working! All that was missing was this new function to submit layers to XrEndFrame via
    OpenXRLayerUtility.AddActiveLayersToEndFrame
    provided by this new package. https://x.com/mikeskydev/status/1783896907615535207?s=46
     
    Last edited: Apr 27, 2024
    De-Panther and Du_Hoang_Unity like this.
  30. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    Hi Eric,
    Can you share more info about the crash?
    How did you set up your scene?
    What steps did you take leading up to the crash?
    Can you copy paste the error message?

    Thank you
     
  31. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    NOTE: THIS IS NOT A COMMITMENT, but we are working towards a verified release FYI. It's hard to say when it will land, but hopefully before the end of the year, if not sooner.
     
    De-Panther likes this.
  32. EricRauschenbach

    EricRauschenbach

    Joined:
    Dec 21, 2023
    Posts:
    2
    Filed Bug report, according to the answer NVIDIA driver causes crash. Stack Trace:
    Code (CSharp):
    1. 0x00007FFC23883DAD (nvwgf2umx) NVENCODEAPI_Thunk
    2. 0x00007FFC24B56DCF (nvwgf2umx) NVAPI_Thunk
    3. 0x00007FFC24B56D2F (nvwgf2umx) NVAPI_Thunk
    4. 0x00007FFC24B4BBFD (nvwgf2umx) NVAPI_Thunk
    5. 0x00007FFC24F9F24A (nvwgf2umx) NVDEV_Thunk
    6. 0x00007FFC254B236E (nvwgf2umx) NVDEV_Thunk
    Still, updating Win + GPU drivers did not help.
    If time allows, I might try it on another machine within the next days.

    Moreover, any clarification regarding depth submission in combination with Projection Eye Rig?
     
  33. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    Depth submission is currently not supported. But that feature is on our radar, and will maybe make it onto the roadmap in the future.
     
  34. Mike_Icosa

    Mike_Icosa

    Joined:
    Apr 2, 2021
    Posts:
    11
    I see OpenXR 1.11 has been released properly, but without the compositor layer support. Can this be restored in a 1.12 experimental? Otherwise its hard to build package dependencies on top of this awesome feature.
     
  35. Du_Hoang_Unity

    Du_Hoang_Unity

    Unity Technologies

    Joined:
    Oct 7, 2020
    Posts:
    12
    Hello Mike, this is NOT A COMMITMENT but we are looking to release another exp. package that will work with OpenXR 1.11 this summer. Thank you for your interest.
     
    Mike_Icosa and De-Panther like this.