Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
New Forum User Notice Update to the Unity Editor Software Terms

Question Is it possible to render UI toolkit elements behind a UGUI canvas that is in screen space camera?

Discussion in 'UI Toolkit' started by Nilavazhagan, Aug 1, 2022.

  1. Nilavazhagan

    Nilavazhagan

    Joined:
    Jul 26, 2021
    Posts:
    7
    Hi,

    I am currently migrating my project's UI from UGUI to UI toolkit. We're planning for an incremental migration, so they need to co-exist until the entire UI is migrated. Currently the UGUI camera is rendered in Screen Space - Camera mode using a UI FX Camera. I converted some of the UI to the UI toolkit, but they seem to always be rendered in front of the existing UGUI canvas. And, I want them to be rendered behind the canvas. I tried playing with the Sorting order of the canvas, panel settings asset, UIDocument and none of it helped.

    This is my UGUI canvas settings
    upload_2022-8-1_16-12-30.png

    And, this is my PanelSettings asset
    upload_2022-8-1_16-13-7.png

    Thanks in advance.
     
    Nilavazhagan, Aug 1, 2022
    #1

  2. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    737
    Hello,

    As far as I know the Sort Order settings are compatible with UGUI only for the "Overlay" mode, to not "Camera".

    I believe a workaround might be to render UI Toolkit though a Render Texture first.
    Then render it on a quad in the Scene. The event ordering might be tricky to get right though. Do you need the UI Toolkit UI to receive events that aren't handled by UGUI first ?

    https://docs.unity3d.com/2022.2/Doc...e/UIElements.PanelSettings-targetTexture.html
     
    antoine-unity, Aug 1, 2022
    #2

  3. Nilavazhagan

    Nilavazhagan

    Joined:
    Jul 26, 2021
    Posts:
    7
    For most cases, the UGUI objects would be hidden when I need to interact with the HUD (which is what I'm rendering with the UI Toolkit now). And, when they're visible, they would be blocking the HUD anyway and there isn't a need to interact with the HUD. So, I think that won't be a problem.

    And, I'm not sure what you mean by a quad? I've only used the Render Texture on a RawImage component, and I'm guessing the RawImage component cannot receive any events. Is there a way to render the UI through a Render Texture and also make it receive all the events?
     
    Nilavazhagan, Aug 2, 2022
    #3

  4. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    737
    Sorry, I should have said render it on a "plane", like the primitive from Unity.

    There are 2 main issues with asking UI Toolkit to render to a texture, that you need to somehow handle yourself:
    - Event processing order
    - Converting the screen space positions

    Event processing order

    It all depends on what you need to achieve for this. If UGUI and UI Toolkit never overlap on screen or if they don't render at the same time, you're fine.

    Otherwise, my conclusion, after spending quite a bit of time looking at the use case of "UI Toolkit behind UGUI when using Screen Space - Camera", is that there is no way to make it work properly for the general use case.
    I think you'd have to find ways to disable the event processing in UI Toolkit temporarily when UGUI is on top (for example by using SetEnabled(false) or adding an overlay with pickingMode=Ignore on top of everything).

    Converting the screen space positions

    This is necessary to turn points in screen space into a local space for the UI Toolkit panel, to process events.
    For example, if the RenderTexture size doesn't match the size of the UI, or if the RenderTexture is applied to some mesh in the scene.

    This is achieved by implementing a function like 
    Vector2 ScreenCoordinatesToRenderTexture(Vector2 screenPosition)
    and assign it with 
    panelSettings.SetScreenToPanelSpaceFunction(ScreenCoordinatesToRenderTexture);
    .

    Here is an example of an implementation of this function, which assumes you have a some MeshRender setup to display the render texture:

    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3. using UnityEngine.UIElements;
    4.  
    5. public class PanelOnPlaneHelper : MonoBehaviour
    6. {
    7.     public PanelSettings panelSettings;
    8.     public Camera targetCamera;
    9.  
    10.     void OnEnable()
    11.     {
    12.         if (panelSettings != null)
    13.         {
    14.             panelSettings.SetScreenToPanelSpaceFunction(ScreenCoordinatesToRenderTexture);
    15.         }
    16.     }
    17.  
    18.     void OnDisable()
    19.     {
    20.         if (panelSettings != null)
    21.         {
    22.             panelSettings.SetScreenToPanelSpaceFunction(null);
    23.         }
    24.     }
    25.  
    26.     private Vector2 ScreenCoordinatesToRenderTexture(Vector2 screenPosition)
    27.     {
    28.         var invalidPosition = new Vector2(float.NaN, float.NaN);
    29.  
    30.         screenPosition.y = Screen.height - screenPosition.y;
    31.         var cameraRay = targetCamera.ScreenPointToRay(screenPosition);
    32.  
    33.         RaycastHit hit;
    34.         if (!Physics.Raycast(cameraRay, out hit))
    35.         {
    36.             return invalidPosition;
    37.         }
    38.  
    39.         var targetTexture = panelSettings.targetTexture;
    40.         MeshRenderer rend = hit.transform.GetComponent<MeshRenderer>();
    41.  
    42.         if (rend == null || rend.sharedMaterial.mainTexture != targetTexture)
    43.         {
    44.             return invalidPosition;
    45.         }
    46.  
    47.         Vector2 pixelUV = hit.textureCoord;
    48.  
    49.         //since y screen coordinates are usually inverted, we need to flip them
    50.         pixelUV.y = 1 - pixelUV.y;
    51.  
    52.         pixelUV.x *= targetTexture.width;
    53.         pixelUV.y *= targetTexture.height;
    54.  
    55.         return pixelUV;
    56.     }
    57. }
    58.  
    Hope this helps.

    I am working on a more general sample to explain this with assets, a Scene, etc.
     
    antoine-unity, Aug 2, 2022
    #4
    ProFiLeR4100, mikejm_, KamilCSPS and 2 others like this.

  5. Nilavazhagan

    Nilavazhagan

    Joined:
    Jul 26, 2021
    Posts:
    7
    Hi, thanks for this. But, I felt this was too complex for a temporary workaround. So, I ended up switching the UI toolkit UI to a kinda disabled mode by rendering them on a RawImage whenever UGUI screens need to be overlayed over the UI toolkit panel. But, unfortunately it has it's own set of problems.

    So, I decided to switch the UGUI canvas to ScreenSpace - Overlay mode and migrate the problematic areas first. The sorting order works as expected, but the UGUI objects don't receive any events at all. This is the setup I have so far

    UGUI (Sort order: 0)
    UI Toolkit Panel (Sort order: 1)
    UGUI (Sort order: 2)
    Secondary UI Toolkit Panel (Sort order: 3)
    UGUI (Sort order: 4)

    Even when a UGUI panel with sort order 4 is visible (covering the entire screen), it does not receive any events and the UI toolkit panel beneath them receives the events. Is there a way to get the events to work properly?

    If not, what's the preferred way to do the UI migration? Is there no way for UGUI and UI Toolkit to co-exist?
     
    Nilavazhagan, Aug 11, 2022
    #5

  6. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    737
    This should work.

    If you can confirm your event set up matches what is described in this page under "UI Toolkit elements and uGUI components with ..." then it looks like a bug.

    Possibly UI Toolkit panels are blocking events for UGUI panels below them because they have a fully transparent root taking the whole screen. This should be set as pickingMode=Ignore.

    But that wouldn't explain why the UGUI Canvas on top wouldn't get events.
     
    antoine-unity, Aug 11, 2022
    #6

  7. Nilavazhagan

    Nilavazhagan

    Joined:
    Jul 26, 2021
    Posts:
    7
    Hey, turns out my child canvases were missing a GraphicRaycaster component. I didn't know each individual canvas required a separate GraphicRaycaster component. It's working fine after I added it. Thanks for the help :)
     
    Nilavazhagan, Aug 12, 2022
    #7

  8. Nilavazhagan

    Nilavazhagan

    Joined:
    Jul 26, 2021
    Posts:
    7
    Hi, I've run into another problem unfortunately. The system works completely fine in the Editor. But, the UI Toolkit elements do not receive any events at all in the standalone build. I noticed that the PanelSettings game object with the PanelEventHandler and the PanelRaycaster components is not created at start in the standalone build. It is only getting created if I disable and re-enable the UIDocument after startup. How can I make the PanelSettings game object to always be created in the startup? Or, is it possible to manually create them and just keep them in the scene?
     
    Nilavazhagan, Aug 18, 2022
    #8

  9. Giantbean

    Giantbean

    Joined:
    Dec 13, 2012
    Posts:
    142
    If each world space UI toolkit needs its own Render texture and GraphicRaycaster component isnt that going to cause performance issues fairly fast? Would it be beter to use UI tool kit for static overlay HUD while using UGUI in world space behind the HUD? Is the performance the same ewither way? It seems like world space and z-index need to be worked out befor the new UI toolkit can realy replace UGUI.
     
    Giantbean, May 8, 2023
    #9

  10. dugsteen

    dugsteen

    Joined:
    Dec 9, 2015
    Posts:
    8
    Hi, was this ever done? I'm looking for something similar: I'd like to create sprite animations on a canvas in front of a fullscreen UIDocument. The animations don't need to be interactable, so I'm not worried about the canvas receiving events. My knowledge of render textures and the like is quite limited, so a full-blown example would be fantastic. Thanks
     
    dugsteen, Jun 13, 2023
    #10

  11. jodiw

    jodiw

    Joined:
    Feb 25, 2023
    Posts:
    2
    I came across this thread when having problems getting a Canvas and UIDocument on a component to work properly. Thanks to this post, I discovered that the sort order on the UIDocument component is not acting on the overlay sort order, maybe just the sort order of UIDocuments within the same component. The setting that will interact with the canvas is in sort order in Panel Settings. The Unity documentation indicates that sort order is a factor, but doesn't make it clear where the setting is.
     
    jodiw, Oct 16, 2023
    #11