Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Use of Materials / Shader for BackgroundImages in VisualElements

Discussion in 'UI Toolkit' started by mrSaig, Aug 9, 2020.

  1. mrSaig

    mrSaig

    Joined:
    Jan 14, 2010
    Posts:
    68
    Hey,

    i was wondering if there is support or if its planned to make it possible to have materials or somehow can use shader for the Background images in Visual Elements...

    i came up with a soultion but this one is kind of "Static"

    Code (CSharp):
    1. public UIDocument uiDoc;
    2.     public VisualElement imageHolder;
    3.  
    4.     public Material material;
    5.     public Texture2D src;
    6.     public Texture2D tex;
    7.     // Start is called before the first frame update
    8.     IEnumerator Start()
    9.     {
    10.         imageHolder = uiDoc.rootVisualElement.Q<VisualElement>("image");
    11.         tex = new Texture2D(512,512);
    12.         imageHolder.style.backgroundImage = new StyleBackground(tex);
    13.         float t = 0;
    14.         while (true)
    15.         {
    16.             t += Time.deltaTime;
    17.         material.SetFloat("_Grey", Mathf.Sin(t));
    18.         RenderTexture r = new RenderTexture(512, 512, 16);
    19.         Graphics.Blit(src, r, material);
    20.  
    21.         RenderTexture.active = r;
    22.         tex.ReadPixels(new Rect(0, 0, r.width, r.height), 0, 0);
    23.         tex.Apply();
    24.  
    25.             yield return new WaitForSeconds(1);
    26.         }
    27.     }
    because everytime (in this exampe every second) i update the texture the visual element flickers (it seems to try to resize?!) and if i put the blit in the update the visual element basically stays in flickermode.

    Any Ideas or Information when this will be possible?
     
    cz-ds and v-strelchenko like this.
  2. SimonDufour

    SimonDufour

    Unity Technologies

    Joined:
    Jun 30, 2020
    Posts:
    567
    Hi!

    I think the flashing you see is currently being worked on, but we have a workaround if it is the case.

    The flashing you see is because UI toolkit uses the active RenderTexture to do its layout. You will need to back it up at the beginning of your redraw cycle and restor it afterwards :
    Code (CSharp):
    1. var backup = RenderTexture.active
    2.  
    3. ...
    4.  
    5. RenderTexture.active = backup
    Let me know if this solves your problem.

    Then, I am going to take the liberty of doing a few other comments regarding your code:
    • You don't need to use a Texture2d. The ReadPixels is expensive as it needs to synchronize with the graphic card to read back to the CPU what is on the active render texture. If you don't need to manually edit or save the image, it can stay just on the GPU by assigning the RenderTexture to the style.
    • Graphic.blit set RenderTexture.active to the destination and leave it like this. So event if you remove the tex2d, you still have to do backup and restore workaround.
    • Keep in mind that RenderTexutre and Texture2d have a native resource associated. To have a clear lifecycle, I would add a private field containing the generated textures and destroy it with the object in OnDestroy for example.


    To finish answering your question, there is currently no way to specify a custom shader and for your use case I think you are better with the way you are handling things.

    Don't hesitate to ask for clarification or post back your code if you want more feedback!
     
    v-strelchenko, Nexer8 and mrSaig like this.
  3. mrSaig

    mrSaig

    Joined:
    Jan 14, 2010
    Posts:
    68
    oh thanks for the detailed Answer and explaining how it works behind the scenes! I'll try this later today, because this one actually is regrading my private side project ;D ... And thank you really for the code review ... i mean this was just a quick and dirty proof of concept type of code from my side and some of your comments i already had in mind but i actually didn't realize that i could assign the render texture to the style directly!
     
  4. mrSaig

    mrSaig

    Joined:
    Jan 14, 2010
    Posts:
    68
    Okay thx again ... with the backup and restore workaround the flicker is gone !
    But what i dont get is how i would get rid of the texture2D ? ... i cannot assign the render texture directly to the style ?!
    Code (CSharp):
    1. new StyleBackground(tex);
    StyleBackground only takes Tex2D or Vectorimage?! am i missing something here?
     
  5. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    771
    We're actually looking into adding RenderTexture support to the background style.

    Alternatively you can use an
    Image
    element which is designed to display Render Textures already (see
    Image.image
    ).
     
    mrSaig likes this.
  6. SimonDufour

    SimonDufour

    Unity Technologies

    Joined:
    Jun 30, 2020
    Posts:
    567
    I didn't realize that it was not exposed yet in the StyleBackground. Thanks Antoine!

    Code (CSharp):
    1. var item = new Image();
    2.  item.image = renderTexture;
     
    v-strelchenko and mrSaig like this.
  7. mrSaig

    mrSaig

    Joined:
    Jan 14, 2010
    Posts:
    68
    Thank you very much both of you!
    works like a charm!

    Didnt realize there is an Image element ... got to look more in the docs and not only in the uibuilder i guess ;D
     
    Kazko and SimonDufour like this.
  8. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    How did you manage to set it up? The StyleBackground does not take Image, only it's .vectorImage field, but the texture is assigned to the .image field. I'm kinda confused here.

    Would you mind sharing your solution? I can't seem to make this work. Thanks for any info!
     
  9. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    Nevermind, I figured it out. If anyone is as confused as me, the talk is about Image - an Element.
    Despite explicitly stated, because it is not available directly through builder, I thought they meant the Image Component.
     
  10. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    I'd like to bring the original topic up again. Has something changed in regards to "using custom shaders and/or materials in UIToolkit"?

    Examples where this is needed include
    • stencil settings
    • custom texture fetching (e.g. we're using multi-tap supersampling in UI for VR apps)
    • blend modes for individual elements/textures
    • combining multiple textures on a quad
    The "workaround" of rendering everything into RTs, then using them in plain images is neither performant nor practical on mobile devices.
     
    Midiphony-panda likes this.
  11. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    771
    Nothing has changed in this regard. Unlocking this kind of scenarios is important to us, and we will get there eventually, but it likely won't be through custom shaders or materials, at least not the way it's done like the UGUI system.
     
  12. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    @SimonDufour Hi, a while back I followed this thread to display a shader/material in the Image Element and it worked perfectly. After some time and the editor upgrade (currently using 2022.1 beta), this stopped working. No errors, but the shader does not display. Has anything significant changed that invalidates this approach? Thanks for any info!

    Code (CSharp):
    1. m_Texture2D = Texture2D.blackTexture;
    2. m_RankWidgetRenderTexture = new RenderTexture(200, 200, 24, RenderTextureFormat.ARGB32);
    3.  
    4. WidgetMaterialImage.image = m_RankWidgetRenderTexture;
    5.  
    6. m_RankWidgetMaterial = new Material(materialReference);
    7.  
    8. Graphics.Blit(m_Texture2D, m_RankWidgetRenderTexture, m_RankWidgetMaterial);
     
  13. SimonDufour

    SimonDufour

    Unity Technologies

    Joined:
    Jun 30, 2020
    Posts:
    567
    Nothing has changed regarding this. But I have seen some change related to the camera in HDRP.

    Can you verify that the result that you are expecting is written in the rendertexture?

    If you are able to provide us (by doing a bug report in the editor) with a project working in a previous version of unity that does not work in a newer version of the editor, we would investigate what is happening.
     
  14. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    @SimonDufour thanks for the reply.

    I am not very familiar with RenderTexture and Graphics.Blit, so I might be doing something wrong, however for sure it used to work (don't remember the exact Unity version). I tested the result texture (had to use style.background = Background.FromRenderTexture(...)) so that the created texture would show up in the UITK Debugger. But the texture looks like a solid blue square.

    So I tested in a new URP project (same editor version - 2022.1.0b3) - created just a simple version without UITK.

    Code (CSharp):
    1. public class Tester_RenderTextureWidget : MonoBehaviour
    2. {
    3.     public RenderTexture targetRenderTexture;
    4.     public Material materialReference;
    5.     [ReadOnly] public Material createdMaterial;
    6.  
    7.     [Button]
    8.     public void TestRenderTexture()
    9.     {
    10.         var _Texture2D = Texture2D.redTexture;
    11.         createdMaterial = new Material(materialReference);
    12.         Graphics.Blit(_Texture2D, targetRenderTexture, createdMaterial);
    13.         // Graphics.Blit(_Texture2D, targetRenderTexture);
    14.     }
    15. }
    The result is the same: the texture turns blue (viewed only through inspector of the RT asset). If using the commented line, the texture turns red, as expected. The shader and the material look OK.

    Any ideas or improvements to this test?

    Thanks!
     
  15. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    Here is the test project with the shader/material.
     

    Attached Files:

  16. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
    @SimonDufour Hi, just checking whether there was any development on your side concerning this issue. Has the attached test project helped you identify any leads? Do you have a suggestion how should I improve or change the test?
    Thanks for any info!
     
  17. SimonDufour

    SimonDufour

    Unity Technologies

    Joined:
    Jun 30, 2020
    Posts:
    567
    No progress yet, I had a few errors with odin + URP, so I left the tab open and figure I would come to it later.

    Usually the forum provide the quicker feedback, but we are more systematic with the bug reported in the editor. (Help menu->report a bug) We also have more feedback on the version you are using.

    Based on your comments, it seems like the issue is with URP as UI toolkit has no involvement with the raw call to the following lines:

    1. Code (CSharp):
      1. Graphics.Blit(_Texture2D, targetRenderTexture, createdMaterial);
      2.         // Graphics.Blit(_Texture2D, targetRenderTexture);
    You can check that the targetRenderTexture contain the expected result by selecting the asset in the project browser without any involvement of UI toolkit.

    I am thinking maybe the default Graphics.Blit use a material/shader incompatible with URP. But this is outside my area of expertise.

    If you can do a repro project containing only the minimum to reproduce (I think a source texture, a c# script and destination texture?) and report in the editor with clear reproduction steps, you should have an answer.
     
  18. ThunderThortle

    ThunderThortle

    Joined:
    Dec 11, 2020
    Posts:
    11
    Any news on using custom materials for the background image? Basically I have some icons that match the player colors in a local multi-player game, and I am using materials made with the AllIn1SpriteShader asset which lets me have 1 sprite and change one (or even more) of the colors of a sprite for another one of my choice (the player colors). And would be glad to use it in UIToolkit too. Currently it is critical for my project. Otherwise would need to go back to the old-school ui system. Repeating assets with different colors is off the table, and I can't just create a white asset because it has multiple colors on it
     
  19. DanteLore

    DanteLore

    Joined:
    Apr 2, 2021
    Posts:
    1
    Still struggling with this...
     
    fleity and FizzFab like this.