Search Unity

UI Roadmap

Discussion in 'UI Toolkit' started by benoitd_unity, Aug 27, 2019.

  1. DonLoquacious

    DonLoquacious

    Joined:
    Feb 24, 2013
    Posts:
    1,667
    Updating a ListView while people are scrolling through the same list, without losing their places, would be a really bad user experience IMHO, but still technically doable I think. If many players are finishing simultaneously (ie, playing in groups of 10 or something, so 10 entries are added/update at once when they finish), the 'update the user interface' function that you make should process them all at once so you just do 1 rebuild there instead of 10. A message that appears somewhere saying "new entries, press X to refresh" or something would feel far better from a user perspective I think, as well as being far better for resource usage, and simpler to implement.

    A game usually runs 60 frames a second minimum, all players aren't all going to be pushing updates to the list every second, so you've got around 99% "no activity" and 1% "updating ListView" frames on there, even if you went with your realtime updates idea (and you can make this even less frequent by checking a ConcurrentQueue every 10 seconds or so instead of really being "realtime")- that's the "assumption" of frequency being talked about here, and by Google in the talks that they've given, and basically everywhere else- and the reason why Unity devs should just stay the course on this product. In my experience, the assumption is almost always correct, and it's what makes systems like this possible, so it's a good one to make. ListViews have apparently already been the focus of a lot of the optimization here, being one of the elements that's updated the most often (but still very infrequent), so I don't think you have a lot to worry about there as long as your usage isn't abusive. If you have a situation in which that assumption is not correct, well, I won't go so far as to say "you're doing something wrong", but you're definitely not in the "common use-case" area anymore, so you'll probably have to deal with it on your own (as with all things in game dev with abnormal requirements).

    As for animating... well, drag-and-dropping UI elements is also not too difficult, even right now at runtime (adapting a system yourself from UI elements), and will be much easier once they convert over the drag-and-drop system already in the UnityEditor to work in runtime so you don't have that extra work making it yourself. Animating UI elements is also very easy for simple translation (moving from A to B, swapping elements in a list, etc). Check out the graph view for Shader Graph, or Asset Graph, or any of Unity's graph-based systems and you'll find that dragging things around is very smooth and efficient. That's not structural, as said above. If you need structural changes "animated", use IMGUI instead IMHO, as it renders every frame and makes no assumptions you can do whatever you like with it. You can even make an IMGUI container as a UIElement, so that it's still slotted into the same system.

    Add an IMGUI container, animate it, do what you need to, then remove it, add in a new traditional UIElement to the list as it finishes (once it won't change anymore), and you're done. If you need sparklies and other "effects", use shaders, not the UI system itself, as that's what they're there for (custom shader support is being added to it now).
     
    Last edited: May 4, 2020
  2. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    792
    Position-changes don't change the hierarchy. (USS/CSS position)
    For the ordering CSS Flexboxes has the "order" property, but I don't think USS supports it yet..

    In CSS/HTML for this scenario i would simulate the absolute position, No change to the hierarchy necessary. And add/remove items doesn't happen every few frames.

    But I would like to know whether the change in the order of Elements is considered a hyrachy change or not,
     
    Thaina likes this.
  3. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    838
    At the moment there exist layout groups, and moving them causes a rebuild. Or for example changing text contents also makes a layout rebuild. Perhaps its coincidence the naming is the same?
     
  4. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Scrolling list surely not change anything in hierarchy itself, but let me split into clearer scenario

    Scenario 1 : Scrolling list could change the scrolling panel and all of the child item will be translate efficiently by matrix multiplication from parent to children so it surely efficient and no need to concern at all

    But, Scenario 2 : resizing some item in the list is another story. It still not change hierarchy though, but it need to change the position of every item affected by the resized item, the layout could be changed. This is limitation since our current ScrollRect that it easily support list of the same height, but having a little bit to concern for list of item with varied height

    And Scenario 3 : changing order of the item in the list. Normally list view like a ScrollRect will render each item ordered by its index in the hierarchy. So if we need to reorder them, we also need to reorder the hierarchy itself

    Each scenario is not exactly the same as just scrolling (which is only the Scenario 1). These are my concern since our current UI system and it continually concerned over the new UIElement system
     
  5. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Yeah, these are exactly my point here
     
  6. mcoted3d

    mcoted3d

    Unity Technologies

    Joined:
    Feb 3, 2016
    Posts:
    1,003
    The UI Toolkit renderer provides several ways to optimize draw calls, even for dynamic elements. I'll try to summarize a few details here.

    When an element is moved around, the renderer will try to nudge the vertices instead of re-tessellating the entire mesh. This helps with performance, but we can do much better if the user provides us with hints:
    https://docs.unity3d.com/ScriptReference/UIElements.UsageHints.html

    Setting the DynamicTransform hint on an element will store its transform in GPU memory. Transforming these elements is very efficient since no geometry needs to be updated. Note that these optimized transforms are inherited by child elements, so children will be affected by their ancestor's dynamic transforms as well.

    Dynamic transforms are opt-in by the user through the VisualElement.usageHints property. Note that setting this hint on every element may defeat the purpose of the optimization (nested dynamic transform a bit more costly to process).

    There are other properties that are optimized for dynamic changes: tint color and opacities are both stored in GPU memory. Changing these values is very efficient and won't break draw calls. We are actively working to optimize even more use cases.

    Clipping is optimized as well. For axis-aligned clipping, the clip rectangle is stored in GPU memory. When a clipping shape is used (think rounded rectangles), we revert to stencil clipping with an optimization to avoid batch breaks. Both cases can be handled in a single draw-call.

    There are changes that will inevitably trigger a geometry regeneration (for example, changing the border-radius of an element). This is slow, but we have plans to optimize the tessellation process to minimize the impact.

    Hope this helps!
     
    Last edited: May 13, 2020
    Stardog, Thaina, Baste and 4 others like this.
  7. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    838
    Thanks a lot for the detailed answer! Sorry this thread got so derailed
     
    mcoted3d likes this.
  8. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    One thing that makes UI's hard to debug is that it can be hard to tell why it did a bunch of updates. We can see in the profiler that they are spending a bunch of time redrawing, but if it's not obvious why, we can end up having to binary search deactivate our UI, which is a pain. Will there be any good way to get good info about why a UT renderer has updated, and which elements prompted the update?

    When we make our own systems, we can just jam a breakpoint or log statement in where we need it, but for something built-in, we kinda need some better ways to get the info out.
     
    DonLoquacious and Noisecrime like this.
  9. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    838
    Something similar to this would be really great! Perhaps even some way to warn or inform users of the usage hints.
     
    mcoted3d likes this.
  10. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    This is truly agreeable. Highly appreciate. Even more so for the clarification
     
    mcoted3d likes this.
  11. mcoted3d

    mcoted3d

    Unity Technologies

    Joined:
    Feb 3, 2016
    Posts:
    1,003
    As of today, the only way to have an idea of why batch are breaking is by capturing a GPU frame with RenderDoc, and step through to see what's going on. This is less than ideal. We are planning to provide integrated analysis tools for UI Toolkit, this is important to maximize performances.
     
  12. ayellowpaper

    ayellowpaper

    Joined:
    Dec 8, 2013
    Posts:
    52
    Are there plans to add a system for validation? Like adding a validator to a TextField that requires E-Mail as input and possibly even disallows certain other inputs. I can imagine this for other things as well. For example add a char validator to a TextField so you can only input a single character. Or only allow input of characters for valid filenames. I think there are many use cases, from ingame UI (login form, create player name) to Editor Tooling.
     
  13. Antony-Blackett

    Antony-Blackett

    Joined:
    Feb 15, 2011
    Posts:
    1,778
    Is there planned workflow tools to help us develop nice UI animation and particles? These things have always been a pain to implement in Unity and it would be nice to see you guys plan a good workflow into your new system.
     
    vadersb_ru and kvfreedom like this.
  14. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Validation process contains too many possibility. I think we should just write custom event with onChange or onSubmit ourselves
     
    JoNax97 likes this.
  15. ayellowpaper

    ayellowpaper

    Joined:
    Dec 8, 2013
    Posts:
    52
    I'm not suggesting they implement all possible validations, but a system for us to easily extend and add to it while also containing a few, often used validators. But I get what you mean.
     
  16. Digika

    Digika

    Joined:
    Jan 7, 2018
    Posts:
    225
    Okay, if what I learned about UIToolkit being based on https://github.com/facebook/yoga/ is correct then we wont see grid support at least for decade:
    https://github.com/facebook/yoga/issues/867

    The request is almost 5 years old, haha. Man, Unity manages to come up with some good thing for once and then they screw it up as usual. Imagine having this vast, breathtaking freedom of not being dragged down by the need of backwards compatibility like browser-world and COMPLETELY throw this opportunity to support latest tech away .
     
    Thaina likes this.
  17. PassivePicasso

    PassivePicasso

    Joined:
    Sep 17, 2012
    Posts:
    100
    Is there any intention of ever providing the ability to extend USS with custom pseudo classes?
     
  18. uDamian

    uDamian

    Unity Technologies

    Joined:
    Dec 11, 2017
    Posts:
    1,231
    It's not a feature we'd like to implement anytime soon. We actually discourage use of pseudo classes and instead encourage use of regular USS classes.
     
  19. PassivePicasso

    PassivePicasso

    Joined:
    Sep 17, 2012
    Posts:
    100
    Could you explain why that is?
     
  20. uDamian

    uDamian

    Unity Technologies

    Joined:
    Dec 11, 2017
    Posts:
    1,231
    Mostly because regular classes can do what pseudo classes can. There is little difference between a selector like:
    .my-button:hover
    and
    .my-button--hover

    in terms of what it can do.

    Also, our pseudo states are implemented as an enum/flags which it's difficult to extend them, even more difficult to allow user-extensions. It would turn into a string-based system that would mostly duplicate the existing system of manage class lists.
     
    PassivePicasso likes this.
  21. RikiKlein

    RikiKlein

    Joined:
    Dec 14, 2018
    Posts:
    17
    Will it be possible to create node based Editor UI for example for a state machine?
     
  22. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    569
    dannyalgorithmic likes this.
  23. dannyalgorithmic

    dannyalgorithmic

    Joined:
    Jul 22, 2018
    Posts:
    100
  24. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    569
    Not sure what you mean? That project is just an extension of the node UI that Unity itself uses for Shader Graph and VFX Graph.
     
  25. unknowndevice

    unknowndevice

    Joined:
    Sep 13, 2016
    Posts:
    86
    Since this is an entirely new UI system is there any plans to add text support for fonts solution like pathfinder, slug, or other SDF alternatives? I realise it's probably not a high priority, but the change gives a good opportunity to add it. I think it would convince some people the switch over is worth it.

    Thanks!
     
  26. Thimo_

    Thimo_

    Joined:
    Aug 26, 2019
    Posts:
    59
    Unity 2020.1 is now out of beta and I was wondering if there is an indication for when UI toolkit gets its first version that is out of preview mode?
     
    Last edited: Jul 30, 2020
  27. JuliaP_Unity

    JuliaP_Unity

    Unity Technologies

    Joined:
    Mar 26, 2020
    Posts:
    700
    Thimo_ likes this.
  28. Thimo_

    Thimo_

    Joined:
    Aug 26, 2019
    Posts:
    59
    I want my UI to have an animation so I can hide it with a swipe like the Android burger menu. Is that already implemented or planned for the first verified release?

    I'm also wondering how lists work. Do they work like the Android recyclerview?
     
    Last edited: Jul 30, 2020
  29. uDamian

    uDamian

    Unity Technologies

    Joined:
    Dec 11, 2017
    Posts:
    1,231
    There's no animation support out of the box. But you can implement animations fairly easily with C#. Animations are planned for next year.

    UI Toolkit comes with ScrollView (a container that just adds scrollbars if the UI inside it gets bigger than the container) and ListView (which uses callbacks to only create the elements that can be seen and reuse them as you scroll, binding them to new data).
     
    Thimo_ likes this.
  30. ErikSutton

    ErikSutton

    Joined:
    Apr 25, 2019
    Posts:
    6
    @uDamian Any news or updates to when we will be able to make Treeviews with UI toolkit? Been waiting over a year for this feature.
     
  31. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Now in 2021. Did it already supported? So I could start using it
     
  32. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    306
    Thaina likes this.
  33. Splash_SA

    Splash_SA

    Joined:
    Jul 25, 2019
    Posts:
    1
    Hi this is a great plugin i am busy designing a learning game in South Africa for a client and had assumed that Unity has developed such already and only to find that you are also in testing phases so i support the roadmap UI and there are more other issues i would like to be in communication with of you guys