Bug UI Toolkit accumulating extra millions of extra vertices to render when objects made Hidden/Visible

I am attempting to improve my UI Toolkit app performance and found what I believe is a bug that is interfering with my attempts.

Setting objects to VisualElement.style.visibility = Visibility.Hidden can save a massive amount of processing when those objects do not need to be seen. Switching them back to StyleKeyword.Null costs a bit of processing for a brief moment, but it's not too bad or laggy if you're not doing too many at a time. This then allows optimization of things like custom scroll views or app UI panels where you might want to hide a panel or elements that are out of view.

However, there is a bug in doing so. When you go back and forth between these two states, Unity is accumulating vertexes and draw calls above what it should be.

This is easy to demonstrate. I created a demo app with many visual elements on screen. Every time I click the mouse, a certain number are toggled between the Visibility.Hidden and StyleKeyword.Null state.

State 1: INITIAL LOAD ("X" small number of elements visible)

Upon loading, most of the elements are hidden, and you can see it has 8.2K vertexes and only 3 draw calls with used buffer of 2.4 MB. Very quick and easy.

State 2: MAX ELEMENTS SHOWN ("Y" large number of elements visible)

If I click to toggle the state, I now have a very large number of all elements set to visible (StyleKeyword.Null) and it goes to 480K vertexes with 53 draw calls. It's a torture test so intentionally extreme.

State 3: CLICK AGAIN TO GO BACK TO STATE 1 ("X" small number of elements again visible)

We are now back to the exact same visibility/null state as in State 1. The exact same number of elements are set to visible and null. However, we are now up to 59K vertexes with used buffer at 5.9 MB. In state 1, the exact same scenario only cost us 8.2K vertexes with a 2.4 MB buffer. Thus some inefficiency has piled up here and is not going back down.

This is an extra 51K vertexes that are being drawn compared to State 1 while still having the same number of visible elements and triangles.

State 4: CLICK AGAIN TO GO BACK TO STATE 2 ("Y" large number of elements again visible)

We are now back to the exact same state as State 2 where all elements' visibility are set to StyleKeyword.Null. However, we now have 714K vertexes which is the highest so far.

This is an extra 230K vertexes over State 2 while still having the same number of visible elements & triangles.

Further toggling will not worsen the condition. It will remain between the State 3 and State 4 vertex conditions after that. But I can find no way to restore it to the much better State 1/2 vertex conditions.

I suspect this issue also explains an annoying and impossible to fix problem where I notice if I toggle around in my app between different screens (panels with visibility on/off in this manner), the performance is great at first but then drops and gets laggy.

On Android, when this happens, sometimes if I "tab" out of the app to see the screen where I can select another app, then go straight back into the Unity app, the performance is improved. I haven't tested that yet, but I suspect it might be clearing these extra vertexes/buffers by doing so.

Either way, this is something that would be great to have fixed as we cannot optimize our apps by setting objects to Hidden if we are still going to pay a vertex/buffer fee for their existence once we toggle this state. We can currently never get back to a fully efficient vertex/buffer state.

Is there some way you can make it cull all the extra vertexes and buffer space so this doesn't happen and rendering states are predictable and consistent? Or give us a command to be able to do this?

I submitted my bug project with case #1422986.

Thanks for your help.

 

Thanks. Just to clarify two things:

1) I already tested when I was doing all this as I was trying to figure out what optimizations worked, and changing opacity will not help with optimization. Objects with zero opacity do not reduce the vertex/triangle count or buffer size. So setting opacity to zero will make something invisible, but not reduce the rendering burden. Only setting them to Hidden cuts the vertex/triangle count or buffer size and is an optimization. (Unless I am missing something?)

2) It seems evident from what you describe, those extra buffers are NOT being pruned. To clarify again, in my demo case, State 1 and State 3 have exactly the same objects on screen and thus the exact same number of triangles. Yet state 3 has over 5 times as many vertices and over twice the buffer utilization.

That number of vertices and buffer utilization NEVER goes back down to State 1 levels, even though it is showing the same number of objects as state 1 (and the exact same objects even more specifically). It is not at those high levels for a few frames. It is at those high levels permanently until you restart the app all over again.

The situation "stabilizes" in a State 3/4 conformation in that the number of vertices and buffer utilization doesn't go higher than in those states. But as you can see those states are dramatically more expensive than they should be and there is massive waste going on compared to the otherwise identical States 1/2 which I can't reverse.

So I presume this is a failure of the buffer pruning from what you describe.

Thanks for looking at it and I appreciate any efficiencies and fixes you can provide. I am developing for mobile and have thousands of UI elements. So having 100s of thousands of extra vertices and double to triple the buffer size not clearing may add up. Every bit of performance is needed.

Thanks Simon for your thoughts as well. I appreciate you checking it and of course if 2022.2 improves efficiency that will be welcome when I have access to it.

 

Just to add a bit more data and perspective, I did some testing with my real world app, and the results are pretty dramatic. I think this almost certainly explains the main and mysterious performance issues I have been suffering with from UI Toolkit on an ongoing basis.

CASE 1 - SHOWING/HIDING "SCREENS":
In my UI app, I have multiple "screens" you can click between which just hide or show certain high level portions of the hierarchy. This is a basic necessary function.

Here is one identical screen before and after clicking around the app.

Both before and after clicking around, we can see there are 4K triangles and 43 textures indicating these are IDENTICAL HIERARCHY/VISIBILITY STATES. Yet we have gone from a normal level of 12.2K vertices and 12 draw calls to an insanely high 970K vertices and 115 draw calls.

This is cripplingly high for showing an identical screen and hierarchy state. These numbers/screencaps are all steady state and not momentary spikes.

Before Clicking Around:

After Clicking Around:


CASE 2 - SHOWING/HIDING OBJECTS INSIDE/OUTSIDE SCROLLVIEW:
I would also like to show/hide objects that are inside/outside of a custom made scroll window. In principle this cuts the number of objects to render at any given time. If you have a massive scroll window (like a long chat) this is necessary. On load, you can see a major saving by doing this. However, once you start scrolling around, because of this vertex accumulation issue, everything deteriorates.

Here is a real world scroll window where I am hiding any elements outside of view and showing them once in view. In both cases, the screencap is of the exact same resting position and hierarchy state, demonstrated by again identical triangle/texture counts.

Scrolling up and down a few times increases the draw calls and vertices dramatically. We go from a normal level of 11 draw calls and 8.7K vertices to 40 draw calls and 42K vertices to show the exact same thing after scrolling with the show/hide function.

Before scrolling:

After scrolling:


I have been working on my app for a long while now in UI Toolkit and couldn't figure out why the performance was so variable and why it would have trouble with simple tasks and get laggy sometimes for seemingly no reason. I think such massive accumulation of vertices and draw calls may certainly explain the problem.

Frankly, I am glad to have found this as I would otherwise have no good explanation for why I have been experiencing so many performance issues. It has been a constant stress at the back of my mind not knowing why it is happening. I need a smoothly working app that can function across a variety of devices.

I hope and presume the problem should be findable and fixable since it is so dramatic and easy to reproduce. Thanks again for your help.

 

Hey guys, I am sorry to be a bother but this issue is becoming quite bad as I continue to develop my app. I am over a year into development so don't have much choice at this stage except to continue. But without a fix for this problem efficiency is so poor Unity struggles to do even the simplest tasks on the most advanced mobile devices on the market.

I am seeing steady state situations like this now during actual usage:


This is just absolutely insane. Of course no mobile device can smoothly sustain rendering 228 draw calls and 1.5 million vertices every frame, and there is no justification when there are only 7.5k triangles.

Have you been able to figure anything out about why this is happening or when a solution might be possible? I can keep working with everything responding choppily and in slow motion behind the scenes, but I will otherwise be dead in the water and certainly any chance of releasing anything is dead until it is fixed.

Thanks again.

 

Hey, I already posted a demo project that illustrates the problem in the bug report cited in the OP. You can see in the demo project I attached to the bug report how just simply showing and hiding elements by clicking dramatically increases the vertices and draw calls. That was an intentionally simple project to show the issue as simply as possible.

I am not sure how I could demonstrate how bad the scale of the problem becomes when it is scaled up to a real world project. But I am now seeing my absolute worse and most crushing results. It gets worse the more my project progresses as the more I am dependent on showing and hiding objects.

For example, here is my real project on load:


Nothing too dramatic. 8.3K triangles leads to 22 draw calls, 19 batches, 8.3k triangles, and 41.6k vertexes. Usually it would only be 2-4 draw calls for this many vertexes in my experience with UI Toolkit so I presume the "show/hide" issue is already affecting this as I am setting "show/hide" parameters on start up, but it is still not too bad.

With just one click to another panel in my UI Toolkit app, setting a parent element of another panel visible and the other unneeded ones to hidden, I get this absolute disaster which is the worse I've ever seen:


There are not that many elements still visible. There are only 14.2k triangles which is not even double the starting state. But insanely I now have 7.2 MILLION VERTICES, 312 BATCHES, & 432 DRAW CALLS. This is completely crushing the performance.

I see my bug report was reported as accepted and that your team has reproduced the issue:

My understanding of bug reports is the minimal project to reproduce the issue is always best. I provided a simple project with only ~200 lines of code as I recall that reproduces it in that report and with the massive draw call and vertex accumulation described and screencapped in the OP.

The only update I can add is I have now tested both style.display and style.visibility and they both exhibit the problem. I edited my test project to add a simple toggle so you can see both of these easily.

Here is my Bug Report project on load:


Here it is after toggling style.visibility (while showing the exact same thing - triangles are same but now 10-20x as many vertices, draw calls, and batches):



Here it is after toggling style.display (again showing same thing with same problem, just slightly less severe):


So I can say the problem exists with both methods of showing and hiding VisualElement. I was not sure how to upload an addendum to my bug report so I submitted the new project which just has a simple toggle in it to change which hide method is used here: IN-7309.

Thanks Simon but there is no platform specific issue. It happens in Unity Profiler on Windows and it happens on Android phone. Once the vertexes, draw calls, and batches (it happens to all together) get insane like this the performance of simple tasks like scrolling becomes unmanageable. There is major lag and fps drop as expected. I have had some rare crashes on mobile when it gets out of control now. I haven't been able to catch one of these crashes on debugging as they are rare but I presume it is just resources running out as mobile devices struggle severely to keep up with all this at all.

I presume it should be expected that a problem like this will scale up or down with app complexity. It just unfortunately seems to be getting worse and worse and way out of control the farther I go as a result.

I hope you will be able to look at my bug project demonstrating it and a solution will be possible.

In the mean time, I will keep working on my project as I am stuck with no other choice at this stage. Thanks again to you both for all your help and an otherwise fantastic UI Toolkit and development environment.

 

Thanks a bunch. I'm glad you were able to replicate it and see where it was coming from. I have actually built some custom listview type classes of my own to reduce my repetitive objects (eg. in scrolling views) but there is only so much I can reduce as it is a very complex UI/text based app that replicates many social media functions and thus requires lots of labels, elements, scroll views, panels, etc.

Alternatively, recycling objects by changing the hierarchy continuously is also costly

I am not quite ready for launch in any case at this stage. So I will just keep going while waiting for the fix.

My two main barriers with Unity itself for project completion are: (1) This vertex/batch/draw call issue, and (2) Needing working TextField for mobile, which I was told is in the works for 2022.2.

Currently TextField on mobile does not allow editing in place, midpoint editing, scrolling, text wrapping inside the textfield, or proper opening or closing of the keyboard on normal cues that would do this (manually or automatically). I described the TextField problems as best I could see them with screencaps here:

https://forum.unity.com/threads/how...ked-on-ie-to-make-a-custom-textfield.1262723/


While I have your attention, if I can bother you one moment further, I am wondering if you know if solutions are still impending for that as well? ie. Working mobile Textfield editing in place, scrolling, text wrapping inside textfield, midpoint editing, and manual commands to open and close the keyboard/textfield that override the (faulty) automatic behaviors?

I can keep working in the mean time but it is always nice to know there is a plan to fix these things. I have a constant mild background fear that I will eventually finish my app/network after years of work and then find out I'm still dead in the water indefinitely with no plan due to these limitations.

Thanks again. Honestly I really like UI Toolkit and I appreciate all the work you guys have done to put it together. I am grateful for it all.

 

Thanks mcoted3d! But at this stage, don't sweat it on my accord! I actually spent the last 3 days figuring out how to build my own custom UI Toolkit TextField by deriving from basic classes. It was painstaking and brutal work but I was successful, so I now have everything working in that regard (midpoint editing, caret, etc).

The only remaining bug I am experiencing with the TextField is there is a major problem pertaining to the TouchScreenKeyboard.hideInput function in Android which crashes the Keyboard any time you click in the area of the hidden input field:

https://forum.unity.com/threads/tou...s-functions-in-current-android-unity.1303749/

I thought this might have been a problem with the UI Toolkit TextField initially, but I now see this happens whether I use UI Toolkit's default TextField or my own custom one (or just manually open the TouchScreenKeyboard any way) so it seems to be a more general problem.

I submitted a bug report for that as linked in that thread so hopefully a fix is possible or some suggestions could be provided for the Android code to manipulate so I can try to fix it myself.

Might you be able to ask one of your Android team members to look at that if you know anyone who would know or provide any direction on what types of Android codes might go in the right direction to fix it?

Thanks again for all your help. It's really been critical to keeping me moving.

The only three significant issues I personally have now are:

1) Massive vertex accumulation with showing/hiding VisualElement: this is probably the biggest problem as described in this thread, as there is nothing further I can do to fix it on my end - I have already cut my VisualElement count as low as I can and I am otherwise at your mercy.

2) TouchScreenKeyboard.hideInput is crashing keyboard in Android: as noted in the linked thread above - this is also a huge issue but I am optimistic it might also be possible for me to solve on my own with a bit of direction for relevant Android codes.

3) No built in UI Toolkit Emoji support currently: and no effective way I have found to workaround, though possibly I can build a sprite sheet workaround with a lot of labor as discussed here: https://forum.unity.com/threads/emojis-support.1233478/

I will likely be done my app in the next few months which will be around 2 years work total. It is heavily based on UI Toolkit and I am very grateful for the system. I couldn't have built it without it, but I will also need at least #1 and #2 working somehow or I won't be able to launch.

(ie. I can live without Emojis or do a painstaking workaround but can't function on average mobile devices with millions of vertices accumulating or constant crashing keyboards.)

I very, very sincerely appreciate all your responses and any help you and your team can provide. Thanks for the communication and being so active in the forum and with everything that's going on.

 

Please make sure input fields on mobile also support auto fill/suggest feature like filling the login form with saved email and password. That'd be simply amazing

 

UI Toolkit is awful compared to UGUI.

For me about 20 progress bars adds about 900K verts which reduces the performance significantly.
How would one create enemy health bars if it reduces the performance down by around 50% ?