Feedback Unity 2021.1 Performance Overview

I think someone has to report the incremental GC dropping performance radically though. It's not what users would expect to happen.

 

First thing I've done is disable it. Makes me wonder if Unity should have it disabled by default?

Maybe it should have a disclaimer that it costs performance and really you should be eliminating garbage collection where possible?

 

Wait, it's on by default? That sounds bad.

 

Sorry I don't know for sure, shouldn't have phrased it like that. I don't remember turning it on but haven't tried making a new project yet to confirm so maybe I did turn it on.

 

If you create a new project (3D Template) in Unity 2021.1.0a9, then the

Incremental GC

is turned on by default. It's probably in older Unity versions turned on by default too.

I actually think it's not bad that it's turned on by default. Because then you have a chance to squeeze out a little more performance if you turn it off later in the development. That way it runs "smooth" by default and if you commit time to optimizing for GC, then you get some performance back in return.

 

I just checked, it's on by default.

If it has a performance overhead, I think it's a bad thing to have on by default.

(but then again Unity needs to find what their target audience is, because parts of the engine seem to target large studios and other parts newbie users that don't know what they're doing, so defaults being good or bad depend on what the target audience is)

 

It's definitely not what I expected. The documentation doesn't mention it's a trade-off between "fewer GC spikes for steady lower performance" too. I actually assumed Incremental GC would help performance to be honest.

However, I'm not sure if the drop in performance is even by design. It would be very helpful if a GC expert from Unity (@jonas-echterhoff ?) could tell if this is by design. If it's not, I can file a bug-report if its helps.

What baffles me the most is that my profiling project is quite GCAlloc optimized, there is 0 per frame gcalloc most of the time. However, turning off Incremental GC is still giving back up to 0.5ms per frame. So even though Incremental GC should have nothing to do, it still is doing something.

Again, could be totally by design.

 

I'll be really glad to be proven wrong, but I'm pretty sure the overhead is by design, which is why it's baffling that it's on by default.

 

I always ran/run performance tests with VSync turned off. Is this perhaps a reason why I see the Incremental GC cost?

Yes, thank you! The project attached to the bug-report was submitted with Unity 2017 a few years ago. If it helps, I can also submit it again with 2021.1 and the options I used for the latest test.

Otherwise here are the Player Settings that I used for the latest tests. Create a Win64 build with and without Incremental GC to see the difference. Also make sure to run it on similar hardware that I used to submit the bug-report (Case 1108597). You most likely can't reproduce it on higher-end hardware.

Spoiler

 

It's more than a few. The 20 samples you see in the image below represent 20 seconds, where each sample is the average frame-time over the last second.

If the game runs at 200 frames per seconds (200fps = (1/5ms)*1000), that would be 4000=200*20 samples in total.

Spoiler: Image

I repeated these tests three times and used the minimum average frame-time in the graph, this should remove a lot of noise.

Since the test goes over 20 seconds and the average frame-time gets reset each second, I don't think a single GC spike at the beginning would explain the entire test to be slower.

I can capture that.

Alright, will do that!

 

Here you go:
(Case 1300104) 2021.1: "Incremental GC" cost

Please make sure to watch the attached video(s) in the bug-report, where I explain how to build a player and how to run the test.

 

for what its worth, this thread and associated work having tracked down the performance issue to incremental GC is making me think I might make the step to migrate from 2018.4 to a later version at some point. Thanks again Peter!

 

This shouldn't stop you from upgrading. After all, you can turn it off when you notice it affects your game negatively and then it's the same behavior as in 2018.4.

 

GPU bound? The test runs (should run) at 320x240 resolution to avoid being GPU bound, I'm more than surprised. Did you run it on Windows or OSX?

I've uploaded Unity profiling data recorded with the Profiler.enableBinaryLog functionality, please see:
(Case 1300171) 2021.1: Follow-up on (Case 1300104) "Incremental GC" cost

Here are some screenshots so everybody can follow along...

Spoiler: No Incremental GC

Spoiler: With Incremental GC

Spoiler: Comparison

Please note that the Unity profiling data recording itself costs performance too, so these numbers don't necessarily match the graphs posted earlier.

PS: The profiler data captures seem to contain "CPU Usage" only, all memory alloc etc seem to be missing? I've asked whether this is by design here: https://forum.unity.com/threads/profiling-data-is-missing-a-lot-of-information.1024171/

 

QA was able to reproduce this issue. Thumbs up! Interesting that gcalloc doubled going from Unity 2019 to 2020.
https://issuetracker.unity3d.com/issues/ui-components-cause-gc-allocation-when-enabled

 

Isn't this an issue with all monobehaviour and not just UI? I think in the past setactive always had garbage and caused some fps spike too, was something improved already?

 

Not that I know of. My understanding is this:

• It's not exclusive to UI, but the UI Components were a trivial case to report for me. Not every UI Component causes gcalloc btw.
  
• It's not specific to the MonoBehaviour/Component base classes, but to the specific implementation of that MonoBehaviour/Component. Not every MonoBehaviour/Component causes gcalloc.

 

Nice, very useful to know this thanks

 

Hi,

I followed this threads about comparing versions for a longer time.

Spoiler

(I know Peters tests prevent GPU loads)
For my project I wrote a win based tool to make changes to all unity GPU core scripts at once. First, to make sure that there are no incompatibilities when updating GPU core scripts to a newer version. And for sure to have all changes in any Unity GPU core Script outside of unity itself, and pass changes into any version at any time with one click.

This thread reminds me to compare the GPU versions 2018.2.xx with 2020.x. based on all the shader & core stuff.

Further. I can definitely remember when comparing the first time Net4.x beta vs Net.3x all the scripting stuff was about 30%-70% faster on Net4.x in every situation... but after about 2 - 3 builds later the speed advantage was completely gone. That was a bit annoying, because the NET team shouted "it's so much faster" permanently.

 

After discovering the issue with Incremental GC. Is the mobile performance of 2019/2020 still considered poor compared to 2018? I want to upgrade my project to 2020 to utilize visual shader tools but only if I can get as good of performance on old devices as I do on 2018.

 

I just received a message from Unity QA, writing:

Kudos if this is really fixed now! Anyone up to give it a try and report back if really fixed?

 

Unity 2018 ran perfectly for me. Unity 2021 makes my game run like crap. Huge FPS loss to where I'm finding myself micro optimizing just to get that performance back. From terrible lighting performance to terrible particle system performance to annoying popup windows for every little thing I touch, click etc. Why do I have to perform lighting hacks of turning lights on and off to not blow my triangle counts out of proportion when it never used to happen? Why do I have to adjust my particles to be what I don't want, lower the resolution etc. to squeeze back performance I had in 2018? I could seriously write a huge list of issues I have with Unity 2021 vs using Unity 2018 which was excellent. I'm almost afraid to make any changes as I feel those changes will be deprecated in a few months and the cycle starts all over again.

Seems like backwards progression if I'm being honest.

 

If you're interested in an

old mono fork

vs

new mono fork

performance comparison, don't look any further, it's here:
https://forum.unity.com/threads/unity-future-net-development-status.1092205/page-4#post-7378859

 

I upgrade my project to 2020.3.18 from 2018.4.34. The uGUI setactive had big Jank. 2019.4 did not have this problom. I'm puzzled by this problem~~~

 

Very true, i see the same with 2018 being the exponentially faster version, 2019 is still passable and anything after that is like watch a slide show of popup windows.

Seems Unity 2020+ needs a monster PC, while 2018 run lightning fast on my 4 years old laptop.

The difference is so big that feels like another software altogether at this point.

 

I noticed the same today, I was on 2020 and upgraded to 2021 and the framerate was noticeably worse :/

 

so for best performance should i use 2018 or is any fixes?

 

Unity 5 or older could go strong too.