2020.2.0b1 Performance Reduction

But we can start to try to isolate what causes this and why reproducing it might've failed (post excluding stats view FPS changes, yep the change was confusing internally too)

@DrabanL What kind of GPU and CPU are you using?
Do either of you (i.e. also @Creepgin ) have any overclocking, vSync affecting or otherwise graphics driver/ gaming overlay/ hardware tool running?

Also I'm assuming based on what I can see that this is on windows 10. Are you both using URP sample projects created with the Unity Versions you are testing on or are any upgraded through the versions? Are they using the same Graphics API and general settings?

And which URP Versions are you using in each Unity version and does the regression change if you update it to the highest available version in each?

 

@samsv Performance drops not identifyed by the fps counter in the stats view but by a Time.deltaTime based fps counter/fraps or better still he profiler? Is this for upgraded projects or newly created default URP projects too? Which URP versions?

 

sorry saw the note on built-in RP too late, please start a new thread and/or bug report on this if you've double checked that it isn't just the stats view FPS calculation change we discussed in this thread. This issue here has been plagued by ambiguity and confusions enough to stall it so far and I'd like to keep these separated until proven that they share the same root cause. And the profile Analyzer data shown here so far hint at an issue in URP and rendering performance.

 

Actually, I guess the Empty scene isn't really using URP but built-in?

Anyways, between post holiday catch-ups I've done some initial cursory attempts at reproducing this and so far failed with my uGUI, Recorder and deltaTime based ms and FPS display to see anything but a very mild performance improvement from 20.1 to 20.2, unless I toggle on the stats view which has more elements to render in 20.2 and adds an about 0.3ms regression. Have you tried this with just the Fraps/own fps display and without the stats view open

 

Alright. I've gone over this a couple of times now. I can't reproduce a performance regression in Empty Scenes. I can reproduce on in the Sample Scene.

Note: We have somewhat different machines. And I think this is down to newer versions taking more CPU time for rendering but way less GPU time, which could explain why its so tricky to reproduce the regression, because the overall frame time doesn't change much for me, but depending on what hardware you have, you might be better or worse off with this shift.

For reference:
My machine
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz 2.40 GHz (up to 5GHz)
GeForce RTX 2080 with Max-Q Design

I am using my performance testing Windows Custom Power Plan with 99% settings on the Processor Power Management to avoid CPU scaling to kick in (effectively locking my CPU in at 2.28GHz)


I've used the attached scripts and canvas to help in the process. For the measurements I've disabled the canvas and just used the 5s measurements triggered with Shift+F10 (to average FPS up over 5s) and Shift+F12 (to generate Editor Profiler data for 5s without the Profiler Window open) to keep things as comparable as possible. I've checked debug and release editors. I've made sure that GI was done before measuring and the fps stabilized. There is a difference between creating a new Empty Scene, and taking the existing Sample Scene and deleting out everything but the camera and light, at the very least the skybox is different, so I checked that emptied URP scene against a newly created one, though there is hardly a difference in the highlevel metrics (collected with the Shift+F10 so not profiling data, just accumulated Time.deltaTime):

2020.1

Empty Scene
Debug Frames: 627 - Average Frame Time 7.98ms - Average FPS 125.3
Release Frames: 631 - Average Frame Time 7.93ms - Average FPS 126.1

Empty Scene URP
Debug Frames: 627 - Average Frame Time 7.98ms - Average FPS 125.3
Release Frames: 642 - Average Frame Time 7.79ms - Average FPS 128.3

Sample Scene URP
Debug Frames: 600 - Average Frame Time 8.35ms - Average FPS 119.8
Release Frames: 620 - Average Frame Time 8.07ms - Average FPS 123.9
(alternate Release run) Frames: 627 - Average Frame Time 7.99ms - Average FPS 125.2

2020.2

Empty Scene
Debug Frames: 633 - Average Frame Time 7.90ms - Average FPS 126.5
Release Frames: 644 - Average Frame Time 7.76ms - Average FPS 128.8

Empty Scene URP
Debug Frames: 631 - Average Frame Time 7.93ms - Average FPS 126.1
Release Frames: 643 - Average Frame Time 7.78ms - Average FPS 128.6

Sample Scene URP
Debug Frames: 599 - Average Frame Time 8.35ms - Average FPS 119.7
Release Frames: 610 - Average Frame Time 8.21ms - Average FPS 121.8
(alternate Release run) Frames: 603 - Average Frame Time 8.30ms - Average FPS 120.4

2021.1

Empty Scene
Debug Frames: 628 - Average Frame Time 7.97ms - Average FPS 125.5
Release Frames: 646 - Average Frame Time 7.75ms - Average FPS 129.0

Empty Scene URP
Debug Frames: 619 - Average Frame Time 8.09ms - Average FPS 123.6
Release Frames: 648 - Average Frame Time 7.72ms - Average FPS 129.6

Sample Scene URP
Debug Frames: 597 - Average Frame Time 8.39ms - Average FPS 119.2
Release Frames: 601 - Average Frame Time 8.32ms - Average FPS 120.1

So... Empty scenes look pretty much unchanged, Sample Scenes 0.3ms slower since 2020.2 but interestingly ONLY in Release! So lets look at the Profiler Data:

Check out the right panel in particular.
The frame time distribution in 2020.2 is less wide, i.e. the extrema are brought in, the peaks are lower, the entire upper quartile actually progressed, and part of this could very well be down to changes to Time.deltaTime stabilization. But the Median has indeed gone up by these 0.3ms.

Since some marker/function names changed, a direct apples for apples comparison in PA is a bit tricky, more so because this is down to a shift in the frame rendering pipeline and as we'll see in the next screenshot, time spend waiting on the Render Thread/GPU has gone down but would still be added to the main thread frame time here and there is no really good way to calculate that out of the overall frame time (checking only specific rendering sample times works but isn't necessarily a direct map to the frame time). I'm pondering some ways to make this more apparent, happy for suggestions.
2020.1

2021.1


So the rendering times have doubled (3ms -> 6ms) but the time spend waiting on the GPU has fallen drastically too (5.5ms -> 1ms). So not having any insight into what changed, I could assume the rendering is now preparing stuff better for GPU consumption, or commands to the GPU are spread out over a longer period so the GPU has more time to process them? Bare in mind this is a sample scene and depending on the load and content, this could scale better or worse?

Anyways, I've reopened the report with this info and let the URP team decide that. Just posting here to keep you updated and provide a bit of insights into this.

 

How much of that is actual FPS drop and how much just the change in the way the stats view calculates FPS as discussed in this thread?

 

We didn't. Sounds like you were and are CPU bound (you should use the profiler to confirm this) and URP Render thread performance regressed, as I outlined above, and as we are tracking as issue 1277222.

Regarding the build time regressions, I think that would.be worth a separate thread and maybe a bug report.

 

Wait, you're using HDRP? This thread and the performance regression bug we're all talking about in here is URP only. So whatever you're experiencing is it's own thing and maybe you should report a bug for that so that it can get fixed.

I might be biased but I'm a fan of challenging/validating your performance assumptions by using appropriate tooling, like the Profiler.

 

URPs version shipped with 2020.2 is taking more time on the Render Thread, meaning you can get CPU bound where you previously wouldn't be, but it depends on your hardware setup. For HDRP, it uses the same underlying SRP framework but could be vastly different in the usage so I would not conclude the same root cause for both issues but rather track these separately.

Now 2020.1 will no longer receive any fixes including those you might need to be able to ship to some platforms as they change their requirements so you do NOT want to lock down on that version for good, but aim at eventually upgrading to an LTS version like 2020.2 which will turn into 2020.3/LTS. I understand you might not want to stay on it for the time being until the issues are resolved but since we haven't identified your issue here, we don't know if it is even known and getting fixed so you might be waiting indefinitely.

If you don't want to end up in a situation where you are somehow forced to update Unity while the regression is still on the version you'd upgrade to, just to be able to ship, and then ship a slower build, you should really investigate that issue (I've outlined how that could look like above), report it as a bug and only then roll back.

Making assumptions about your performance and about issues being known and getting fixed is not going to help anyone much. But if you want to discuss this further, I'd suggest to create a new thread for HDRP specifically to not take this one further off topic.

 

you sure you didn't post that in a different thread?

 

If you are basing the assumption of when the LTS version should get released on the year it is released in, our addmittedly confusing version naming tricked you. The 2020 LTS version branch has been branched off of the main line in 2020, as 2020.2 turned beta. That branch is now just receiving fixes until it is marked LTS. I guess back in 2017 we should've started calling the version we were working on 2018, so that it is geared towards the LTS release date, but that ship has sailed I guess...

As for the bug, I'm not working on it and just glanced at the bug ticket. It has an internal note on it from today so it is actively being worked on and the cause is still being narrowed down.

 

If you came to that conclusion based on the fps in the Stats overlay of the Game View, please read through this thread carefully.

 

Not sure if this is a Linux issue but could you please file a bug report for that anyways? (Ideally a specific one, e.g. for the shader drop-down)

 

The fix for Issue 1277222 landed on 2021.1.0b10 i.e. before 2021.1.1 so what you are seeing is a different issue. Can you please file a bug report on it?

 

QA wasn't able to reproduce it, so it was closed and never made public. If there's more hints as to how to repro it, it can get reopened though. For those waiting for something to be fixed, hoping it's the same issue they are facing: please just report individual bugs. Chances for overlap between these with just such a high level problem description seeming to match is low so you might just be waiting for the wrong fix. Also, more reports means more data and context to find such regressions better.

 

Are you referring to the game view's stats display showing you that FPS? If so then yes, you were looking at a faulty number pre 2020.2. also, you kinda still are: The only performance that should really matter for shipping is measured in a built Player on your target platform, not in the Editor.

 

The original state of the stats view? I agree, hence why we changed it. That performance is different in a built Player on your target platform than it is in the Editor should be rather self explaining but still causes people to stumble.

It isn't that the number previously was pure fabulation, it just attempted to hide the time spend on computing the surrounding editor, which also shifted all thread and GPU frame syncs around so you could miss ilthat you would be GPU or Render thread bound.

The Profiler works and always has. It never attempted to hide EditorLoop time, it just deemphasizes it and tries to reduce the profiling overhead by not collecting samples for any calls within the EditorLoop, unless you explicitly try to profile the Editor. Profiling overhead is a thing, especially when Deep Profiling, but the times it reports is what is being measured.

But yes, maybe the stats view needs a stronger disclaimer, that fps in the editor are just a rough and relative measurement. It might've wrongfully been assumed as a given understanding. The profiling documentation is already pointing this out in most places.