A Unity ID allows you to buy and/or subscribe to Unity products and services, shop in the Asset Store and participate
in the Unity community.
Discussion in '2020.1 Beta' started by Peter77, Mar 12, 2020.
I'm curious to see what % slowdown 0.6ms represents
what's the total ms of your baseline?
Yeah that information is missing in the new graphs. Here are the numbers in the old layout...
PS: Unfortunately the forum doesn't allow me to upload 6 images in the same post, otherwise I'd added them to the other post.
Have you considered sampling these things with the profiler, and using the profiler analyzer to compare what's taking time in the different versions?
Yes I've done that from time to time:
But it feels like it just makes things more complicated with no additional benefits for this specific "Performance Overview" type of thing and would basically just waste even more time on my end.
I don't care what specific part it slower. I just see performance worsen over time. That's all what matters from my perspective.
It's great if I wanted to optimize > compare > optimize the game, but that's not what I'm doing. They have to do this, not me.
This sounds quite worrying and I "hope" this is tied to your project setup.
I still have the 2018.4 xbox build installed on the kit so I'll be soon testing a 2019.4 build in there. 1ms would be catastrophic loss, moreover specting performance up duew to new versions, also 2020 was promising speed boost on code side....
Disclaimer: these graphs are sampled on Android. Each platform has its own team. So at similar hardware speed it's not certain that this 10% decay (huge) is also seen on iOS, the Switch or your toaster oven.
It would be useful for us if Unity released their internal results. Transparency is good.
What graphs do you mean? Did you forget to attach some images to your post? The graphs I posted are from a Standalone Windows 64bit Player.
Yeah, performance always depends on the project. There are most likely projects out there that run faster in newer Unity versions, but I've not seen those yet.
I've updated two projects to a new Unity version within the last months (1 PC, 1 Android), but both run slower now. Unity confirmed these regressions, so I have hope it gets fixed
Please let us know how your project runs after the update.
I thought they were from low end android phones like in your other post.
Then, everyone, ignore my disclaimer, you can start panicking.
I don't know if anyone has ever shared this talk, or whether the regression in this thread matches the situation described in the talk but here's "Performance Matters" by Emery Berger, where he talks about when performance gains are accidental and brittle, and sometimes performance loss can caused by factors other than code change, and a good way to analyze performance without these outside factors:
Have you ever tried to turn on graphics jobs?
I know adjusting projects post upgrading is not the point here and that this would mean comparing apples to oranges with regards to versions pre 2019.3, where graphics jobs came out of experimental. But for new projects on 2019.3 or newer, this option is now on by default, while I assume that upgraded projects remain with the option turned off, so I guess it would be off in your case. So this should be a simple, switch on improvement from 2019.3 forward and I was just curious how much, if any, of an effect this would have in your case.
All my "Performance Overview" tests (since 2017) are using graphics jobs already. The project even helped to identify at least one graphics jobs issue (link).
Oh cool!* I was only looking through the 2019.3 thread and this one before asking and didn't see any mentioning of them so I thought I'd check.
*Well, aside from the regressions obviously
so, I did a build on 2019.4.1f1, deployed only to the Xbox S devkit.
At a first glance what is clear is that is not working faster but I can´t say its running slower.
I've yet to swich to retail mode, install.. test etc...
But overall feelings where not great. Notice tho, its not just an upgrade from 2018.4 I've also switched from Gamma to Linear and added some extra lights (without shadows).
With 11 Ai tanks, performance was kind of meh. I've graphics jobs on in both cases. I may test DX12 and see if there is any improvements.
Hello again, I'm sorry I can´t yet provide feedback on 2020 beta but after several builds on dx11 and dx12 on the Xbox I can´t find a regresion on performance on my project, at least for dx11 there is an unrelated ESRAM issue with dx12 on Xbox.
If anything it could be slightly faster. The only change vs 2018.4 project is that I switched to linear and slighly modified the postproces stack profiles to fit new color space.
Tested in same map with same AI in retail with console in retail vs current in debug build with debug on (cold be slighly slower because of that).
You know it's Saturday when you look at a new performance graph
Today I re-ran the performance test with 2020.1.0b15, 2018.1.9f1 and 2020.1.0b4. I profiled all builds listed in the graphs below during the same session.
Unity 2020.1.0b4 is the oldest beta I have installed, it would be b1 otherwise. It allows to see how the beta performance improved from "first beta" to "latest beta".
Unity 2020.1.0b15 continues to be slower than 2018.1.9f1 with my project.
Unity 2020.1.0b15 is faster than 2020.1.0b4, which is a good trend.
Still 10% drop. On PC = who cares. On a Switch = ouch.
Hey @Peter77 great reports as always! Would it be possible to post a comparison between current 2019.4 LTS and the latest beta, so we can see if there are any optimization trends there? Thanks!
Yes I can do that. Will post an update at the weekend.
I've seen the 2019.3 comparison post and this. Before anything, great job @Peter77!
In my recently experience, I made my first game in 2016 in Unity 5.3.5. I remember that I was improving the performance of the game around one month, and the result was great.
Recently I comeback to game world and I decided actualice my game to Unity 2019.4.2LTS. After update every libraries and adapted it to new options and requisites, I noticed that the game's FPS is worst, for that I come at here.
After read all, I've downgraded to Unity 5.6.7 and the game's FPS returns to be great (like always was). I don't know if I update to Unity 2018.1.9 the game's FPS will continue be great or it will be worst again. If I do it, I'll show the results here to try to help, I know that this kind of issues are a authentic nigthmare.
I want to start a new game and I don't know which Unity's version use to make it
It would be great if you could share some profiler data to try to track down where the slow down is coming from
Exactly. Testing overall performance on different versions of Unity is informative but might be misleading. It may as well have better performance but because of some misconfiguration or user mistakes - ends up being worse.
Well, I explain my tests.
I updated the game to 2018.1.9 and 2017.4.4 versions getting a wrong FPS.
I updated the game to 2017.1.1, 2017.1.2 and 2017.1.5 but in any case the game works, because to update from 5.6.7 to any of this versions, the scripts not appears correctly in its game objects (I think that @Peter77 says some about that)
Finally, I did two captures to profile executing the game in 5.6.7 and 2017.4.4, and here the results:
The first and second images are of 5.6.7 version and the another two of 2017.4.4.
If I deactivate the v-sync option in profile (from 2017.4.4) the FPS appears in 60, it seems that v-sync is doing that game use 30fps more...
I'm not shure, but I suspect that the problem could come from XR settings, because this settings was introduced in 2017.2 version, the problem is that I couldn't run my game in 2017.1 to corroborate this, I don't know if I could re-created my scene to try it.
I hope this is useful.
in my view, the first 2 cpu profilings look pretty much the same, in the 60fps range
I found the problem! (At least in my case)
I changed the option Project Settings -> Quality -> Other -> V-Sync Count from Don't Sync to Every V blank and I get a games's FPS great again in 2017.4.4! I think that is a logical bug (when you indicate yes, it understands no and vice versa) because in 5.6.7 the V-Sync Count is in Don't Sync. I tested this in 2019.4.2 and I get a games's FPS great too!
The first imagen is 2019.4.2 and second is 2017.4.4. It seems that in 2019.4.2 the FPS is slighly less that in 2017.4.4, I guess is that last issue that you discuss. By the way, every tests I'm doing are on Android (Xiaomi Mi 6).
Now, I can continue updating my game. Thanks and good luck with the graphics issue!
Yes, the first and second image are the same version in the more or less same instant (5.6.7), and the third and fourth image are of the other version (2017.4.4).
VSync will just lock your FPS to your screen's refresh rate, so it has to be turned off (Don't Sync) to conduct any meaningful performance testing. Let's say you have your VSync off and you have 20 FPS in your game. By turning VSync on, you will get 60 FPS, but you will notice a pretty significant input lag, because the VSync is just compensating for the frames that were never really rendered.
Yes, I understand it but like I showed, with the V-sync In off in Quality settings in 2017, after install the app, the FPS was about 30 and Vsync appears in the profile. Later I change to Every V Blank and it works fine, without lag. In fact, in the profile the vsync doesn’t appear
It seems a option bug...
Here is a 2019.4.4f1 vs 2020.1.0b16 comparison:
I'll post my regular 2018.1 vs latest beta, as well as full LTS comparison tomorrow.
Thanks Peter! Seems like there's a slight performance improvement there - nothing major - but a good sign overall.
Here is the regular 2018.1 vs latest beta comparison.
Here are the LTS comparisons
Thanks for the LTS comparison. It seems like 2018.4 is a good compromise between perf and modernness (modernity? recentlessness?)
Can you remind me what each samples comprise of and if some are more GPU/physics/cpu heavy?
It's explained in the "How does the test work" spoiler in the first post. The tests should be rather light on the GPU, I've reduced the resolution to 320x240 to avoid being GPU bound.
I was just repeating the test with 2018.4.24f1 and enabled to log the profiling data (link) to load them in Profiler Analyzer afterwards and then post images here, so you can look at it in more detail.
However, trying to load the profiler recordings in Profiler Analyzer shows:
Incorrect file version in performance.pdata : System.Exception: File version unsupported : 1429423184 != 7 expected
at UnityEditor.Performance.ProfileAnalyzer.ProfileData..ctor (System.IO.BinaryReader reader) [0x0008d] in Library\PackageCache\firstname.lastname@example.org\Editor\ProfileData.cs:266
at UnityEditor.Performance.ProfileAnalyzer.ProfileData.Load (System.String filename, UnityEditor.Performance.ProfileAnalyzer.ProfileData& data) [0x000c8] in Library\PackageCache\email@example.com\Editor\ProfileData.cs:349
Soooo... I was creating a bug-report for above issue, but while creating the bug-report, the editor froze and didn't recover. So I submitted a bug-report for the editor freeze instead (Case 1264239).
I have enough Unity for today now.
treat yourself to EmbergenFX
@Peter77 your perseverance and efforts are greatly appreciated. Unity should have already dissected the issue and explained the difference in performance, at least that.
Is it possible to get 19.4LTS performance fast as 18.1.9?
@Peter77 so, the slowdown spreads overall or you managed to track to an specific area ¿?¿?
I didn't really look into it, but the last time I used Profiler Analyzer, it showed overall script execution got slower:
I'm not going to look into what specifically is slower, Unity Technologies can do this. I've done this in the past, but it doesn't really accelerate their fixes, so it's really just wasting time on my end. Perhaps it's something simple like turning on a certain option and suddenly every newer Unity release outperforms 2018.1.
They have the project, they can investigate if they're interested.
Hi, I worked on the Profiler Analyzer and just spotted this post.
It sounds like you may have tried to load the file you output using Profiler.logFile and Profiler.enableBinaryLog directly into the Profile Analyzer?
The file you output is a .raw file that can be loaded into the Unity profiler. Once load into the Profiler you can use the Profiler Analyzer to pull the data from the Profiler.
We plan to add support for loading this file directly into the Profiler Analyzer in a future release, but unfortunately this feature didn't make it to the 1.0.2 release.
That's ... not how VSync works. if you have 20FPS without VSync and turn VSync on, you'll get 15 FPS on a 60 Hz monitor. If you have 120 FPS and turn it to Every VBlank, or 60 FPS and set it to Every Second VBlank, you'll half your frame rate and your Input will only be processed every 16ms or 33ms respectively but besides 33ms being too infrequent for e.g. fighting games, I don't see the Input Lag problem you described...
Something to keep in mind is also the Application.targetFrameRate setting and how mobile devices will always be capped to their screen refresh rate and might go for a lower rate if default settings are used.
Now there was a bug on 2019.3/4 about input lag but that should be solved. Beyond that, I'd search for issues in your own handling of the Input. Also, 2019.3 comes with On-demand Rendering to decouple Input and Game Logic cycles from the Rendering frequency somewhat so that might be interesting to look at if you want to poll input at a higher frequency than your effective rendering frame rate...
Back to the topic of this thread, it'll be interesting to see how the reshuffling of the Update loop to fix the fluctuations in the deltaTime (landed for most Platforms for 2020.2) will affect Peter's test cases...
Yes, that's what I tried. Thanks for the tip to load it in the Profiler instead and then pulling the data in Analyzer.
Once 2020.2 enters the beta cycle, I'll post a "Performance Overview" for that version
@MartinTilo what is the reshuffling all about? What are the "expected" results? It sounds quite interesting.
Only seeing that development from the sidelines here but basically: Moving the order in which systems are updated and timings querried so that Time.deltaTime should be waaaaay more stable frame over frame, particularly when vSynced or with Application.targetFrameRate. There is likely to be a writeup, possibly a blog post on this later that might be able to provide more details. Ideally you shouldn't notice much more than tings being smoother, particularly with stuff like Timeline (the sequencing tool, not the Profiler one ).
There's a detailed writeup of the problem, independent of Unity, here: https://firstname.lastname@example.org/the-elusive-frame-timing-168f899aec92
Unity 2020.2 performance overview thread can be found here: https://forum.unity.com/threads/unity-2020-2-performance-overview.952580/