Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Enter the 2020.2 Beta Sweepstakes for a chance to win an Oculus Quest 2.
    Dismiss Notice

Feedback Unity 2019.3 Performance Overview

Discussion in '2019.3 Beta' started by Peter77, Aug 27, 2019.

  1. MechEthan

    MechEthan

    Joined:
    Mar 23, 2016
    Posts:
    166
    What I don't understand is:
    Why do @Peter77 's quarterly performance overview posts seem to come as "new information" to Unity?

    Shouldn't whatever he's using have long since been integrated into automated nightly tests that help bisect the code and identify the sources of perf. regressions?

    I assume Unity has tests LIKE this, and maybe what Peter77 is running is catching something they are overlooking, whether it be due to the test environment, or some other factor? Or are the regressions known, and an accepted cost of other improvements or features?

    I DON'T mean for this post to be a dump on Unity, nor to trivialize what is obviously a very complex problem, I'm just truly confused here and looking to improve my understanding of the situation.
     
    Ruslank100, Lorash, Cynicat and 8 others like this.
  2. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.0f1, which is Release Candidate 1. I profiled all three builds listed in the graphs below during the same session.

    2019.3.0f1 continues to be the slowest Unity release since 2018 with my project.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
  3. Roni92pl

    Roni92pl

    Joined:
    Jun 2, 2015
    Posts:
    342
    Nice. Did you checked if there is something specific that is getting slower in profiler or it is more general slowdown?
     
  4. interpol_kun

    interpol_kun

    Joined:
    Jul 28, 2016
    Posts:
    133
    I didn't catch up with Unity 4.X, but I started with 5.X and was always sticking to the recent version. The first time I noticed the performance problems was the 2018.1, then I started using SRP's and found out that performance (both editor and mobile runtime) became far worse than it was before.

    Every other version some new stuff started to pop-up. Slower entering play mode (it was always fine for me and my medium mobile projects), then script reloading times, asset importing, editor launch times, creating new resources, compilation times, etc.

    At first, I thought it was because of my dying hard drive, but I checked it and it's quite healthy. And I don't see any regression in games and other software.

    So, from my subjective experience, I noticed a few (at least this is what I noticed) significant changes which made Unity (both Editor and Runtime) slower:
    • Introduction of SRP
    • Package Manager and everything related to that
    They showed an Editor performance improvements that will come in 2020.X cycle, so maybe they'll compensate a bit.

    But anyway, I can't understand the situation here. Can someone clarify?
    Unity had performance regressions through 2017-2019 which were reported not only by @Peter77 (who did the best job) but by other users too. Okay, forums are like hit or miss, maybe you didn't notice (but it's obvious you did, you neglect it). So what about your QA, automatic tests, performance reports? Sorry to ask, but do you even have them? I know that you guys always saying that all performance regressions should be treated as bugs, but you do nothing.

    Ask anyone, you can even create a survey, about overall editor and runtime performance experience. Lots of people switched their projects from 5 or 4.X to 2018 LTS and they constantly crying about their FPS.

    I just don't understand. You have so many people talking about engine became slower through the past few years and all you can do is telling us to bug report that cause you don't see any regressions at all. How could you not notice that?! And I am not even talking about your "production-ready features" like LWRP. It was PrR since 2019.1, but even with 2019.2, you can catch a few serious bugs in the first day of your new project.

    I am really tired of that releases full of bugs, regressions and half-baked features. I sound like an old granddad who is tired of everything, but I really can't cope with that stuff. I don't enjoy Unity anymore. It used to be like that: I got an idea, created a project and launched the editor (like in a minute) and started my stutter-free development with fast iteration cycle. Now it's like open hub, create a new project and wait 4-6 minutes till all the stuff is imported and the editor is ready and you can start your semi-slow development.

    I recall someone from Unity said you have a goal to reduce script reload times to 0.5s (was that Joachim?) and you will have a new compiler do the job. Compiler did nothing, because times increased. 0.5s was an important number for a fast iteration workflow (and it's not my words, it's yours), but you failed with that goal and in the end, you gave us some hacky workaround which will literally disable the stuff you failed to optimise.

    I do really enjoy all the DOTS stuff (even considering I won't use any of ECS stuff except for educational purpose), but it can't come for the price of everything else. You know that a lot of people are not switching solely because of C#, not because they are happy with the recent Unity releases (me included, but I am already one foot UE4).

    Unity has to evolve in every part. You say it in every hype blogpost, but it's not true.

    This is one of my longest posts on the forum yet. And I wouldn't bother at all if I didn't care about Unity. But I do care, and sometimes I think that it's the dead end.
     
  5. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,223
    I use 2020.1alphas. The performance is exactly the same if not worse than 2019.3 and Unity hasn't mentioned they are looking at it or anything. Basically they hope that you will rewrite all your existing apps and code to use their new DOTS system on the premise that that's the only way to get speed improvements. At least that's the feeling I'm getting. If you are able to keep using Unity 5... do so. Anything after that is a drastic slowdown on mobile and editor.
     
  6. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Hi Will,

    nobody reached out to me yet, which is fine if you don't need more info. If however you can only fix these issues with more details from my side, please let me know, so I can help.
     
    AskCarol, willgoldstone and Prodigga like this.
  7. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    3,827
    Oh man, same! I thought I was going nuts! 2019.3 is much faster with tons of objects onscreen though, faster than 4.7 at least as I haven't touched 5 but the editor feels sluggish with tons of hiccups, some times an orbit becomes a select for no reason and I think that's related.

    Given the global trend towards electron slugware, decay isn't surprising, what's surprising is the complete silence of @Joachim_Ante because he cares about performance.
     
  8. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    70
    Hey Peter,

    My team has been using your project for testing. I'm not sure how much I can share on the forums, but I will drop you a report sometime next week. I'll outline how the project is used, what new procedures it has triggered and the issues it's helped to find.

    I'll also let you know how we plan to use it for future testing.

    I want to say thanks again for sending it in, it is being used. If anyone else has performance regressions. Please log your bugs and send the files if possible, they will be taken seriously.
     
  9. willgoldstone

    willgoldstone

    Unity Technologies

    Joined:
    Oct 2, 2006
    Posts:
    778
    Hey Peter, sorry I didn't reply - i'm in same office as @JohnC_Unity and will catch up with him next week. Thanks so much for your work on this!
     
  10. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Hi John,

    thanks for your reply! I'm looking forward to the report and any improvements you came up with in the meantime.
     
    phobos2077 and JohnC_Unity like this.
  11. benperry

    benperry

    Joined:
    Nov 9, 2015
    Posts:
    2
    I've got another performance regression to add.
    IJobParallelFor job scheduling is completely broken for me in 2019.3.0f1.
    When updating from 2019.2.14f1 to Unity 2019.3.0f1, my PlayerLoop (in editor) went from ~4ms to ~10ms!!!
    IJobParallelFor jobs seem to be stuck running ONLY on the main thread most of the time... blocking main thread when it could be doing work. This is true in both editor and player.

    2019.2.14f1 profiler results:
    2019.2.14f1 perf.PNG
    2019.3.0f1 profiler results:
    2019.3.0f1 perf.PNG

    As a result, 2019.3 is completely unsusable if I want to ship a performant product.

    I've submitted a bug (1202317) with a small repro project.

    My guess it is a regression caused by this change in the 2019.3.0f1 changelog:
    • Kernel: Improve main thread utilisation for IJobParallelFor jobs
     
    joshcamas, Cynicat, Lynxed and 6 others like this.
  12. Freznosis

    Freznosis

    Joined:
    Jul 16, 2014
    Posts:
    280
    I remember how bad the performance was for Unity 2017. I tried it out and it was unbearable. You can still go back and see the crazy amount of threads about how much performance decreased. I think I was stuck using Unity 5 until 2017.4 had a couple of patch releases because it took them 4 cycles to finally fix the engine. And then when 2018.1 came out they did it again! I only recently switched to 2018.4, but it seems they are doing the same pattern over and over. So I think it's safe to say I won't be using 2019 until 2019.4 patch 11 is released. And by then, they will be pushing 2020.3 very hard. It's a never ending cycle, but since they won't bring a lot of improvements and features to older releases, you are stuck in between better performance and benefiting from new features while getting slow downs.

    I really hope Unity take a very hard look at where the engine is and where it should be by now. They weren't as big of a company before, so it was understandable to have some problems here and there. But in the whole time I've used Unity from 2 to Unity 5, this was never a problem. I wonder what changed starting with Unity 2017? It seems they are trying to do way too much, too fast. They have introduced several new systems, features, and workflows while the rest of the engine is still in a slump.

    I also agree with what was said above. It gets pretty annoying that most of the responses to horrible performance is "submit a bug report." Even if it's a completely new, blank, empty project. I understand it, because as developers we have to go through the same thing with people reporting stuff to us without any information to help us track down the problem. But if they can't track down this slow down after literally years and multiple life cycles of the engine since 2017. Then I don't know how much faith I can continue having in Unity going forward.
     
  13. Charles_Beauchemin

    Charles_Beauchemin

    Unity Technologies

    Joined:
    Jan 18, 2017
    Posts:
    444
    Hi Ben, would you mind trying the last beta version to see if it was indeed a regression introduced in f1?
     
    Lynxed, phobos2077 and Thriving like this.
  14. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,037
    This is a feature package (terrain tools), but something that absolutely shouldn't be happening off in individual features, this is a huge hit on load time iterating through all assemblies and types. Don't you have a single place in the loading cycle to do this that packages can hook into?

    NoiseLib in the terrain tools package, GatherNoiseTypes.
     
  15. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    3,827
    How is compile time when you change a script?
     
  16. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    My very non-scientific approach to measure it was like:
    1. Hit "Reimport" on a script and start stopwatch on my cellphone at the same time
    2. Place mouse pointer over a window splitter in Unity, so it changes from "arrow" to "east-west" icon
    3. Wait until Unity freezes (domain reload) and move mouse pointer somewhere else
    4. Wait until mouse pointer changes from east-west to arrow again (domain reload done, Unity responsive again)
    5. Stop stopwatch

    Code (CSharp):
    1. 2017.4.34f1  11s
    2. 2018.1.9f2   11s
    3. 2018.4.12f1  13s
    4. 2019.3.0f4   9s
    2019.3 compiles quite fast, most of the time seems to be spent on the domain reload. From my observation I would say perhaps 4s compiling (rotating spinning icon), 5s domain reload (Unity frozen).
     
    laurentlavigne and MechEthan like this.
  17. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    70
    Peter77 and Lars-Steenhoff like this.
  18. FcsVorfeed

    FcsVorfeed

    Joined:
    Aug 15, 2014
    Posts:
    33
    We don't want any "Enter Play Mode", because i have to change all of my 100000000+ scripts ,if not? when i "Enter Play", all of system will print errorBUG

    We just want click "enter play" and every thing is like Unity 5 fast

    If you really tell me have to change 100000000000000000+ script , i have a question:
    "Have you ever really developed a game? "
     
  19. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Thanks for the tip! I believe this new feature isn't going to help when compiling scripts, as in order to use the new script code, unity is doing the assembly/domain reload per definition. But I'm no expert on this, but @alexeyzakharov can most likely answer it.
     
  20. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    70
    Following up on the status of this issue.
    I looked into the bug it seems they have found where it regressed. Not sure on the status of a fix, but I'll find out and keep you posted.

    Many thanks.
     
  21. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    3,827
    Also the caveats of domain reload are substantial and require a lot of verbose code.
    I see that you folks already considered whiping the static variables, I skimmed over the reasoning, maybe it'll work in 50% of all cases and enough for us?
    Pretty nice, it seems slower than 4.7 which is the only version that etched itself on my mind.
     
    Last edited: Dec 20, 2019
  22. MechEthan

    MechEthan

    Joined:
    Mar 23, 2016
    Posts:
    166
    Is "domain reload" an all or nothing thing for the Unity "Enter Play" step? Or can it be limited to specific assemblies?

    For example, if I have my code broken up evenly between 10 assemblies (via asmdef files), and I adopt these manual reset-statics patterns, then change a script in 1 assembly:
    1. Do all of my 10 assemblies need to be reloaded? (along with various Unity assemblies)
      OR
    2. Can just the 1 assembly that got recompiled also be the only assembly needing a "domain reload"?
    (Thanks in advance!)
     
    hurleybird and Peter77 like this.
  23. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    MechEthan likes this.
  24. rastlin

    rastlin

    Joined:
    Jun 5, 2017
    Posts:
    127
    In general, you cannot EVER unload assembly from app domain. Only way to replace the in-memory assembly is to dispose and start new AppDomain.
     
  25. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.0f5, which is Release Candidate 5. I profiled all three builds listed in the graphs below during the same session.

    2019.3.0f5 continues to be the slowest Unity release since 2018 with my project.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    mGMM, Griz, arkogelul and 13 others like this.
  26. spaceemotion

    spaceemotion

    Joined:
    Sep 29, 2015
    Posts:
    91
    Would love to see if the jump between f3, f4 and f5 did anything. You'e only listed f5 in your charts.
     
  27. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    I've uninstalled f3 and f4 already, but I did profile f4 and have the results! :) No results for f3 though.
    scene_4_3.png

    scene_5_4.png

    scene_6_8.png

    I didn't post the f4 results on purpose, because it was during the holidays season. I thought it would be unfair to post it, if Unity staff would be on holidays and can't respond/comment. I don't know, felt wrong.
     
  28. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.0f6. I profiled all three builds listed in the graphs below during the same session.

    2019.3.0f6 continues to be the slowest Unity release since 2018 with my project.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    DannyWebbie, pcg, iamarugin and 11 others like this.
  29. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.4f1. I profiled all three builds listed in the graphs below during the same session.

    2019.3.4f1 continues to be slower than Unity 2018.1 with my project.

    I've also submitted bug-report Case 1222370, which is related to my work and important to me, but hasn't looked at by QA yet. If you could process this report, that would be splendid: (Case 1222370) 2019.3: Android performance regression

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    Last edited: Mar 7, 2020
  30. Freznosis

    Freznosis

    Joined:
    Jul 16, 2014
    Posts:
    280
    Good to see Unity performance is getting better. I still wonder what is making it slower by a whole 1ms though.
     
    Ruslank100 and mpgholden like this.
  31. Yasei_no_otoko

    Yasei_no_otoko

    Joined:
    Nov 25, 2013
    Posts:
    13
    This is a great test. What happens in 2018.2/3/4?
    Unity's .NET runtime is Mono, but the compiler has been changed to Roslyn (Microsoft) from 2018.3. In addition to bugs, the difference in fine behavior may affect speed between 2018.1 and 2019.x.
     
  32. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    That's up to the people at Unity to figure out :)
     
    Ruslank100 and JamesArndt like this.
  33. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Hey @LeonhardP ,

    could you ask QA to take a look at:
    (Case 1222370) 2019.3: Android performance regression

    I've submitted the report 2+ weeks ago, but it hasn't been processed yet. Thanks in advance!
     
    Ruslank100, LeonhardP and JamesArndt like this.
  34. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,836
    Can you link to the bug report perhaps? I'm curious about upgrading to 2019.3 from 2018.4 and the bulk of my work is for the Android platform. I'm hoping for overall performance improvements using the new URP vs the old rendering pipeline.
     
  35. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    I will, once QA processed it and made it available in the public issue tracker.
    Here is the description:
    upload_2020-3-12_17-32-39.png
     
  36. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    JamesArndt and valarus like this.
  37. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.6f1, the LTS releases and of course 2018.1.9f1 which has been the benchmark for performance for a while. I profiled all builds listed in the graphs below during the same session.

    @richardkettlewell
    You probably notice the graphs changed! The graphs are a bit more zoomed in, but everything is also about 0.4ms faster now.

    Why is that you ask?

    The reason why everything is about 0.4ms faster now is that earlier tests had the "Script Debugging" build option enabled. I've now turned it off, for this and upcoming tests, because the script debugging cost won't exist in the final build, thus I don't care about it.

    All earlier tests are still valid and continue to show regressions, they're not wrong, they just include a cost that's irrelevant for the final release that won't be included in the next tests.

    2019.3.6f1 continues to be slower than Unity 2018.1 with my project.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    Last edited: Mar 22, 2020
    Thriving, Gearmoon, MechEthan and 2 others like this.
  38. MechEthan

    MechEthan

    Joined:
    Mar 23, 2016
    Posts:
    166
    I don't want to clutter this informative thread, but I definitely want to put out a public THANK YOU to @Peter77 for continually posting updated information.

    For us smaller dev teams, this stuff is very useful to know and we don't always have the time to do proper investigations and comparisons like this.
     
  39. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Hey thanks :)

    It's quite some time that goes into these updates. Downloading and installing new Unity releases, re-building the project with each version, running the tests and then combining the results in graphs. That's easily a few hours that went into the last update I posted.
     
    MechEthan and Thriving like this.
  40. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.6f1 using Unity's different Graphics API's. I profiled all builds listed in the graphs below during the same session.



    scene_4_3.png

    scene_5_4.png

    scene_6_8.png


    I've done the same test during Unity 2019.1 beta, see:
    https://forum.unity.com/threads/performance-overview.637783/#post-4377217

    What can we see?
    • D3D12 performance slightly improved compared to 2019.1, but still way behind D3D11.
    • OpenGL Core regressed somewhere between 2019.1.0b9 and 2019.3.6f1, as the current peaks where lower when comparing to the earlier captures. (don't let the different graph y-range confuse you, it looks worse than it is) @JohnC_Unity
     
  41. interpol_kun

    interpol_kun

    Joined:
    Jul 28, 2016
    Posts:
    133
    You would be surprised when you'll run those tests on mobile phone (Android).
     
  42. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    I probably would be :) Do you have any stories to share on that topic?
     
  43. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    70
    Thanks a lot, Peter. I've passed on the post to the graphics team to see if that can give you some more information.
     
    Thriving and Peter77 like this.
  44. interpol_kun

    interpol_kun

    Joined:
    Jul 28, 2016
    Posts:
    133
    The most interesting story was the one when I noticed huge FPS drop using LWRP+Vulkan (2019.1) on low-end device from 2018 (Huawei P Smart).

    I had my first game created in 2016 with Unity 5.X and I sure have some sort of ~feeling~ that new versions of Unity are noticeably slower on phones. Considering the fact that Unity is the 1st engine on mobile, that would be quite upsetting.
     
    studentvz and FcsVorfeed like this.
  45. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    70
    Hey folks,

    Sorry for the delay. I had a chat with the graphics team. Their goal is to get performance parity with Vulkan for 2020.2

    Hope that helps.
     
  46. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,223
    I haven't followed this thread for a while. Is 2019.3 finally at the same speed than 2018 or still sucking big time?
    What about 2020.1? Did it manage to come close to 2018 or still going the 2019 route of "slow is a feature, not a bug".
     
    JamesArndt likes this.
  47. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    pt-paulrahme and JamesArndt like this.
  48. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,223
    Thanks. So I see it's still sucking big time and no benefit in performance from 2020 to 2019 or 2018.
     
    Zandarn and JamesArndt like this.
  49. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,836
    Peter77 likes this.
  50. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,226
    Today I re-ran the performance test with 2019.3.14f1 and of course 2018.1.9f1 which has been the benchmark for performance for a while. I profiled all builds listed in the graphs below during the same session.

    scene_4_3.png


    scene_5_4.png

    scene_6_8.png
     
unityunity