Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. We are looking for feedback on the experimental Unity Safe Mode which is aiming to help you resolve compilation errors faster during project startup.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Feedback Unity 2019.3 Performance Overview

Discussion in '2019.3 Beta' started by Peter77, Aug 27, 2019.

  1. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    Today I ran the performance tests of my game with Unity 2019.3.0b1, 2019.2.0f1 and 2018.1.9f2. I present the results below, just like I did before, during older beta cycles.

    According to my test, 2019.3.0b1 is slower than 2019.2 and significantly slower 2018.1. I profiled all three builds listed in the graphs below during the same session.

    You can run the test using the project attached to the following bug-report:
    (Case 1108597) Project to find performance regressions and bugs

    The (first-person) game I'm working on features an automated performance test. It works like this:
    • Camera is placed at a defined position in the scene
    • Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)
    • Every second the test captures the average CPU frame time of the last second
    The game runs just like it would normally do, but the camera/player does not move and the AI in unable to see the player.

    It measures the "base-line" of the game so to speak. If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.

    I run this test in a Standalone Windows 64bit (Mono) Player:
    • 2018 = .NET3.5
    • 2019 = .NET4.x
    I've also enabled "Incremental GC" on both 2019 versions.

    The following Physics settings are applied:
    • Physics.autoSyncTransforms=false
    • Physics.autoSimulate=true
    • Physics2D.autoSyncTransforms=false
    • Physics2D.autoSimulate=false
    The resolution is set to 320x240 (D3D11) to make sure the bottleneck isn't GPU.

    The y-axis represents the average CPU time in milliseconds, how long "one frame took". The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test. Fewer milliseconds (vertical axis) indicate better performance.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png



    These issues are probably not reproducible if you're running the game on high-end hardware. Please use a rig similar to the hardware I used to submit the bug-report.

    EDIT: The 2020.1 performance overview can be found at:
    https://forum.unity.com/threads/unity-2020-1-performance-overview.845506/
     
    Last edited: Mar 14, 2020
  2. carmonava2

    carmonava2

    Joined:
    Apr 22, 2018
    Posts:
    1
    Thanks
     
    Prodigga and Peter77 like this.
  3. darkydoodle

    darkydoodle

    Joined:
    Oct 27, 2018
    Posts:
    54
    Personnaly I have much more problems with GPU performance. Granted, I use a GTX770M, but even with a simple shader on HDLit master node (or even PBR), I get near 2000 fragment shader code lines. I can spend a lot of time optimizing my shaders (and I do it through custom nodes), but it's a drop in the water next to what HDRP is generating at the end.
    It's much easier to optimize the CPU side (no game objects except when absolutly necessary, instancing, threading, etc...). I have lots of behaviour trees AI running at the same time, some meteo calculations, minimaps, etc but I think that even with twice what I'm doing now, I'd still be GPU bound.
     
  4. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    Yeah I'm also noticing a big performance drop on my mobile test. My android device used to run at 30fps no problem and it's usually around 24fps now. That's a 20% drop on performance from 2019.1 and I can't test it now but I think 2018 was even faster.
     
  5. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    67
    Hey Peter,

    Perhaps im blind and can't see it but do you have your machine specs listed somewhere? Thanks.
     
  6. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    As far as I know, Unity collects hardware specs when submitting a bug-report. If that's true, you should be able to find the specs in Case 1108597.

    Otherwise please let me know and I will add them to the report.

    Edit: I added them to a forum thread a while ago too:
    https://forum.unity.com/threads/performance-overview.637783/#post-4298446
     
  7. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    67
    Awesome! Thanks for that Peter.
     
    Peter77 likes this.
  8. konsic

    konsic

    Joined:
    Oct 19, 2015
    Posts:
    995
    Vsync in game mode is not present in 2019.3.
    Could this be the cause of CPU/GPU overhead?

    My CPU/GPU fans begins to spin rapidly when entering game mode in HDRP.
     
  9. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    3,211
    I've been following your performance test results in the Betas for quite a few versions now, always interesting stuff. There's something I don't really understand. I can understand the Unity Editor itself getting slower with each new major version. (And I think that's somewhat reasonable.) But I really don't understand why an optimized build of a game would have worse performance in each version of Unity. If anything, I would expect build performance to improve, not worsen.

    Has there ever been an explanation of why build performance decays? It would be one thing if you were using new features of a newer version, and having to accept a performance hit to enjoy the new functionality. But it seems that's not the case with your performance tests. So, do we know why builds are slower in newer versions?
     
    Peter77, mGMM and JohnC_Unity like this.
  10. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    I think the best explanation would be this one:
    https://forum.unity.com/threads/performance-overview.637783/page-2#post-4642300

    I imagine depending on Unity versions, it's just different fixes/improvements that cause the build to run slower. Regression here, regression there.

    That's correct. The project does not use any of the new features, except for things Unity Technologies created to improve performance. For example, starting with 2019.2?, the test uses the incremental GC. But otherwise it's really just a 2 years old copy of my actual project that I didn't change since then.
     
    Last edited: Feb 1, 2020
  11. konsic

    konsic

    Joined:
    Oct 19, 2015
    Posts:
    995
    @Peter77 do you happen to know if Unity performance on Linux is better than on Windows?

    I saw this video for Blender. Might be the same for Unity. Opening Unity project takes longer and longer.

     
    laurentlavigne and Peter77 like this.
  12. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    Nope, I don't know. I've never used Unity on Linux.
     
  13. arkogelul

    arkogelul

    Joined:
    Nov 14, 2016
    Posts:
    104
    Wait... They had a fix that actually improved the performances in 2019.2 and now it's not improved anymore .. I don't get it. What did I miss ? How come 2019.3 is worth that 2019.2 after all they found to be faulty ?

    I ran some tests on my game project comparing the same project on 2018.3 (the version we're currently working on) and 2019.3 (the long awaited version that includes all the bugs we reported, being fixed...). And I found out that both in editor and in build, we have a rough 30% performance drop.

    My game is a sprite based pixel art tactical, so nothing hardcore. And seeing the performances drop version after version is a bit worrying. Especially when we hear about performance improvement quit a lot from Unity.
     
    polysoft3D and mGMM like this.
  14. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    It doesn't have to be the same issue.

    The performance regression I reported during 2019.2 might be fixed and the regression we see in 2019.3 could be something differently.

    Perhaps it's also a different regression in your project than the one I found in my project.
     
    polysoft3D likes this.
  15. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    I never understand why almost every single release Unity posts: performance improvements. But all I see every single release on all my projects is: performance drops.
    I have no idea what performance they improve, but it's definitely not the one for regular projects. At the beginning the drop was just a little but now it's very very noticeable. A game that ran just fine on an iPhone 5 compiled with unity 4? I don't even know how many years ago... same game without changes runs about the same speed on an iPhone 7 using unity 2019.3. The performance drop is gigantic.
     
  16. konsic

    konsic

    Joined:
    Oct 19, 2015
    Posts:
    995
    It's probably becuase multiple version are sprinting by different teams and regressions are fixed afterwards in official version postlaunch.
     
  17. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    7,501
    Can you keep reporting issues for this thing? We definitely are open to fixing performance problems when we see them, it's just that they tend to be super project specific. The fastest way to get them addressed is to figure out what got slower and then report it to us.
     
  18. arkogelul

    arkogelul

    Joined:
    Nov 14, 2016
    Posts:
    104
    It's quite hard to pin point what goes wrong exactly.
    I tried to see why my project runs slower in 2019.3 but couldn't find anything specifically faulty.

    I reported a bug (Case 1186691)
    I made a simple 2D project.
    2 sprites, 1 tile and 1 script (a simple fps counter).
    In the main scene there are around 50 animated GameObjects and a pretty big tilemap (all prefabs).
    It represents an average scene in my game project.

    Here's the fps in 2018.3. (105.3 fps)
    2018.3.png

    Here's the fps in 2019.3. (71.2 fps)
    2019.3.png

    How can I detect what's wrong ?
     
  19. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    7,501
    Can you profile the build instead of the editor? Profiling the editor is not going to be an accurate representation of what frame rate your game will have.

    Also, your screenshot shows 0.1 ms difference on the CPU. That is statistically insignificant to say for sure that 2019.3 is slower.

    Lastly, your frame rate counter doesn't appear to work correctly. The editor numbers don't match your numbers.
     
  20. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    It's very hard to report this thing without including full project. But to give you an example, this is my game:

    That video was uploaded on 2015. Latest unity version back then. Running on android and iOS at 30 fps (or more) on an iPhone 5 and one of those crappy android phones (using android 4.4, I would need to look for the exact phone model).
    It included:
    Unity terrain+over 10k trees+water+all multiplayer going on+one directional light+clouds+ real time weather (you can see on the video raining, thunderstorms and such).

    Today, the same game compiled on Unity 2019.2 or 2019.3, cannot be run at more than 10 fps on those phones. So currently:
    - I had to remove that water quality and put a crappy one. Until a much improved version of the community ocean water was released but the original one was way slower on latest version of unity.
    - Mesh terrain is used instead of unity terrain.
    - I've removed real time weather, it's always sunny, no rain, no heavy clouds.

    Everything else I was able to keep. But instead of adding stuff thanks to performance improvements on Unity, I keep removing stuff until the previous phones get so old that people upgrade I can finally put back what I had years before.
     
    Neogene, KokkuHub, JamesArndt and 5 others like this.
  21. arkogelul

    arkogelul

    Joined:
    Nov 14, 2016
    Posts:
    104
    fraps.png

    The fps counter I use seems fine according to Fraps.
    As far as I know, the rendering statistic window only shows an estimated fps of what it might be in build:
    https://forum.unity.com/threads/sta...66.1944009711.1569165747-108023183.1568208825

    The project I sent with my bug report (Case 1186691) is as simple as it can be.
    Make builds and you'll see that, even if the numbers are slightly higher in builds, there is still a significant difference in performance between 2018.3 and 2019.3.
     
    Last edited: Sep 25, 2019
  22. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    7,501
    That measures the editor FPS, rather than game FPS.

    Anyway, thanks for the bug report - QA will take a look.
     
    Roni92pl likes this.
  23. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    821
    Off topic, since this doesn't actually appear to be the problem here, but yeah, the thing to remember is that the editor *attempts* to remove its own update time from statistics about everything. In my experience it is rather unsuccessful at that. This is especially important to note if you are making anything like a rhythm game, where music is playing entirely unconstrained by the editor update and the latter doesn't actually run in real time as such. FRAPS will show you that the editor window is rendering a x number of FPS but the game view is only a subset of that.
     
  24. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    Today I re-ran the performance test with 2019.3.0b9. I profiled all three builds listed in the graphs below during the same session.

    2019.3.0b9 continues to be the slowest Unity release since 2018 with my project. Performance didn't improve between 2019.3.0b1 and b9.

    What's fascinating (though not related to 2019.3) is that the spikes found in the 2019.2 graph happened again. Last time I thought it could be an one time thing due to some OS activity or so, but it seems it's actually related to Unity or it's really just a major coincidence.


    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    MiFrilke, JohnC_Unity, mh114 and 7 others like this.
  25. polysoft3D

    polysoft3D

    Joined:
    Aug 3, 2015
    Posts:
    21
    Unity 2019.2.1f1 48fps in my project on my xiaomi redmi note 7 64go
    Unity 2019.3.0b9 32fps in my project on my xiaomi redmi note 7 64go

    Exactly same scene and configurations

    What happened?
     
    mGMM likes this.
  26. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    One of the many performance improvements. I keep saying that my game ran just fine on iphone 4 with unity 3 or 4. And now it needs a 64 bits processor, iphone 7 and runs slower. Same game. Not sure why they keep adding stuff and slowing it so much for existing games. I keep having to remove more and more stuff from my game so it can keep up the fps with every Unity performance improvement. (just been sarcastic because that's usually what the release notes say).
     
  27. polysoft3D

    polysoft3D

    Joined:
    Aug 3, 2015
    Posts:
    21
    What about Unity 2020? nothing change?
     
  28. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    yeah, many more performance improvements
     
  29. willgoldstone

    willgoldstone

    Unity Technologies

    Joined:
    Oct 2, 2006
    Posts:
    773
    Hey @Peter77 thanks for all this data, its super useful, would love to talk more about your findings, one of us will reach out to you directly about the test project in more detail. Feel free to email me on will at unity3d dot com too.
     
    Peter77 likes this.
  30. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    2,116
    Would it be possible for you to share your projects with us in a bug report so we can investigate these performance regressions?
     
    Peter77 likes this.
  31. Driven

    Driven

    Joined:
    May 31, 2013
    Posts:
    69
    I can just confirm, the performance dropped drasticaly between 2018 and 2019 and not only between these versions also earlier ones like Unity 5 to 2018 etc. Games once ran smooth and had a small build size on crap devices but not anymore. Also it's said that using always the most recent API's will increase performance but i noticed using vulkan decreases performance even more compared to open gl 2. For me it rather sounds like regression by default than perfomance by default.
    Also i tried setting a lower framerate manually just to see if it stays stable like Application.targetFrameRate = 10; but even this function does not work anymore on Mobile as in ealier versions and just sets the framerate to maximum -.- (~60).
     
  32. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    nvm, message deleted. It was just a big rant and I don't think Unity cares anymore so why should I.

    Good luck with your performance enhancements.
     
    Last edited: Nov 5, 2019
    GoranRiddle and mGMM like this.
  33. polysoft3D

    polysoft3D

    Joined:
    Aug 3, 2015
    Posts:
    21
    i cant upload, my upload speed 80ko/s i will upload my 2go 2 years sorry.
    Just create new scene 2019.2.1f1 and 2019.3.0b9 and use LWPR renderer with 512*512 shadow 80 distance with 1 terrain 4 texture + normal without antialiasing and run on Android device
     
  34. JohnC_Unity

    JohnC_Unity

    Unity Technologies

    Joined:
    Jun 7, 2017
    Posts:
    67
    Hey, polysoft3D what Andriod device are you using. Thanks!
     
  35. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,799
    Am I missing something or do we not have performance metrics for the 2018.4 LTS versions?
     
  36. Freznosis

    Freznosis

    Joined:
    Jul 16, 2014
    Posts:
    270
    Wasn't it because 2018.3 was basically renamed and updated to be 2018.4? There were no betas I believe, which is where he usually does these reports.
     
    JamesArndt likes this.
  37. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    Today I re-ran the performance test with 2019.3.0b11. I profiled all three builds listed in the graphs below during the same session.

    Performance improved between 2019.3.0b9 and b11. However, 2019.3.0b11 continues to be the slowest Unity release since 2018 with my project.

    Two things that changed:
    1. I upgraded from b9 to b11
    2. I deleted the library directory before opening the project (because of the magic that is sometimes going on)
    scene_4_3.png

    scene_5_4.png

    scene_6_8.png


    I haven't profiled 2018.4. I can do this later this weekend.
     
    MechEthan, mGMM, LeonhardP and 5 others like this.
  38. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    I would have liked to update the 2018.3 thread, but I can't post there anymore. I also didn't want to start a new thread, so I post the LTS performance metrics here instead.

    It's located under a spoiler to avoid confusion with actual 2019.3 data...

    2017.4 and 2018.4 because these are the current LTS releases. I've also added 2018.1 to graph, because it's the fastest Unity release for me so far.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png

    PS: While I had 2017.4 open, I noticed how much faster/responsive the 2017 editor is compared to 2019.3! It's such huge difference like day and night. Yet the 4.6 editor is even more responsive than 2017!
     
    MechEthan, JamesArndt, mh114 and 2 others like this.
  39. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    Yeah, every new release is worse and worse. Not sure why this is not a critical thing to look at. Every single game is affected by performance degradation.
     
    JamesArndt likes this.
  40. Freznosis

    Freznosis

    Joined:
    Jul 16, 2014
    Posts:
    270
    I changed all of my major projects over to 2018.4 and this graph make me happy to have done so.

    But I have always wondered for many years, what made Unity releases less and less performant. This has been a consistent issue for every single release since the first 2017 release I believe. I know there are regressions and changes and more features, but it seems to almost always get worse and worse as time goes on. Not just talking game performance, but editor performance. I hope they really sit down and look at overall performance and iteration speed in the near future. (I doubt it though, with how fast everything is moving these days.)
     
    mGMM, JamesArndt and interpol_kun like this.
  41. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,799
    Thank you very much for these tests! I have also noticed the blazingly fast editor usability in 2017.4 vs 2019.xx. Something as simple as right clicking and creating an object is even faster. I have no idea why the editor itself has become much slower and less responsive?
     
  42. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    That's indeed an interesting question. I don't know that either, but perhaps Unity staff can shed some light into it.
     
  43. mh114

    mh114

    Joined:
    Nov 17, 2013
    Posts:
    291
    Personally I started to notice slow down in editor when all these extra "services", such as ads, analytics, etc. started to appear. Looking from the styling of those screens they seem to be implemented in different way to the rest of the editor, but perhaps these days with the new theming and UI Elements stuff this is no longer the case (we're still in 2018.4, and will be for a good while). And of course there are likely other things contributing to the slowness but I suspect these may be one of the causes.
     
  44. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,221
    To me if it wasn't because of the Terrain improvements (they delayed 7 years to implement), I would still be using Unity 4/5 or maximum 2017. Anything after that is just a horrible performance drop not worth to update unless absolutely necessary. It's absolutely horrible and there is no mention whatsoever of anyone looking into it on the betas/alphas.

    I don't care what caused it, I care that they fix it. It doesn't make sense you now need an iPhone 11 to run the same game you could easily run on iPhone 5 a couple years ago with same features and at the same FPS.
     
  45. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    4,995
    That's true. But it's also important to understand what caused this issue to learn from it. Otherwise we end up with the same problem over and over again, no matter how often it "got fixed".
     
    BTStone likes this.
  46. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    6,892
    At this point, this is my number one issue. I have an older version of our project in Unity 4.x that I sometimes open for reference and I am always amazed how fast and snappy everything is.

    The Unity developer experience is in free fall for many years now, Unity keeps announcing they are fixing it, but it seems to keep getting worse.
     
    GoranRiddle, Freznosis, jdtec and 3 others like this.
  47. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,799
    So who knows who works on the Editor team(s) at Unity? Can we tag them in this thread and perhaps gain some insight?
     
  48. konsic

    konsic

    Joined:
    Oct 19, 2015
    Posts:
    995
    This could be used for better frame stutter analysis.
     
    Last edited: Nov 26, 2019
  49. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    3,211
    The thing that's most concerning about such significant performance decline over each version is my previous understanding that moving Unity to a package-based approach was supposed to make Unity lighter and leaner. I was expecting that if enough functionality was moved into packages, and I only include a subset of functionality, then I should see faster performance overall. But it seems like things are actually going in the other direction. It makes me wonder if moving things into packages hasn't had the performance impact that was intended. Maybe to make packages work modularly enough, they needed to add a bunch of extra code that previously was just optimized into the engine. Who knows. I know I've certainly had to accept performance loss as a trade-off for flexibility in applications I've written.
     
    interpol_kun, Roni92pl and JamesArndt like this.
  50. MechEthan

    MechEthan

    Joined:
    Mar 23, 2016
    Posts:
    166
    My understanding wasn't this at all. I thought the reason Unity went with the package-based approach was to decouple development cycles of features from the core engine/toolchain development cycle.

    A side benefit could be improved build times, and maybe smaller build sizes, but not a faster engine. I mean, if you didn't use a feature, it probably wasn't being executed as a part of the update loop / rendering path. Pulling out stuff like AR/VR or 2D Animation isn't going to speed the engine up if you never used them to begin with.
     
    koirat and LeonhardP like this.
unityunity