Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Feedback Unity 2020.1 Performance Overview

Discussion in '2020.1 Beta' started by Peter77, Mar 12, 2020.

  1. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    I'm curious to see what % slowdown 0.6ms represents
    what's the total ms of your baseline?
     
  2. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Yeah that information is missing in the new graphs. Here are the numbers in the old layout...

    scene_4_3_absolute.png

    scene_5_4_absolute.png

    scene_6_8_absolute.png

    PS: Unfortunately the forum doesn't allow me to upload 6 images in the same post, otherwise I'd added them to the other post.
     
  3. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    Have you considered sampling these things with the profiler, and using the profiler analyzer to compare what's taking time in the different versions?
     
    Peter77 and phobos2077 like this.
  4. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Yes I've done that from time to time:
    https://forum.unity.com/threads/unity-2020-1-performance-overview.845506/#post-5586094

    But it feels like it just makes things more complicated with no additional benefits for this specific "Performance Overview" type of thing and would basically just waste even more time on my end.

    I don't care what specific part it slower. I just see performance worsen over time. That's all what matters from my perspective.

    It's great if I wanted to optimize > compare > optimize the game, but that's not what I'm doing. They have to do this, not me.
     
  5. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    This sounds quite worrying and I "hope" this is tied to your project setup.

    I still have the 2018.4 xbox build installed on the kit so I'll be soon testing a 2019.4 build in there. 1ms would be catastrophic loss, moreover specting performance up duew to new versions, also 2020 was promising speed boost on code side....
     
  6. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    Disclaimer: these graphs are sampled on Android. Each platform has its own team. So at similar hardware speed it's not certain that this 10% decay (huge) is also seen on iOS, the Switch or your toaster oven.

    It would be useful for us if Unity released their internal results. Transparency is good.
     
    phobos2077 likes this.
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    What graphs do you mean? Did you forget to attach some images to your post? The graphs I posted are from a Standalone Windows 64bit Player.

    Yeah, performance always depends on the project. There are most likely projects out there that run faster in newer Unity versions, but I've not seen those yet.

    I've updated two projects to a new Unity version within the last months (1 PC, 1 Android), but both run slower now. Unity confirmed these regressions, so I have hope it gets fixed ;)

    Please let us know how your project runs after the update.
     
  8. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    I thought they were from low end android phones like in your other post.
    Then, everyone, ignore my disclaimer, you can start panicking.
     
  9. Neonlyte

    Neonlyte

    Joined:
    Oct 17, 2013
    Posts:
    513
    I don't know if anyone has ever shared this talk, or whether the regression in this thread matches the situation described in the talk but here's "Performance Matters" by Emery Berger, where he talks about when performance gains are accidental and brittle, and sometimes performance loss can caused by factors other than code change, and a good way to analyze performance without these outside factors:
     
    Last edited: Jul 1, 2020
    Inter-Illusion likes this.
  10. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Hey @Peter77,
    Have you ever tried to turn on graphics jobs?
    I know adjusting projects post upgrading is not the point here and that this would mean comparing apples to oranges with regards to versions pre 2019.3, where graphics jobs came out of experimental. But for new projects on 2019.3 or newer, this option is now on by default, while I assume that upgraded projects remain with the option turned off, so I guess it would be off in your case. So this should be a simple, switch on improvement from 2019.3 forward and I was just curious how much, if any, of an effect this would have in your case.
     
    Peter77 and QFSW like this.
  11. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    All my "Performance Overview" tests (since 2017) are using graphics jobs already. :) The project even helped to identify at least one graphics jobs issue (link).
     
    MartinTilo likes this.
  12. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Oh cool!* I was only looking through the 2019.3 thread and this one before asking and didn't see any mentioning of them so I thought I'd check.

    *Well, aside from the regressions obviously
     
  13. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    so, I did a build on 2019.4.1f1, deployed only to the Xbox S devkit.
    At a first glance what is clear is that is not working faster but I can´t say its running slower.
    I've yet to swich to retail mode, install.. test etc...

    But overall feelings where not great. Notice tho, its not just an upgrade from 2018.4 I've also switched from Gamma to Linear and added some extra lights (without shadows).
    With 11 Ai tanks, performance was kind of meh. I've graphics jobs on in both cases. I may test DX12 and see if there is any improvements.
     
  14. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    Hello again, I'm sorry I can´t yet provide feedback on 2020 beta but after several builds on dx11 and dx12 on the Xbox I can´t find a regresion on performance on my project, at least for dx11 there is an unrelated ESRAM issue with dx12 on Xbox.

    If anything it could be slightly faster. The only change vs 2018.4 project is that I switched to linear and slighly modified the postproces stack profiles to fit new color space.

    Tested in same map with same AI in retail with console in retail vs current in debug build with debug on (cold be slighly slower because of that).

    Cheers.
     
    MartinTilo likes this.
  15. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    You know it's Saturday when you look at a new performance graph :)

    Today I re-ran the performance test with 2020.1.0b15, 2018.1.9f1 and 2020.1.0b4. I profiled all builds listed in the graphs below during the same session.

    Unity 2020.1.0b4 is the oldest beta I have installed, it would be b1 otherwise. It allows to see how the beta performance improved from "first beta" to "latest beta".

    Unity 2020.1.0b15 continues to be slower than 2018.1.9f1 with my project.
    Unity 2020.1.0b15 is faster than 2020.1.0b4, which is a good trend.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
  16. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    Still 10% drop. On PC = who cares. On a Switch = ouch.
     
  17. Refeas

    Refeas

    Joined:
    Nov 8, 2016
    Posts:
    192
    Hey @Peter77 great reports as always! Would it be possible to post a comparison between current 2019.4 LTS and the latest beta, so we can see if there are any optimization trends there? Thanks!
     
    JamesArndt likes this.
  18. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Yes I can do that. Will post an update at the weekend.
     
  19. JCorfer

    JCorfer

    Joined:
    Apr 22, 2016
    Posts:
    16
    Hi guys!,

    I've seen the 2019.3 comparison post and this. Before anything, great job @Peter77! ;)

    In my recently experience, I made my first game in 2016 in Unity 5.3.5. I remember that I was improving the performance of the game around one month, and the result was great.

    Recently I comeback to game world and I decided actualice my game to Unity 2019.4.2LTS. After update every libraries and adapted it to new options and requisites, I noticed that the game's FPS is worst, for that I come at here.

    After read all, I've downgraded to Unity 5.6.7 and the game's FPS returns to be great (like always was). I don't know if I update to Unity 2018.1.9 the game's FPS will continue be great or it will be worst again. If I do it, I'll show the results here to try to help, I know that this kind of issues are a authentic nigthmare.

    I want to start a new game and I don't know which Unity's version use to make it :(
     
  20. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    It would be great if you could share some profiler data to try to track down where the slow down is coming from
     
    phobos2077 likes this.
  21. phobos2077

    phobos2077

    Joined:
    Feb 10, 2018
    Posts:
    350
    Exactly. Testing overall performance on different versions of Unity is informative but might be misleading. It may as well have better performance but because of some misconfiguration or user mistakes - ends up being worse.
     
  22. JCorfer

    JCorfer

    Joined:
    Apr 22, 2016
    Posts:
    16
    Hi again!

    Well, I explain my tests.

    1. I updated the game to 2018.1.9 and 2017.4.4 versions getting a wrong FPS.
    2. I updated the game to 2017.1.1, 2017.1.2 and 2017.1.5 but in any case the game works, because to update from 5.6.7 to any of this versions, the scripts not appears correctly in its game objects (I think that @Peter77 says some about that)
    3. Finally, I did two captures to profile executing the game in 5.6.7 and 2017.4.4, and here the results:

      profile-5.6.7-1.png profile-5.6.7-2.png profile-2017.4.4-1.png profile-2017.4.4-2.png
    The first and second images are of 5.6.7 version and the another two of 2017.4.4.

    If I deactivate the v-sync option in profile (from 2017.4.4) the FPS appears in 60, it seems that v-sync is doing that game use 30fps more...

    I'm not shure, but I suspect that the problem could come from XR settings, because this settings was introduced in 2017.2 version, the problem is that I couldn't run my game in 2017.1 to corroborate this, I don't know if I could re-created my scene to try it.

    I hope this is useful.
     
    Last edited: Jul 15, 2020
  23. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    in my view, the first 2 cpu profilings look pretty much the same, in the 60fps range
     
  24. JCorfer

    JCorfer

    Joined:
    Apr 22, 2016
    Posts:
    16
    I found the problem! (At least in my case)

    I changed the option Project Settings -> Quality -> Other -> V-Sync Count from Don't Sync to Every V blank and I get a games's FPS great again in 2017.4.4! I think that is a logical bug (when you indicate yes, it understands no and vice versa) because in 5.6.7 the V-Sync Count is in Don't Sync. I tested this in 2019.4.2 and I get a games's FPS great too!

    profile-2019.4.2-novsync.png profile-2017.4.4-novsync.png

    The first imagen is 2019.4.2 and second is 2017.4.4. It seems that in 2019.4.2 the FPS is slighly less that in 2017.4.4, I guess is that last issue that you discuss. By the way, every tests I'm doing are on Android (Xiaomi Mi 6).

    Now, I can continue updating my game. Thanks and good luck with the graphics issue!
     
    phobos2077 likes this.
  25. JCorfer

    JCorfer

    Joined:
    Apr 22, 2016
    Posts:
    16
    Yes, the first and second image are the same version in the more or less same instant (5.6.7), and the third and fourth image are of the other version (2017.4.4).
     
  26. Refeas

    Refeas

    Joined:
    Nov 8, 2016
    Posts:
    192
    VSync will just lock your FPS to your screen's refresh rate, so it has to be turned off (Don't Sync) to conduct any meaningful performance testing. Let's say you have your VSync off and you have 20 FPS in your game. By turning VSync on, you will get 60 FPS, but you will notice a pretty significant input lag, because the VSync is just compensating for the frames that were never really rendered.
     
  27. JCorfer

    JCorfer

    Joined:
    Apr 22, 2016
    Posts:
    16
    Yes, I understand it but like I showed, with the V-sync In off in Quality settings in 2017, after install the app, the FPS was about 30 and Vsync appears in the profile. Later I change to Every V Blank and it works fine, without lag. In fact, in the profile the vsync doesn’t appear o_O

    It seems a option bug...
     
    phobos2077 likes this.
  28. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Here is a 2019.4.4f1 vs 2020.1.0b16 comparison:
    scene_4_3.png

    scene_5_4.png

    scene_6_8.png

    I'll post my regular 2018.1 vs latest beta, as well as full LTS comparison tomorrow.
     
    Joc36 and richardkettlewell like this.
  29. Refeas

    Refeas

    Joined:
    Nov 8, 2016
    Posts:
    192
    Thanks Peter! Seems like there's a slight performance improvement there - nothing major - but a good sign overall.
     
    Peter77 likes this.
  30. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Here is the regular 2018.1 vs latest beta comparison.

    scene_4_3.png

    scene_5_4.png

    scene_6_8.png
     
    Joc36 likes this.
  31. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Here are the LTS comparisons :)

    scene_4_3.png


    scene_5_4.png


    scene_6_8.png
     
  32. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    Thanks for the LTS comparison. It seems like 2018.4 is a good compromise between perf and modernness (modernity? recentlessness?)
    Can you remind me what each samples comprise of and if some are more GPU/physics/cpu heavy?
     
    Peter77 likes this.
  33. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    It's explained in the "How does the test work" spoiler in the first post. The tests should be rather light on the GPU, I've reduced the resolution to 320x240 to avoid being GPU bound.

    I was just repeating the test with 2018.4.24f1 and enabled to log the profiling data (link) to load them in Profiler Analyzer afterwards and then post images here, so you can look at it in more detail.

    However, trying to load the profiler recordings in Profiler Analyzer shows:
    Code (CSharp):
    1. Incorrect file version in performance.pdata : System.Exception: File version unsupported : 1429423184 != 7 expected
    2.   at UnityEditor.Performance.ProfileAnalyzer.ProfileData..ctor (System.IO.BinaryReader reader) [0x0008d] in Library\PackageCache\com.unity.performance.profile-analyzer@1.0.2\Editor\ProfileData.cs:266
    3.   at UnityEditor.Performance.ProfileAnalyzer.ProfileData.Load (System.String filename, UnityEditor.Performance.ProfileAnalyzer.ProfileData& data) [0x000c8] in Library\PackageCache\com.unity.performance.profile-analyzer@1.0.2\Editor\ProfileData.cs:349
    Soooo... I was creating a bug-report for above issue, but while creating the bug-report, the editor froze and didn't recover. So I submitted a bug-report for the editor freeze instead (Case 1264239).

    I have enough Unity for today now.
    upload_2020-7-19_7-52-12.png
     
  34. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    treat yourself to EmbergenFX
     
  35. DavidBVal

    DavidBVal

    Joined:
    Mar 13, 2017
    Posts:
    206
    @Peter77 your perseverance and efforts are greatly appreciated. Unity should have already dissected the issue and explained the difference in performance, at least that.
     
    JesOb, Ruslank100, NotaNaN and 4 others like this.
  36. valarnur

    valarnur

    Joined:
    Apr 7, 2019
    Posts:
    438
    Is it possible to get 19.4LTS performance fast as 18.1.9?
     
    JamesArndt likes this.
  37. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    @Peter77 so, the slowdown spreads overall or you managed to track to an specific area ¿?¿?
     
    phobos2077 likes this.
  38. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    I didn't really look into it, but the last time I used Profiler Analyzer, it showed overall script execution got slower:
    https://forum.unity.com/threads/unity-2020-1-performance-overview.845506/#post-5586094

    I'm not going to look into what specifically is slower, Unity Technologies can do this. I've done this in the past, but it doesn't really accelerate their fixes, so it's really just wasting time on my end. Perhaps it's something simple like turning on a certain option and suddenly every newer Unity release outperforms 2018.1.

    They have the project, they can investigate if they're interested.
     
    stonstad, JesOb, NotaNaN and 6 others like this.
  39. lyndon_unity

    lyndon_unity

    Unity Technologies

    Joined:
    Nov 2, 2017
    Posts:
    64
    Hi, I worked on the Profiler Analyzer and just spotted this post.
    It sounds like you may have tried to load the file you output using Profiler.logFile and Profiler.enableBinaryLog directly into the Profile Analyzer?
    The file you output is a .raw file that can be loaded into the Unity profiler. Once load into the Profiler you can use the Profiler Analyzer to pull the data from the Profiler.

    We plan to add support for loading this file directly into the Profiler Analyzer in a future release, but unfortunately this feature didn't make it to the 1.0.2 release.
     
    Peter77 and MartinTilo like this.
  40. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    That's ... not how VSync works. if you have 20FPS without VSync and turn VSync on, you'll get 15 FPS on a 60 Hz monitor. If you have 120 FPS and turn it to Every VBlank, or 60 FPS and set it to Every Second VBlank, you'll half your frame rate and your Input will only be processed every 16ms or 33ms respectively but besides 33ms being too infrequent for e.g. fighting games, I don't see the Input Lag problem you described...
    Something to keep in mind is also the Application.targetFrameRate setting and how mobile devices will always be capped to their screen refresh rate and might go for a lower rate if default settings are used.

    Now there was a bug on 2019.3/4 about input lag but that should be solved. Beyond that, I'd search for issues in your own handling of the Input. Also, 2019.3 comes with On-demand Rendering to decouple Input and Game Logic cycles from the Rendering frequency somewhat so that might be interesting to look at if you want to poll input at a higher frequency than your effective rendering frame rate...

    Back to the topic of this thread, it'll be interesting to see how the reshuffling of the Update loop to fix the fluctuations in the deltaTime (landed for most Platforms for 2020.2) will affect Peter's test cases...
     
  41. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Yes, that's what I tried. Thanks for the tip to load it in the Profiler instead and then pulling the data in Analyzer.
     
    MartinTilo likes this.
  42. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Once 2020.2 enters the beta cycle, I'll post a "Performance Overview" for that version :)
     
  43. ImpossibleRobert

    ImpossibleRobert

    Joined:
    Oct 10, 2013
    Posts:
    521
    @MartinTilo what is the reshuffling all about? What are the "expected" results? It sounds quite interesting.
     
  44. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Only seeing that development from the sidelines here but basically: Moving the order in which systems are updated and timings querried so that Time.deltaTime should be waaaaay more stable frame over frame, particularly when vSynced or with Application.targetFrameRate. There is likely to be a writeup, possibly a blog post on this later that might be able to provide more details. Ideally you shouldn't notice much more than tings being smoother, particularly with stuff like Timeline (the sequencing tool, not the Profiler one ;)).
     
  45. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
  46. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    colltowncolltown likes this.
  47. colltowncolltown

    colltowncolltown

    Joined:
    Aug 23, 2019
    Posts:
    3
  48. colltowncolltown

    colltowncolltown

    Joined:
    Aug 23, 2019
    Posts:
    3