Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Performance Overview

Discussion in '2018.3 Beta' started by Peter77, Oct 1, 2018.

  1. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Today I ran the automated performance test of my game with Unity 2018.3.0b3 and present the results below, just like I did before, during older beta cycles:
    According to my tests, 2018.3.0b3 is the slowest of the 2018 series with my project. It's about 1ms to 1.5ms slower than 2018.1 for example.

    I haven't looked into what exactly is causing this performance difference yet.

    The (first-person) game I'm working on features an automated performance test. It works like this:
    • Camera is placed at a defined position in the scene
    • Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)
    • Every second the test captures the average CPU frame time of the last second
    The game runs just like it would normally do, but the camera/player does not move and the AI in unable to see the player.

    It measures the "base-line" of the game so to speak. If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.

    I run this test in a Standalone Windows 64bit (Mono) Player. The following Physics settings are applied:
    • Physics.autoSyncTransforms=false
    • Physics.autoSimulate=true
    • Physics2D.autoSyncTransforms=false
    • Physics2D.autoSimulate=false

    The y-axis represents the average CPU time in milliseconds, how long "one frame took". The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test. Fewer milliseconds (vertical axis) indicate better performance.

    2018_3_scene_4_3.png
    2018_3_scene_5_4.png
    2018_3_scene_6_8.png
     
  2. KillHour

    KillHour

    Joined:
    Oct 25, 2015
    Posts:
    49
    I wonder if it's physics related like my performance issues were.
     
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    It was my first assumption too.

    Physics regressed several times with new Unity versions in my project. I have a simple test project that I used to submit those physics performance regression issues in the past. Unfortunately, I wasn't able to reproduce the performance drop with this test-case in 2018.3 anymore, so I'm not really certain it's (only) physics.

    I need to investigate this in detail, just need to find time.

    What I could find so far, it seems there are at least two performance regressions in 2018.3.

    The physics regression you posted:
    https://forum.unity.com/threads/case-1086243-physics-query-performance-vs-2018-2.563626/

    ... and an UI performance drop:
    https://forum.unity.com/threads/case-1081996-ui-performance-regression.556801/
    https://forum.unity.com/threads/case-1082192-massive-performance-hit-due-to-ui.556402/
     
    LeonhardP likes this.
  4. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,132
    Peter77 likes this.
  5. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    814
    For physix you may also give it a go to the physx experimental editor based on 2018.2 that includes 3.4.1 in my tests that did run like a missile. In .3 betas it did not run that smoothly but there where many other things broken that could be making it lag.
     
  6. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,132
    The regression was introduced during the 2018.3 alpha and doesn't affect the experimental build or earlier versions than 2018.3a6. Let's see if we can get 2018.3 back to missile speed once this has been resolved.
     
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    I re-tested Unity 2018.3.0b6, below are the results. It seems the performance drop now became even more significant :eek:

    While building a Player, Unity printed many:
    ... errors. Maybe the performance drop is somehow related to this issue?! I'll re-check if this problem has been fixed.
    2018_3_0b6_scene_4_3.png

    2018_3_0b6_scene_5_4.png

    2018_3_0b6_scene_6_8.png
     
    Last edited: Oct 20, 2018
  8. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,132
    Last edited: Oct 19, 2018
  9. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    If this is the same test each time from Peter, is there a good reason his test isn't part of Katana or other automated testbed yet given it's been consistently useful :p
     
    ProtonOne and dadude123 like this.
  10. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Yeah I know, I need to find time to investigate these issues. It's a more complex issue that takes time to analyze and come up with an isolated repro.

    I just remembered I actually posted this problem a few weeks ago. I first noticed it using b3, but I don't remember if I earlier tested versions too.
    https://forum.unity.com/threads/cas...ect-that-is-part-of-a-prefab-instance.564151/

    Knowing this issue existed when I tested b3, I guess the performance drop is not directly related to it, since b3 ran faster than b6.

    It's the same test. I created a copy of my game a few months ago, which I since then used to test all betas.

    It's the same thing, no modification done to the project copy since the very first performance test, to make sure the tests are meaningful.

    In order to run the test, I have to build a player and start the generated .exe file, which then spits out a .csv file once it's done.
     
    hippocoder likes this.
  11. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    I was investigating the issue and noticed after I profiled b3 last time, I turned off various PostProcessScene steps in the project, to check if it affects the "Destroying a Prefab..." error and I didn't turn those on again.

    Sooo... the huge performance drop from my previous b6 profiling session can be seen as wrong data, because it was measuring the performance without any optimization steps, thus significantly slower than before and not a meaningful comparison to earlier profiling results.

    However, since PostProcessScene causes some issues here, I ran those optimization steps outside "build player" to have the proper optimized scenes and then did a build afterwards.

    The results are much more like Unity 2018.2 now, see below. 2018.3 is still slower than 2018.1 with my project though.

    2018_3_0b6_scene_4_3.png
    2018_3_0b6_scene_5_4.png
    2018_3_0b6_scene_6_8.png
     
    Lex4art, XCO, LeonhardP and 2 others like this.
  12. Feelnside

    Feelnside

    Joined:
    Sep 30, 2016
    Posts:
    83
    Thank you for the info and at the same time it's quite sad to see such results. There is a new version of PhysX which is quite faster but we have the same performance... I hope it's due to the beta version and the final one will be more better.
     
  13. KospY

    KospY

    Joined:
    May 12, 2014
    Posts:
    153
    Yeah it's pretty underwhelming. I had hoped that PhysX 3.4 will boost physic performance but after some tests it's far from the case and it's even worst. Going back to 2018.2 right now :(
     
  14. Alcyone

    Alcyone

    Joined:
    Nov 15, 2013
    Posts:
    13
    Same here, a VR experience was runing at 72FPS in 2018.2.14, and around 65FPS (noticeable frame drops) on 2018.3.0b9 (Still trying to find what is happening... no luck)
     
  15. buFFalo94

    buFFalo94

    Joined:
    Sep 14, 2015
    Posts:
    273
    @Peter77 can you please make your test with 2019.1 alpha?
     
  16. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    I guess this depends a lot on what kind of physics your game relies, in coming from unity 4.7 in 2018.2 I had to reduce the timestep to 1.5 for a joint based tank vehicle not to have explosive physics. In the experimental 2018.2 eidtor using 3.3physx and 2018.3 using 3.4 I can double the dimestep without having noticeable issues hence the cpu stress release, and I could even push higher.
     
  17. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    I tested Unity 2018.3.0b11 today:

    2018_3_0b11_scene_4_3.png

    2018_3_0b11_scene_5_4.png

    2018_3_0b11_scene_6_8.png



    Looking at the output_log.txt player log file, also contains some issues:
    Code (CSharp):
    1. WARNING: Shader Unsupported: 'Legacy Shaders/Diffuse' - Pass 'Meta' has no vertex shader

    Code (CSharp):
    1. Gfx command not handled: -1744732208 (Last command: 10018)
    2. (Filename: C:\buildslave\unity\build\Runtime/GfxDevice/threaded/GfxDeviceWorker.cpp Line: 2489)
    3.  
    4. Gfx command not handled: 493 (Last command: -1744732208)
    5. (Filename: C:\buildslave\unity\build\Runtime/GfxDevice/threaded/GfxDeviceWorker.cpp Line: 2489)
    Is the "Gfx command not handled" a known issue? The error was printed only twice and the game looked fine to me, no idea what's wrong.


    Code (CSharp):
    1. Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
    2. 1/6 scenes use different skyboxes.
    3.  
    4. (Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
    5.  
    6. Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
    7. 1/7 scenes use different skyboxes.
    8.  
    9. (Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
    10.  
    11. Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
    12. 1/8 scenes use different skyboxes.
    13.  
    14. (Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
    15.  
    16. Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
    17. 1/9 scenes use different skyboxes.
    18.  
    19. (Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
    20.  
    21. Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
    22. 1/10 scenes use different skyboxes.
    23.  
    24. (Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
    I reported the "inconsistent Lighting settings" issue during 2018.2 beta, seems it's not fixed yet.
    https://forum.unity.com/threads/war...up-has-inconsistent-lighting-settings.527997/
     
  18. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,132
    Peter77 likes this.
  19. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Did this get reported with a reproduction case? These error messages are an indication of something wrong in the internal graphics command queue and it would be good to be able to reproduce it and debug it and fix it
     
    Peter77 and optimise like this.
  20. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    I didn't report it yet. It's a more complicated case to get a reproduce for it, outside the actual project.
     
  21. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    How can I help with this. I would like to be able to debug the issue to understand it?
    Could potentially make a new build with a hard crash for those error messages and then you can recreate and send the .dmp file over to me for debugging?
    or ideally get the reproduction project here inside Unity to work from
     
  22. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    If you provide a custom build with a hard crash, I can definitely run this. Creating a (small) project to reproduce the issue, is most likely something I don't find time for soon'ish.
     
  23. Chaz32621

    Chaz32621

    Joined:
    Apr 17, 2015
    Posts:
    199
    @Peter77 awesome with the performance overview love seeing this
     
    Peter77 likes this.
  24. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    I tried the whole day to create a small'ish reproduce, but I failed. I would appreciate if you can make this custom build with a hard crash, this would bring us forward.
     
  25. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Ok let me work on that. Just to let you know the player crash is quite well understood.
    The logs in the player about "Gfx command not handled:" are the unknown piece.

    Thanks

    Jake
     
    Last edited: Dec 11, 2018
    Peter77 likes this.