Search Unity

Unity 5 - Profiler shows physics spikes on Android device.

Discussion in 'Physics' started by kimcgm, Apr 8, 2015.

  1. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    that sounds great :) I'd like know more if you find any other issues or anomalies.
    The "Gfx.WaitForPresent" means that the renderer is waiting for the Gpu to be ready for the next set of commands. It's another sync pipeline so I wouldn't worry about that at the moment.
     
  2. Kubold

    Kubold

    Joined:
    May 10, 2012
    Posts:
    359
    I hope this change will go to Unity 5.3 or some sooner official version. I still wonder if this spikes were a problem of just a few people from this thread, or a global one, but nobody cared to talk about it?

    Did you manage to reproduce it at Unity?
     
    Stormy102 likes this.
  3. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    I'm pretty confident this fix will be backported.We always need to put the latest changes into the development code base in order persist the change. Then we can move the changes to other versions in a stable fashion.

    The problem is tricky to diagnose because the spikes are cause by a *lack* of work, not too much work. I.e. the time was spent of the physical synchronization and prioritation of execution on the cpus. Finding reliable cases where the problem occurs can be tricky, but I think the VR team found a reproduction for it. They are also the ones who fixes.
     
  4. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    Thx Morten for passing the link
    Finally I was able to downloaded the experimental Unity build and I make same test.
    Spikes are shown in Rendering and no spikes are shown in Physics. Passing the problem from Physics to Rendering. Same as
    Kubold in the last screenshot.

    I make test also with one video card only and one monitor with same result.
    I take out
    all from the scene, including: light, camera, ambient light, sky-box, etc. And the spikes (80FPS) are still there.

    (1)Looking in the memory section of the Profiler, I found that "
    Mesh" numbers are constant until the spikes.
    In detail :
    Meshes pas form 28 / 500 KB to 118 / 0.8 MB (only in the spikes)
    Total Objects in Scene from 96 to 186 (only in the spikes)
    Total Object Count from 1896 to 1986 (only in the spikes)

    Why there is an incremental of Mesh number and objects in the spikes?
    What are all this objects for?
    Are this spikes made by the Profiler itself?

    In the Profiler is missing an essential line for debugging:
    the "FPS Timeline" itself. Line that explicit shown the fps or ms (shown in Timeline).

    (2)There is also a second fps spike that occurs each time the Total GC Allocated Mono pas from having 6,7MB to 5.1MB

    In conclusion i think that:
    Some times the Total GC Allocated Mono spike(1) combines with the incremental Meshes or Object(2) and produce a bigger Spickes 28ms 37FPS, stopping the smoothness movement of the physics objects.

    Sceenshot.jpg
     
    Last edited: Nov 19, 2015
  5. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    Hi @AlanMattano

    The spikes shown here are not a problem, because they are smaller than 1 millisecond per frame. That is not a concern in this context.

    In your picture you have currently pressed the bottom graph, showing "Memory". If you press the top graph, "CPU Usage", you can both get the timeline view. I think you might be able to see who created the Meshes or at least what their names are.

    The garbage collector does cause spikes, but you can see in the "CPU Usage" view who creates the GC allocations.

    I cannot see your 80ms spikes in this picture?
     
  6. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    View attachment 162634
    Hi @MortenSkaaning,thanks for the help.
    When garbage collector spikes (red line) matches with the Mesh Memory spikes(line with plateaus) then a big spike of 30ms occurs. Forgive my ignorance, 30ms( 34 FPS ) spikes can be a concern when is an empty scene.
    Sin título-1.jpg

     
    Last edited: Nov 19, 2015
  7. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    hi, I agree that 30ms spikes would be a problem. But I can't see them on your pictures?
    I agree with your observation on mesh memory and GC.

    Are you running with v-sync on? Are you using the experimental 5.2.2 build I posted?
    It might be that the gpu for some reason needs to wait for something.
     
    Stormy102 likes this.
  8. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    In that picture is 22ms (45FPS)spikes. My reference for all my ms info spikes are the value shown at the bottom right of the Timeliner,(correct me if i'm wrong) that is the same rough value I get via my custom script shown earlier in this post and included down here.

    The screenshot showing 30ms spikes as mention was not included.(the spike was at the combination) (In my laptop pc spikes are 50ms. vsync off normal build 5.2.2f1. )


    Code (CSharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. public class DebugFPSmin : MonoBehaviour
    6. {
    7.     [Tooltip("Minimum amount of frame speed in FPS (Frames Per Second) that will trigger the screen gui value")]
    8.     public float
    9.         FpsSpikesLimite = 60f;
    10.  
    11.     private float minimumFrameTime, FpsSpike, FpsMinSpike = 1000000f;
    12.    
    13.     void Start ()
    14.     {
    15.         minimumFrameTime = 1f / FpsSpikesLimite;
    16.     }
    17.  
    18.     void Update ()
    19.     {
    20.         if (Time.deltaTime >= minimumFrameTime) {
    21.  
    22.             FpsSpike = 1f / Time.deltaTime;
    23.  
    24.             if (FpsSpike < FpsMinSpike)
    25.                 FpsMinSpike = FpsSpike;
    26.         }
    27.     }
    28.  
    29.     void OnGUI ()
    30.     {
    31.         GUILayout.Label ("FPS spikes:" + FpsSpike);
    32.         GUILayout.Label ("Min spikes:" + FpsMinSpike);
    33.  
    34.         if (GUILayout.Button ("RESET MIN"))
    35.             FpsMinSpike = 1f / Time.deltaTime;
    36.     }
    37. }



    OFF. This last screenshot is v-sync off. Windows 10

    My test on and off getting similar results. In case I go second blank (30fps)V sync ON it get worst. I also try to change via hardware and make combinations getting similar results.


    Yes, experimental 5.2.2 build as you ask.

    I also experiment with 5.2.2F1, experimental 5.2.2 build, 5.3.0F1 and now I return back back to 5.2.3F1.
    All similar problems. I notice this problem since earlier version of Unity4 because I consider a the minimum fps as the value to look in my games. But other users since Unity3.5.6. I think is an old problem amplified in Unity5 rendering system especially profiler ON. In Unity 5 with the introduction of the free profiler the irregularity in fps was more explicit, visible to the large user.

    How much takes the profiler to the fps in general?

    things i notice are:

    • v-sync on or off is the same problem
    • v-sync is on, is not making 60 fps or 30fps constant and perfectly.(in editor)
    • v-sync when is half on (30fps) the spicks are 40ms(24fps) in GPU i7, CPU R7970, 5.2.3f1
    I try to resume in detail because i do not have much time.
    • In a good pc i7 R7970 software empty and v-sync off or on. 30ms spikes using experimental 5.2.2 build
    • In a good pc i7 R7970x2 and 2 monitors software empty and v-sync off or on. 30ms spikes
    • In my old lptop i3 Windows7 using normal build 5.2.2 the spikes are50ms (20FPS) in a same empty
    • spicks since earlier version of Unity4, Windows Xp and other laptop.
    The problem is not the value itself. Is the proportion of "Other" (Mesh?) wait for present eating frames time and the difference between a fastest frame and the slowest, that affects physics visual movement. I think now this is a rendering issue. Do I need to go to other trade to follow this?
     
    Last edited: Nov 21, 2015
  9. BalazsBalazs

    BalazsBalazs

    Joined:
    Oct 7, 2013
    Posts:
    8
    The Unity 5.2.3f1 is totally solved my physics spikes.. now everything is smooth and perfect. :)
    Win 7 (64) Standalone build and editor
     
  10. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    Alan: Can you get 60fps in any simple scene? I.e. with one or two simulated physics objects, in the 5.2.2 experimental build
     
  11. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    Yes, as shown here in screenshot(1), I get 2100 fps indicated in the Game Statics FPS Graphics indicator and looking at the Profiler Timeline more than 200fps, in empty scene or with tree simulated physics objects, in the 5.2.2 experimental build that you gave us.

    @MortenSkaaning : screenshot number 2 and 3 show the magnitude of the spikes in empty scene (pix 2) or with tree simulated physics objects (pix 3), in the 5.2.2 experimental build using i7 and R7970 VSync off.

    I agree BalazsBalazs. It solved my physics spikes but not the rendering spikes.

    @BalazsBalazs can you show a screenshot of the Profiler "GPU Usage"?

    FramesPerSecond pix1.jpg Frames Per Second pix2.jpg Frames Per Second pix3.jpg
     
  12. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    if it performs fine on non-empty scenes then I don't really think it will stop you from shipping games/apps.

    I don't really know a lot about rendering so I can't help you with that. But, just to be sure, I'm making a new experimental build where we can see everything the job system is doing. So if any job is stalling for any reason we can see it.
    I'll post it when it's ready.
     
  13. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    That's great! Thanks @MortenSkaaning also for the great work done! I stay available.
     
  14. BalazsBalazs

    BalazsBalazs

    Joined:
    Oct 7, 2013
    Posts:
    8
    @AlanMattano
    The image is with default good quality settings. Everything is smooth and good.
    The spikes looks big, but I think it is because the graphics height is fix, and the largest spike is always have 100% height in the window, so it is just visually big.

     
    Stormy102 likes this.
  15. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    No, I'm not talking about the CPU usage.

    I wish to confirm that i'm not the only one having spikes in rendering that affects the visual motion of the object. If you can, go to the Profiler, press "Add Profiler" then choose "GPU Usage". @BalazsBalazs can you show a screenshot of the Profiler "GPU Usage"?
     
  16. BalazsBalazs

    BalazsBalazs

    Joined:
    Oct 7, 2013
    Posts:
    8
  17. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    Hi, the profiled experimental build is ready: http://beta.unity3d.com/download/d2a0c23311c2/public_download.html

    Maybe it'll help track down the gpu issues you report, otherwise you would have to take it to the graphics forums.

    This micro tutorial also shown some culling work that is not marked on the profiler:
    upload_2015-11-27_6-59-11.png

    Click on the blue box:
    upload_2015-11-27_6-59-28.png

    Hope this can shed some light,
    Morten
     
    mdrotar likes this.
  18. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Hi, When will the upgrade to PhysX 3.3.3 happen, and how big of a job is this.
    We really need more performance on Android, currently it suffers alot..
     
  19. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    We're currently looking at the upgrade in Q1 2016. But we're working on a simpler solution for skipping the physics update.
     
  20. GenOli

    GenOli

    Joined:
    Apr 21, 2013
    Posts:
    139
    Just like to add I'm seeing these spikes in 5.3.1, even in an empty scene...

    Seems Unity is plagued with spikes at the moment, Occlusion Culling...Physics... :confused:
     
    Stardog and MrEsquire like this.
  21. NovelSpinGames

    NovelSpinGames

    Joined:
    Feb 13, 2015
    Posts:
    1
    I am getting physics spikes, even in scenes consisting only of menus, and this is in the editor itself, not on Android. I'm using Unity 5.3.1. Here is a screenshot, where a spike reaches 95.9ms:

    upload_2015-12-21_23-17-43.png
     
  22. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    Hi, I'm not in control of the spike fixes as they affect whole Unity, but I'm raising priority internally.
     
  23. FuzzyQuills

    FuzzyQuills

    Joined:
    Jun 8, 2013
    Posts:
    2,871
    95ms?!
     
  24. savgrace

    savgrace

    Joined:
    Jul 14, 2015
    Posts:
    5
    I can confirm the same issue. It's really bad! Physics spikes reach well over the 33ms (30 FPS) cutoff! Even in an empty scene, or in one of my project scenes when nothing is running. I've tried various things, including turning off all selections in Physics2D and Physics Collision Matrices.

    Please fix this bug soon Unity! :) Merry Christmas
     

    Attached Files:

  25. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    Hi, we've upped the priority and have people working on testing a fix.

    Note: There is no "cutoff" for time, we just have to try to use as little as possible. (unrelated to current problem :))
     
  26. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Do not understand what you mean by skipping official update? Will this not then include the Android performance updates? We mobile developers when we hear anything performance and Android -) must read for us.. Any updates most welcome
     
  27. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    I was being unclear. I was referring to skipping the physics FixedUpdate call, unrelated to upgrading to a new PhysX version.

    Regarding solving the spike issue Sustained Engineering are testing a fix which we're aiming at releasing at end of January.
     
    dreasgrech likes this.
  28. Stardog

    Stardog

    Joined:
    Jun 28, 2010
    Posts:
    1,913
    I just submitted a bug report project for this anyway.
    unity-physics-processing.png
     
    Last edited: Jan 11, 2016
    MutatedSoftware and MrEsquire like this.
  29. MutatedSoftware

    MutatedSoftware

    Joined:
    Dec 4, 2014
    Posts:
    5
    Unfortunately end of January is a bit of a tight deadline for me and I'm married to Unity 5.2.2p3 for the moment. I'd be highly interested to hear of any workarounds if there are any or if the issue is specific to particular host/target operating systems or target platforms.
     
  30. Lucifuges

    Lucifuges

    Joined:
    Jun 21, 2011
    Posts:
    3
    Repro on 5.3.1f1 / Windows 7 SP1 / Intel Core i5 750 (4 core) / GeForce GTX 750 Ti (v347.52) / V-Sync ON
    Phiysics.Processing causes 31.9ms spike in the super simple scene (no cameras, no lights).
    And in more complex scene, It reaches 80-90ms, sometimes stop few seconds.
    Hope this can be solved sooner...

    クリップボード04.jpg
     
    Stardog likes this.
  31. RedRockInn

    RedRockInn

    Joined:
    Jan 10, 2014
    Posts:
    1
    So, NooB hopping in, but I had a similar issue and found a fix that works for me. I use Visual Studio to code and it appears that I get Physics spikes when Visual Studio is open. After closing Visual Studio, Unity runs without any Physics spikes. Not sure if this in any way helps.
     
    Last edited: Feb 4, 2016
  32. Stardog

    Stardog

    Joined:
    Jun 28, 2010
    Posts:
    1,913
    I've also had the strangest fixes with Unity recently, like having a Youtube video open in Chrome smooths out the broken WASD movement. It must be some video driver issue.
     
    MrEsquire likes this.
  33. KingOfColly

    KingOfColly

    Joined:
    May 25, 2013
    Posts:
    15
    Spikes happening to me too Win7 x64, i5-3570K, GTX980 G1 Gaming.

    What I've noticed on my system is it always occurs from system startup. I'll then do a bit of browsing, watch a you tube vid for a while then when I come back to Unity the spikes are gone :confused:
     
  34. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    4,044
    Not sure why but I'm getting this too - usually between 20 and 40ms spikes in nearly empty scenes. 5.3.2f1 It started happening kind of suddenly.
     
  35. KingOfColly

    KingOfColly

    Joined:
    May 25, 2013
    Posts:
    15
    I appear to have resolved my lag spikes. I went into task manager and noticed 25% of my cpu was being hogged constantly by svchost process. I ran windows update then after that chose to upgrade to Windows 10.
     
  36. mpulkkinen

    mpulkkinen

    Joined:
    Apr 8, 2013
    Posts:
    15
    These lag spikes are happening in 5.3.3 as well, disabling all colliders in scene and un-ticking everything in Layer collision matrix didn't change a thing, we're seeing randomly 25ms spikes in Physics.Processing. Profiler indicates that the spike is happening at the same time as EventSystem.Update->Physics.Raycast ( 2.32ms ).
     

    Attached Files:

    Stardog and MrEsquire like this.
  37. larskroll

    larskroll

    Joined:
    Dec 17, 2013
    Posts:
    52
    I'm also seeing some weird spikes in my project.
    Maybe unrelated, because it only seems to happen in rather dense scenes. The weirdness of my spikes is, that it seems certain things simply aren't being shown in the profiler, making it kinda hard to isolate the problem...

    e.g. :I Have a spike on GPU. In the hierarchy, it shows 26.9% on Camera.render, 0.5% on Mashskinning and 0% on rendertexture.setactive. And NOTHING else. So, total of 27.4% of the GPU time accounted for.
    Most frames in that run accounts for 80% of the GPU time.
     
    MrEsquire likes this.
  38. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    I've been wasting days trying to isolate an annoying stutter happening on certain Android devices (Galaxy S3 seems to be the worst) with Unity 5.3 and I'm starting to think the same.

    Actually, I'm starting to think that the profiler might even be attributing time incorrectly. Something is just way wacky with it. I have about 300 objects in the scene, all moving the same consistent way and randomly I'll get a single object with an OnTransformChanged time of 3-6ms when the rest are 0.03ms. A coroutine, which does the exact same thing every frame randomly spikes from 1ms to 6ms (as reported by the 'self ms' column).

    All while this scene is designed to keep everything as constant and simple as possible to isolate the problem. All it is is 5 columns of 62 UI Images each, where each column is moved vertically by some delta each frame. Each column is exactly the same (except a horizontal offset so I can see them all) and should behave exactly the same on each frame, but they don't according to the profiler.

    Other things randomly spike too... Device.Present, Render.OpaqueGeometry, Overhead, and of course Physics even though there is no physics being used. Deep profiling doesn't seem to provide any useful information.

    I'd like to submit this project as a bug report but I don't really know what to write in the bug report... "Something is mega effed, yo"?
     
    MrEsquire likes this.
  39. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
  40. mpulkkinen

    mpulkkinen

    Joined:
    Apr 8, 2013
    Posts:
    15
    Profiler really doesn't give any usefull information regarding this issue, but for me it seems that it's in line what we're seeing on screen: huge lag spikes in game randomly. I'd like to also point out that this issue is with stand-alone windows build & editor, not just android...
     
  41. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    The stuttering issues are general and not related to physics.
    We're been working on a fix for 2 months, but it's highly complicated and that's why it's taken a long time.
    We're targeting the fix to all 5.3 and above versions so you'll get it once is it finished and verified.
     
  42. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Cool, but do you have a rough date when it might be finished and verified..
     
  43. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    I'm being told "post GDC".
     
  44. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    It's sounds like you have an idea of what's causing it. Can you provide any hints as to how we might work around it for now?
     
  45. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    the problem is about how different work elements in the job system notifies other threads about work. There some issues that causes to work notifications to arrive very late in certain thread+work configurations. I do not if there exists a workaround.
     
  46. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    Is there a bug # we could monitor for updates?
     
  47. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
  48. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Bit no related but we spoke about this on the roadmap:

    Physics: Update PhysX to 3.3.3
    • Update to the latest patch release PhysX 3.3.3
    • Better performance on Android expected
    You mentioned this not going to happen and there some other ways?
    What is the current status of this? shall it be removed from roadmap if no plans.
     
  49. MortenSkaaning

    MortenSkaaning

    Joined:
    Jan 16, 2015
    Posts:
    120
    This is not related to the job system issues and these items have not been removed from the roadmap. They are still underway.
     
    MrEsquire likes this.
  50. Para-bellum

    Para-bellum

    Joined:
    Aug 7, 2013
    Posts:
    3
    There is another way to reproduce it without any colliders. Just create Thread with "while" loop.
    private Thread _tr = null;

    private void Start()
    {
    _tr = new Thread(Job);
    _tr.Start();
    }

    private void Job()
    {
    while (true)
    {

    }
    }
     
    MortenSkaaning likes this.