Search Unity

Game rendering freezes randomly at runtime

Discussion in 'Windows' started by ex_mi, Jan 25, 2016.

  1. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Is the bug confirmed in the beta builds also 5.5?
     
  2. axcs

    axcs

    Joined:
    Jan 6, 2015
    Posts:
    17
    I trie on last beta 5.5, and the bug is still there :(
     
  3. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
  4. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Hi

    I have just run the re-factored (cross platform tidy) code on windows phone again and the crash for me is still fixed. This is based on a build from 5.4.2p3.

    Now need to get the code review process to complete and then I can merge it back into 5.3 & 5.4. The code review process should hopefully be done this week.

    Jake
     
    VladimirTa likes this.
  5. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    Hi @Jake,
    Thanks for the update, but do you have any idea about these crashes caused by the fix?
     
  6. axcs

    axcs

    Joined:
    Jan 6, 2015
    Posts:
    17
    I re-tested my game, but in versions 5.4.2f1 and 5.4.2f2 the "PlayerPrefs" every time you exit the game resets all saves.
    The only version that does not make me that effect is 5.4.2p3.
    For this reason I have tried to apply UnitySetup-Metro-Support-for-Editor-5.4.2f1 (Fix for dual core), but every time I compile the game, it displays errors or crache.
    It does not seem to be compatible. It is possible to perform an experimental builds for version 5.4.2p3.
    I do not understand why the other versions when leaving the game erase me from the data of "PlayerPrefs" :(

    (Seems to be this problem: https://forum.unity3d.com/threads/bug-playerprefs-are-deleted-when-i-restart-my-app.433071/ ) What is going on with UWP games?
     
    Last edited: Nov 8, 2016
    MrEsquire likes this.
  7. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    I am cooking up builds using the final fix (pending code review process) for
    5.4.2f2
    5.4.2p2
    5.3.6p8

    Might be ready later today but more realistically overnight.
     
    VladimirTa and axcs like this.
  8. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    Also Unity 5.4.2p3 please
     
  9. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    5.4.2p3 better to do than p2
     
  10. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Sorry to say I don't have any ideas. I am not a windows phone expert and I don't know what those errors mean (I find it a struggle just to get windows phone development to work / debug - it seems very tough).

    I am just fixing this one problem of hangs (in the job system) when running on hardware with 2 cores or less on UWP or Windows Phones.
     
  11. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    Hi,
    But these errors appeared just after I applied your fix. Also I received lot of negative feedbacks and reviews from my users saying that the game is crashing ( just after I updated my game to Unity 5.4.1f1+your fix).
    Please to double check these errors with your Windows team or with Microsoft guys.
    Now I rolled back my game to Unity 5.3.3p2 and these errors and crashes are disappeared.
     
  12. axcs

    axcs

    Joined:
    Jan 6, 2015
    Posts:
    17
    Thanks for your work.
    When you finish performing these builds, please send me the Links when you ends, for my DM, it can be anytime or at night!
    With these builds, it will help me in a way! Thanks a lot!
     
  13. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Ok some builds are ready. Start a conversation with me if you want to try out a build for:
    • 5.4.2f2
    • 5.4.2p2
    5.4.2p3 & 5.3.6p8 are still cooking.
     
  14. axcs

    axcs

    Joined:
    Jan 6, 2015
    Posts:
    17

    I tested 5.4.2p2 in my game and it Works, no freezing game on Dual Core CPU, and the PlayerPrefs works fine and save all progress.

    I recommend everyone to try this build: 5.4.2p2 with JakeTurner fix

    Thanks a lot, You save me!
     
  15. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Let us know when 5.4.2p3 cooked and I will DM.. cheers
     
  16. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Hi

    I now have builds for these versions:
    • 5.4.2f2
    • 5.4.2p2
    • 5.4.2p3
    • 5.3.6p8
    Please start a conversation with me to get access. All I ask in return is feedback if the test build fixes your issue.

    Just to mention. I only have one 100% reproducible case that I know about for this hang and I have tested that case and debugged it and it is fixed by these changes. To make any more progress on hangs/crashes related to this. I would need:
    • .dmp file + .pdb file to resolve the callstack : just to identify the crash/hang
    • full reproduction case with device details submitted via Unity Bug Reporter : to recreate and debug
    I want to leave this alone for a few days now to let you guys test it.

    The first stage of the code review is completed which means I am unblocked to start the next (major) stage today. After that is completed there will be 1 or 2 more code review stages to go thru before it lands in a patch release for 5.5./5.4/5.3.

    Thanks

    Jake
     
    VladimirTa and axcs like this.
  17. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Tested 5.3.6p8 and haven't run into any trouble so far! :) Will continue testing ...
     
  18. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Tried 5.4.2p3 and excellent build.
    No issues so far on Universal Build (Phone/Desktop/Xbox)
    Will release game update and see what customers say.
    Thanks for the build..
     
    mr_zog likes this.
  19. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    @MrEsquire please keep us updated regarding the release!
     
  20. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    More updates. The code review process is going well and I have started discussions with the release managers about landing this change in a 5.3/5.4/5.5 patch release.

    Here is a bit more info on what the bug was/is and what the fix is.

    The Unity job system chooses how many threads to use for running jobs based on the available number of cores, now on hardware with 2 cores or less this ends up with zero worker threads being assigned to the job system. The job system inherently requires (assumes) there is at least 1 job worker thread. The hangs are caused by the job system waiting on another thread to send it a signal to continue but there are no other job worker threads so nothing can send the signal. The code fix is to remove the job system when there are zero worker threads and just run all jobs in place instead of scheduling them. This is quite a large amount of code change (to make the change tidy and neat and general purpose for all platforms).

    On certain platforms at compile time if we know there will be no job worker threads then we do a slightly similar change. In essence my change was to allow the choice to run without the job system be a runtime choice as well as the existing compile time choice.
     
    mr_zog likes this.
  21. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Thanks for the insight!
     
  22. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Hello again. Does anyone have a project which has quite a lot of content in it (perhaps lots of Unity UI elements?) and would they be willing to try out a different custom build to measure performance on a phone which was previously hanging.

    I want to explore a different change which is much simpler but might have a performance cost.

    I am going to do my own profiling but I don't have a particular realistic real world test for this type of device.
     
  23. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    I'll check with my colleagues
     
  24. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    If you can help please just let me know which Unity version you would need the build on
     
  25. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    OK, I misread ... I thought you require a project.
    Of course we would test a build, 5.3.x would be great!
    (can start testing next week)
     
  26. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Thanks. Should be ready Monday afternoon (UK) time.
     
  27. melle

    melle

    Joined:
    Nov 6, 2016
    Posts:
    16
    I have the same problem on my lumia 520, game freezes at random moments. I am using world space UI, may be the problem?
     
  28. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Yes it sounds very much like it. What Unity version are you using and I can send you a custom build to see if it fixes your problem.
     
  29. melle

    melle

    Joined:
    Nov 6, 2016
    Posts:
    16
    I'm using 5.4.1f1, would really appreciate having this fixed!
    Thanks in advance.

    EDIT: Also tried 5.4.2f2, still happens.
     
    Last edited: Nov 12, 2016
  30. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    Hi Jake,
    I hope the fix will not affect negativly the performance for the low end WP devices (like Lumia 520 and 620).
    I'll see if I can test your fix before monday using my Lumia 920 and I'll let you know.
     
  31. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Thanks what Unity version do you need a build for? This is a new build so I am have to make them specially and yes the low end devices is my concern. I am going to profile on a 520
     
    Last edited: Nov 12, 2016
  32. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    I'll test using Unity 5.4.2p3
     
  33. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Hi,

    I gave it some testing time and didn't notice any performance issues so far.
    At this point I must say, during gameplay, the frames tend to be between 30 and 60 FPS on some devices ... so I am not quite sure what to look for (e.g. 10-20% less perfo wouldn't be "measurable" easily during play, static FPS before the level starts seem to be the same).

    Also, our game isn't really "ui heavy".

    Could you outline, where performance drops could happen and how bad?
    Permanent or temporarily when something happens in the job system or what ever?

    I will also do some more long-term stability tests.

    Thanks!

    PS: If the old patch has nearly passed code review, why consider replacing it with a easier to implement, but possibly performance-killing version?
     
  34. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    Hi,
    I performed a quick test with my Lumia 620, the crashes are completely disappeared but I'm still noticing some hangs, the sound of my game hangs for 1/2 second and return to the normal.
     
  35. PrettyFlyGames

    PrettyFlyGames

    Joined:
    Aug 30, 2012
    Posts:
    74
    Hey there!

    I've tested the special build for Unity 5.4.2f2 and it works great. No more hangs :)

    Great work Jake!
     
  36. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Hi Jake,

    I am not yet sure if it is related to the update, but this occurs on my Windows 8.1 Phones only when they are not connected to the internet.
    I currently think it's gameanalytics related (also new in the build -- will test a build without and report back!), but the callstack looks suspiciously UI related:

    System.Runtime.InteropServices.SEHException was unhandled
    Message: An unhandled exception of type 'System.Runtime.InteropServices.SEHException' occurred in UnityEngineProxy.ni.DLL
    Additional information: External component has thrown an exception.

    If I locate the proxy.DLL, it points me to:
    AutoGeneratedDelegates.cs
    Which I don't have.

    Callstack:
    > UnityEngineProxy.ni.DLL!UnityEngineProxy.InternalCalls.RectOffset_CUSTOM_Cleanup(object self) Unknown
    UnityEngine.ni.DLL!UnityEngine.RectOffset.~RectOffset() Unknown


    I have to let the game run about 15-20 minutes, offline, until it occurs.

    EDIT: It also crashed on one Win 10 phone so far, I am also not yet to tell if it's really neccessary to be offline ... it'd be good if you could tell if the crash looks like it's near the UI stuff you meant.
     
    Last edited: Nov 15, 2016
  37. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    ^ That looks like an unrelated issue. Could you make a bug report on it?
     
  38. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Thanks for the insight. I was able to reproduce the bug on our last known stable Windows Phone 8.1 Unity version (5.3.3p2), so it's not patch related.
    The first occurance came after 45 minutes offline play and then subsequently comes very frequent (<5mins).
    Am working on trying to isolate the issue, I still think it' related to the GameAnalytics-plugin ... not sure if that makes sense, but it's a straw I can hold on to for now ;)
    I will PM you as soon as I got the project ready, to keep this thread clean.
     
  39. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    For the sake of completeness: We will submit an app update for W/P8.1 and W/P10 tomorrow using 5.3.7f1 + the "simpler but maybe ui-costy"-patch from Jake. I will keep you posted regarding our error reports.

    My previous error above seems to have been caused by GameAnalytics when offline (SQLite threading). I got a better callstack on a Win10 phone ... and we now kicked it out again and the crashes disappeared.
     
  40. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    You have to write @JakeTurner a private message to receive the patched version. AFAIK, it's not through official QA yet.
     
  41. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    The code review of the old patch raised some very valid questions and the new simpler version was at the request/reaction of the code review process. I don't think the original patch code change will pass code review. The changes don't affect the end user but the code implementation will need some minor refactoring to move some of the changes from one place to a different place/name.

    The new simpler version will be a much easier & quicker change to get code reviewed.
     
    VladimirTa likes this.
  42. matt33315

    matt33315

    Joined:
    Mar 28, 2014
    Posts:
    94
    Hi Jake, Would this effect the time line / delivery date of the patch? if you now chose to go with the new simpler version now?
     
  43. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Hi

    Sorry for the delay have been away for a week onboarding the new starters in the Sustained Engineering team (which is the team I am part of).

    I am going to cook a 5.4.3f1 custom build today, please send me private message if you like to test it out.

    I have done local profiling and I can't find any significant difference with the new simpler fix and a stock version of Unity without the fix. The code review for the simpler fix is very nearly completed. The hardest part of the change has been approved and the remaining item should be approved very quickly.

    I am hoping to get this landed into the 5.4/5.3 patch branches by the end of the week ready for 5.3.7p3 and 5.4.3p3. There is a very slim chance the change might land in 5.4.3p2.

    Can anyone who tried out the simpler custom builds report any performance change when using it?

    Thanks

    Jake
     
  44. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    The CPU cycles have to come from somewhere e.g. main thread, render thread or a job thread. The potential extra cost is just in changing threads. The UI system is the heaviest user of the job system on these devices which is why I mentioned UI intensive games. In my tests I struggled to make the job thread CPU cost be a dominant cost, all the other threads cost more than it whenever I scaled up the complexity of the scene.

    I would expect a very small impact (<1%) at most and predict no measurable impact.

    The cost would probably be a fixed increase not a transient varying change but it would depend on precisely on what is going on in the scene and could vary from scene to scene e.g. very large number of UI elements.

    Thanks for the report about performance. I missed this message on my travels.

    Jake
     
  45. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    The 5.4.3f1 build is ready just private message me if you want to try it out

    Jake
     
  46. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    @Tautvydas-Zilys I filed a bug report regarding the RectOffset crash. The crash doesnt seem to be new, but occurs more often now (could be biased, only got data for 2 days). https://fogbugz.unity3d.com/default.asp?854590_iv7ljmbt0q3jj9i2

    @JakeTurner So far, customer support has not mentioned any more issues than usual -- and especially, no freeze bugs have been reported since the update. We also have done some more playtesting, and a "freeze/infiniteloop-bug" occured only once. I have no idea how to reproduce or what it's related to ... the frequency is too low for now.
    We will continue testing, but if it stays that rare, we can live with it I guess.
     
  47. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Bit more update. The code review process for the change is completed and now the change is in the queue to be merged to trunk. I am preparing the backports in parallel. At the moment still on track for landing the change in 5.3.7p3 and 5.4.3p3.
     
  48. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Great news. If you get a freeze/hang again try and get a coredump just so we can explore it.
     
  49. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    I'll see if it's possible, most of the time, the DMP file contains "an error occured while creating the dmp file" ;)
     
  50. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Good news the fix has landed in the 5.3 & 5.4 patch branches (it has also landed in trunk).

    All being well the fix will be released to: 5.3.7p3 & 5.4.3p3.

    Thanks for all of your support and help to test and diagnose this problem. It has been a long journey.

    Any more problems please let me know.

    Thanks

    Jake
     
    VladimirTa and MrEsquire like this.