Search Unity

[UserError] How to find your infinite loop

Discussion in 'Windows' started by AVOlight, Jan 8, 2016.

  1. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427

    built and tested on 10586, just tested the same build on 10240 and the speed of the game play is a lot faster.

    anything known about this?
    for now i'll just change the min version for the next update,
     
    Last edited: Jan 11, 2016
  2. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,735
    Would be interesting to profile the game using Visual Studio profiler and see, where's the difference.
     
  3. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Is this purely a timescale issue, or a frame rate issue?
     
  4. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    don't think its frame rate,
    need to test with better conditions, just noticed it when i tried it on 2 other machines that have different hardware and different windows builds so,
    will have to enable hyper-v tomorrow and try out on the emulators.

    my build machine and one other laptop both running 10586 ran the same so i just pointed at the odd man out which was running 10240

    any tips for finding the issue?
    ---
    well runs normal in the 10240 emulator so... guess that's not it
     
    Last edited: Jan 8, 2016
  5. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    while debugging an infinite loop issue i've run into experiencing what a staggered frame rate is like while logging a bunch of stuff

    didn't realize that could speed up the playback of fixedupdate

    can waitforseconds cause a infinite loop type response from the editor (e.g. editor freeze and its cpu to around 15% and memory slowly climbing)
    Code (CSharp):
    1. Debug.Log("Normal: wait");
    2. yield return new WaitForSeconds(Random.Range(0.1f, 2f));
    3. agent.SetDestination(check[Random.Range(0, trace.count)]);
    "Normal: wait" is the last logged statement before editor freezes
    and SetDestination logs its own name before executing anything

    i have lots of waitforseconds so might be because this is nested in its own method called from a
    coroutine method.. so confusing
     
    Last edited: Jan 10, 2016
  6. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Are you sure it's hanging on WaitForSeconds? I'd say that would be pretty unlikely.

    Attaching a debugger and breaking would tell you exactly where it's stuck.
     
  7. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    if i could tell visual studio to break just before a hang i would (if there's a way to do this please tell me)
    every while loop in the project is logged (and has safety counters just because of trying to find this)

    running through thousands of cycles of debugging would drive me more crazy
    so decided to re-write the ai avoiding coroutines and nested enumerators
    need to find a more manageable way of doing ai anyway
     
  8. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,918
    Why not break during the hang?
     
    AVOlight likes this.
  9. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    thank you for the tip
    when i tried that i didn't get expected behavior so i just thought it wasn't supported

    for example after breaking getting an error, and clicking continue it doesn't continue
    and i get "No compatible code running"
     
  10. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    and then when i try to step, "Unable to step. Array subscript out of range."

    I didn't read this the first time i checked (under the title "No compatible code running")

    "The selected debug engine does not support any code executing on the current thread (e.g. only native runtime code is executing"
     
  11. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    So is breaking on a hang supported? and in my case its just because i'm breaking on unity engine code?

    i have no native dlls in my project
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    It's definitely supported. Are you using visual studio to break inside? What does the callstack look like?

    Alternatively, if there's no code running and no callstack available, break using a native debugger, and you should see the native callstack.
     
    AVOlight likes this.
  13. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    That's awesome! Thank You,
    Yes, Visual Studio

    how do i break using a native debugger ?
     
  14. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Instead of doing "Debug" -> "Attach Unity Debugger", do "Debug" -> "Attach to Process" -> select Unity.exe.
     
    AVOlight likes this.
  15. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    Code (CSharp):
    1. >    ntdll.dll!00007fff37b704c2()    Unknown
    2.      mono.dll!free(void * pBlock) Line 51    C
    3.      mono.dll!g_queue_pop_head(GQueue * queue) Line 48    C
    4.      mono.dll!end_runtime_invoke(_MonoProfiler * prof, _MonoMethod * method) Line 3336    C
    5.      mono.dll!mono_profiler_method_end_invoke(_MonoMethod * method) Line 440    C
    6.      mono.dll!mono_runtime_invoke(_MonoMethod * method, void * obj, void * * params, MonoObject * * exc) Line 2629    C
    7.      Unity.exe!scripting_method_invoke(struct ScriptingMethodMono,struct MonoObject *,struct ScriptingArguments &,struct MonoException * *)    Unknown
    8.      Unity.exe!ScriptingInvocation::Invoke(struct MonoException * *,bool)    Unknown
    9.      Unity.exe!Coroutine::ProcessCoroutineCurrent(void)    Unknown
    10.      Unity.exe!Coroutine::Run(void)    Unknown
    11.      Unity.exe!MonoBehaviour::CreateCoroutine(struct MonoObject *,struct ScriptingMethodMono)    Unknown
    12.      Unity.exe!MonoBehaviour::StartCoroutineManaged2(struct MonoObject *)    Unknown
    13.      Unity.exe!MonoBehaviour_CUSTOM_StartCoroutine_Auto(struct ReadOnlyScriptingObjectOfType<class MonoBehaviour>,struct MonoObject *)    Unknown
    14.      00000000237d4e85()    Unknown
    15.      0000000026b77400()    Unknown
    16.      00000000214aa348()    Unknown
    17.      00000000221f28d0()    Unknown
    18.      0000000024de8260()    Unknown
    19.      00000000318e2050()    Unknown
    20.      0000000026b77400()    Unknown
    21.      0000000004c40000()    Unknown
    22.      000000000c8365a0()    Unknown
    23.      000000000c8313f0()    Unknown
    24.      00000000318e20a0()    Unknown
    25.      0100000024de8260()    Unknown
    26.      0000000024de8260()    Unknown
    27.      00000000005fdc10()    Unknown
    28.      00000000005fdb70()    Unknown
    29.      00000000005fe340()    Unknown
    30.      0000000004f367f8()    Unknown
    31.      00000000005fe250()    Unknown
    32.  
    another one
    Code (CSharp):
    1. >    mono.dll!GC_mark_from(GC_ms_entry * mark_stack_top, GC_ms_entry * mark_stack, GC_ms_entry * mark_stack_limit) Line 761    C
    2.      mono.dll!GC_mark_some_inner(...) Line 378    C
    3.      mono.dll!GC_mark_some(char * cold_gc_frame) Line 478    C
    4.      mono.dll!GC_stopped_mark(int(*)() stop_func) Line 607    C
    5.      mono.dll!GC_try_to_collect_inner(int(*)() stop_func) Line 446    C
    6.      mono.dll!GC_collect_or_expand(unsigned __int64 needed_blocks, int ignore_off_page) Line 1110    C
    7.      mono.dll!GC_alloc_large(unsigned __int64 lw, int k, unsigned int flags) Line 60    C
    8.      mono.dll!GC_generic_malloc(unsigned __int64 lb, int k) Line 204    C
    9.      mono.dll!GC_malloc(unsigned __int64 lb) Line 311    C
    10.      mono.dll!mono_array_new_specific(MonoVTable * vtable, unsigned int n) Line 4422    C
    11.      000000000cdf0ad6()    Unknown
    12.      000000000d51f480()    Unknown
    13.      00000000005fdaf0()    Unknown
    14.      0000000023d1f2c0()    Unknown
    15.      mono.dll!mono_object_new_specific(MonoVTable * vtable) Line 3978    C
    16.      000000000d51f480()    Unknown
    17.      00000000005fdaf0()    Unknown
    18.      00000000005fda50()    Unknown
    19.      00000000005fe340()    Unknown
    20.      000000000c6b67f8()    Unknown
    21.      00000000005fe250()    Unknown
    22.  
    then exception on mono_security_core_clr_is_platform_image
     
    Last edited: Jan 10, 2016
  16. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    In the watch window, type this:

    Code (csharp):
    1. mono.dll!mono_pmip((void*)0x0d51f480);
    Where the hex number is the address in the callstack without a name. It will give you a managed function name. Start looking at the unknown frames from the top.
     
    AVOlight likes this.
  17. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    um, step 1 quickwatch expression: "mono.dll!mono_pmip"
    click add watch
    and now there's a watch 1 panel, with the name mono.dll!mono_pmip, and value 0x00....

    then what do i do?
     
  18. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    You need to type address as an argument to that function.
     
  19. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    and then
     
    Last edited: Jan 11, 2016
  20. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    AVOlight likes this.
  21. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    right so i've used it to see what the function names of stuff without names are, some return null some don't
    but what am i looking for?

    the call stack changes as i pause and continue
     
  22. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    You're looking for one of your function names. The higher up the stack you find it, the accuracy of finding your infinite loop will be more precise.
     
    AVOlight likes this.
  23. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    @Tautvydas Zilys THANK YOU!!! :D

    knowing this is awesome
    thank you for taking the time to explain it

    my ai kept on trying to trace a route
    nested coroutines with while loops can be hard to manage
     
    Last edited: Jan 11, 2016
  24. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    Sooo what if I do all that, and instead of a function name coming out it just says "No type information is available for the function being called. If you are calling a function from another module, please qualify the function name with the name of the module containing it"?

    http://puu.sh/pGFYS/806a338b43.png

    I'm seriously at a loss as to what's going on. I hope someone still reads this.

    edit: Further testing reveals ntWaitForSingleObject to be kind of halting everything?

    http://puu.sh/pGGD2/8542211a41.png

    edit2: it's stuck on an overhead?

    http://puu.sh/pGWli/d1f1258c06.png
     
    Last edited: Jun 26, 2016
  25. AVOlight

    AVOlight

    Joined:
    Apr 15, 2014
    Posts:
    427
    try this method

    for me it pointed one below the problem method, (because the null that broke the loop happened there, which i didn't fully catch onto at first as i was mindlessly following the instructions)
    very, very, useful
     
  26. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    It's very likely you're looking at a wrong thread. Load symbols for Blacktop.exe, that should make finding the main thread easier.
     
  27. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    I've been looking for about 10 minutes, can't find a single CMP instruction even though all my debug boxes had been ticked while building.

    Done, but now I don't have any pieces of unknown code anymore:

    http://puu.sh/pHLXO/0e71b2bdd6.png
     
  28. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    That is indeed the wrong thread. You're looking at an audio thread. You'll need to switch to the main thread. That's also the reason why you cannot find "cmp" instructions.
     
  29. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    Ooooh I thought you meant forum thread. Right, how do I switch to the main thread?
     
  30. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    First, you need to locate it. Click Debug -> Windows -> Parallel Stacks. It should show you all thread view. The one with "PlayerLoop" function on the callstack will be the main thread.
     
  31. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    Right, found it.

    http://puu.sh/pI0qv/63449da720.png

    I put a breaking point at the start of the PlayerLoop, let the hang run for a while, but it never reaches the PlayerLoop. What's next?
     
  32. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Looks like it's not an infinite loop it's waiting for! Looks like a deadlock to me. Can you press Debug -> Save dump as and PM the resulting file? Which Unity version are you on?
     
  33. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    :eek:

    Yeah sure, I made a dump file and I'm uploading to mega as we speak. (It's half a GB).

    I'm using 5.4.0B20

    I'll start a conversation when it's done uploading.

    Edit: started a conversation, hope you can figure it out!
     
    Last edited: Jun 27, 2016
  34. JakeTurner

    JakeTurner

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    137
    Can you recreate/repeat the deadlock?

    Also when the deadlock happened was profiling enabled?
     
    Last edited: Jun 29, 2016
  35. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    Yup it's very easy to repeat: I Just have to boot the game a couple of times, and eventually it locks. If it doesn't after 15 seconds I try again. One in ~five times it hangs.

    Profiling was enabled, but I believe it also happens Without any debugging enabled. I'll Have to double check to be certain about that though.

    Should I send the entire project?
     
  36. xzbobzx

    xzbobzx

    Joined:
    Apr 7, 2015
    Posts:
    29
    Yup, freeze also happens in the regular, non development build.