Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Making sense of an il2cpp crash

Discussion in 'iOS and tvOS' started by BJO, Feb 25, 2016.

  1. BJO

    BJO

    Joined:
    Mar 25, 2013
    Posts:
    7
    Hello,

    Our app's most frequent crash has a rather cryptic stack trace, as follows...

    CrashedCheckBellowForHintsWhy() (CrashReporter.mm:97)
    UnhandledExceptionEventHandler_Invoke_m689053609 (Bulk_mscorlib_8.cpp:9214)
    RuntimeInvoker_Void_t2779279689_Il2CppObject_Il2CppObject(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:951)
    il2cpp::vm::Runtime::CallUnhandledExceptionDelegate(Il2CppDomain*, Il2CppDelegate*, Il2CppObject*) (Runtime.cpp:350)
    il2cpp::vm::Runtime::UnhandledException(Il2CppObject*) (Runtime.cpp:433)
    ScriptingInvocation::Invoke(ScriptingException**, bool) (ScriptingInvocation.cpp:202)
    MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodIl2Cpp, ScriptingObject*, ScriptingException**) (MonoBehaviour.cpp:839)
    MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodIl2Cpp, ScriptingObject*) (MonoBehaviour.cpp:859)
    Scripting::SendScriptingMessage(Unity::GameObject&, char const*, ScriptingObject*) (Scripting.cpp:254)
    WebScripting::processSendMessages() (WebScripting.cpp:286)
    PlayerLoop(bool, bool, IHookEvent*) (Player.cpp:1833)
    UnityPlayerLoopImpl(bool) (LibEntryPoint.mm:239)
    UnityRepaint (UnityAppController+Rendering.mm:236)
    -[UnityAppController(Rendering) repaintDisplayLink] (UnityAppController+Rendering.mm:52)
    CA::Display::DisplayLinkItem::dispatch() + 36
    CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 420
    IODispatchCalloutFromCFMessage + 368
    __CFMachPortPerform + 176
    __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 52
    __CFRunLoopDoSource1 + 432
    __CFRunLoopRun + 1796
    CFRunLoopRunSpecific + 380
    GSEventRunModal + 176
    UIApplicationMain + 200
    main (main.mm:32)
    0x0 + 0

    I'm assuming that "t2779279689" is the name of a C# type that has been converted to C++. However, I have no idea how I can link the C++ type back to the original C# type, or discover the offending method call.

    If it matters, this crash is reported by HockeyApp. It appears to be sporadic, and I have not seen it occur while I've been attached to Xcode (meaning it's hard for me to get much more information).

    Does anyone have any suggestions on where I could look to wring more information out of this stack trace?

    Another note, the crash above was from a build running Unity 5.2.3p4. I've also encountered what I believe is the same crash on 5.2.4f1. The only line that's different is the third line, and it's...

    RuntimeInvoker_Void_t_224166001_0_Object_t_Object_t(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:945)

    Thanks for any help you can provide!
     
  2. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    @BJO

    Sure, I can point out a few things. First, "t2779279689" means "type 2779279689" where type is a hash we generated based on the fully qualified type name in managed code. This should not be too important though, as we reserve the right to change the way we name things in generated C++ code in the future. However, you can always map a given function or type back to its managed equivalent by using the generated C++ code.

    In the Xcode project, you can search for any method or type (say UnhandledExceptionEventHandler_Invoke_m689053609 in this case). The implementation of that method in C++ will have a comment just above it that listed to full managed name for that method.

    The method RuntimeInvoker_Void_t2779279689_Il2CppObject_Il2CppObject happens to not be generated from managed code, but instead is a generated helper method that we use. So you won't be able to track down where that method is in C#, for example.

    That is all pretty general information. As to this specific case, note that the call stack you see here is not the call stack where the crash happened. On iOS, if the "Script Call Optimization" option in the player settings has a value of "Fast but no Exceptions" then any exception in managed code that escapes out of the project's script code and back to the engine will cause exactly what you see here.

    Unity will trigger the AppDomain.UnhandledException event. You can register a specific handler for this event, but Unity registers a general handler for it. This handler logs the information about the call stack from the exception for the Unity CrashReport API, then it terminates the application. So you could use that CrashReport API or you could register your own handler for the UnhandledException event and log the call stack elsewhere.

    However, I thought that HockeyApp actually does register this event handler and record the actual manage call stack for the exception. Maybe there is a different configuration option in HockeyApp to enable this, I'm not sure.

    Also, the difference in the third line between Unity versions probably does not matter. We've just changed the way we name things in generated C++ code.
     
    dan_ginovker likes this.
  3. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    @JoshPeterson we have a similar problem with an update we released last week. We're finding a lot of devices are having this crash but we have no idea what's causing it.

    What can we do as developers in general?

    Due to this I've had a quick look at the Crash Reporting API. If that is implemented would it give better information than this?

    I could not seem to get it working at all though. Perhaps you can respond to my post here.

    I also see that the Performance Reporting tool is integrated with Unity 5.4, so once we upgrade I will enable it.

    Screen Shot 2016-08-09 at 12.01.02.png
     
  4. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    @andymads

    I've not used the Unity Crash Reporting API myself, so I can't offer too much practical advice on it. Make sure you select the "Enable CrashReport API" option in the iOS player settings as described here: https://docs.unity3d.com/Manual/class-PlayerSettingsiOS.html. I've pinged our iOS team to respond to the other thread you posted about how to use the crash reporting API.

    You may also want to try to use the AppDomain.UnhandledException event as mentioned above. That should give you a managed code stack trace which might help locate the cause of the issue. Also, is this something you can reproduce locally? If so, catching the crash in Xcode is probably best. You can use an exception break point in Xcode to track down the cause of the issue if it is a C++ or managed exception (See this post for details: http://blogs.unity3d.com/2015/05/20/il2cpp-internals-debugging-tips-for-generated-code/).

    Finally, if this is not occurring locally, you can use a number of different crash reporting systems to help track down the cause of crashes on end user devices. Most of them work with Unity 5.3 and earlier. I've not used them myself, but I've heard of good results others have had.
     
  5. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    @JoshPeterson


    Thanks for the tips.

    Yes, I did enable CrashReport API.

    Any idea how to actually force a crash so I can test the CrashReporting API and AppDomain.UnhandledException?

    We cannot reproduce locally, it's end-user devices, except it's been seen on one device at our other office. I had the crash logs from that device sent to me but they only give me the callstack that I already posted above.
     
  6. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    @andymads

    You can throw a managed exception from your script code. That should trigger the UnhandledException event for testing.
     
  7. thomasn_unity

    thomasn_unity

    Unity Technologies

    Joined:
    Jun 21, 2016
    Posts:
    60
    @andymads

    I just replied on your other thread, but we have a custom preview build that will allow you to capture native crashes like this and see them in our Game Performance dashboard. It will capture both the native stack trace and the managed stack that the exception originated from. Please let me know if you're interested in giving it a try.

    You get a report that looks like this (notice how similar the native stack is to yours):

     
  8. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    Can you give an example?
     
  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    @andymads

    Sure. You can throw a managed exception from the Start method, for example:

    Code (CSharp):
    1. public class Throws : MonoBehaviour
    2. {
    3.     void Start()
    4.     {
    5.        throw new System.Exception("Test exception.");
    6.     }
    7. }
     
  10. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    That's exactly what I was doing, however I was doing it from a button OnClick handler. Now I moved it to Start it worked.

    However, now I've simulated a couple of crashes, the CrashReporting API is itself crashing I think, when I try to use it.

    This is using Mono and not IL2CPP btw.

    * Assertion at aot-runtime.c:1745, condition `!is_wrapper' not met

    Screen Shot 2016-08-09 at 17.20.49.png
     
  11. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    This is shutting the gate after the horse has bolted for us, but I would be interested in getting this stuff into our future builds. Presumably though it's Unity 5.4 only?
     
  12. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    @andymads

    This looks like a bug in the crash reporting API, as you mentioned! Can you get anything useful from it with IL2CPP instead?
     
  13. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    Yes, works ok with IL2CPP, but couldn't get the AppDomain.UnhandledException event to work.
     
  14. thomasn_unity

    thomasn_unity

    Unity Technologies

    Joined:
    Jun 21, 2016
    Posts:
    60
    We currently have preview builds based off of 5.3.5p8 and 5.4.0f3. If you can reproduce the crash in-house, these would give you the managed stack trace. Otherwise, you'd have to ship a new build of your game using the preview build.
     
  15. thomasn_unity

    thomasn_unity

    Unity Technologies

    Joined:
    Jun 21, 2016
    Posts:
    60
    I think the problem there might be that the crash report API is also registered for AppDomain.UnhandledException, and it gets the callback first and triggers the crash, so your callback handler never gets called. I'm speculating a bit on that though. :)
     
  16. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    That's ok. I was just testing really to see what's possible. Am interested in using the Game Performance dashboard going forward if it will give us better info on what's going wrong in end-user builds. I'm currently on 5.3.6, you don't have a preview build for that?
     
  17. thomasn_unity

    thomasn_unity

    Unity Technologies

    Joined:
    Jun 21, 2016
    Posts:
    60
    That build is a couple weeks old and 5.3.6 wasn't out yet when that build was made. I can update it and spin up a new build if you're sure you want to try it. Would you want 5.3.6f1 or 5.3.6p2?
     
  18. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    Don't do anything yet, I need to speak to my boss. I'll come back to you.
     
  19. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    @thomasn_unity ok can we please get the preview build for 5.3.6f1. We are going to update our app next week and it would be good to try the Game Performance functionality out with that.

    What do I need to do?
     
  20. BJO

    BJO

    Joined:
    Mar 25, 2013
    Posts:
    7
    @JoshPeterson First off, thanks for your original reply to this post. I've been meaning to thank you for you response since, well, since you originally posted it.

    @andymads Skimming through this discussion, it seems like your questions might have diverged from my original question. However, in case it's helpful, here's the conclusion I came to...

    As the second line alludes to, "UnhandledExceptionEventHandler", this crash seems to be the result of a managed exception being thrown that is never caught. What was confusing me is that in this situation, two crash reports were generated. One report would have the stack trace above. The other would have a meaningful c# stack trace. What was even more confusing is that not all c#-originating crashes seems to trigger the stack trace posted above. Ultimately, I concluded that crash reports like the one above can be ignored, as they always correspond to a second, meaningful report. One potentially important footnote is that HockeyApp is used for our crash reporting. I have not tried Unity's performance reporting.
     
  21. appsscity1

    appsscity1

    Joined:
    Feb 6, 2020
    Posts:
    20
    Hey @thomasn_unity @JoshPeterson

    We are also getting this crash at our end in iOS.

    Crash log-
    QuartzCore
    CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long)

    Check attach file for crash log.

    Unity Version- 2020.3.33f1

    Please give us a solution because we are getting this crash in production as well. it's related to internet i think.
     

    Attached Files:

  22. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    It looks like many of the frames in the call stack are not symbolicated correctly, so I can't really help track down the cause with this information, unfortunately.
     
    appsscity1 likes this.
  23. appsscity1

    appsscity1

    Joined:
    Feb 6, 2020
    Posts:
    20
    Hi @JoshPeterson

    Thanks for replying soon.

    Here i am sharing symbolicated crash. Check attached file and please let us know about it as soon as you can.
     

    Attached Files:

  24. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    Thanks for the additional details. It is still difficult to tell the cause though. There is no way that there are 8 calls to the selector_thread method, so something is still wrong with the symbolication.

    I can tell that the problem has something to do with the managed code in the Fusion.Runtime.dll assembly, but I don't know much more than that.
     
    appsscity1 likes this.
  25. appsscity1

    appsscity1

    Joined:
    Feb 6, 2020
    Posts:
    20
    Thanks a lot for the information. @JoshPeterson

    I have just find out that we are getting this crash at some other sections also where it's not Fusion. I will share those logs with you. Please check attached logs at let us know if you have some insights on it.

     

    Attached Files:

  26. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,921
    Unfortunately I don't have any insights about these. They don't look like anything we have seen recently, and they may even be from different causes. My advice would be to submit a bug report for them, and then maybe Unity QA will be able to reproduce them and we can investigate them.

    https://unity3d.com/unity/qa/bug-reporting
     
    appsscity1 likes this.
  27. appsscity1

    appsscity1

    Joined:
    Feb 6, 2020
    Posts:
    20
    Okay @JoshPeterson

    I will submit when we will find any reproduction of this issue because currently we are not able to reproduce it at our end.
     
    JoshPeterson likes this.