Search Unity

Can't Debug IL2CPP Builds

Discussion in 'iOS and tvOS' started by Picky-Salamander, Aug 25, 2015.

  1. Picky-Salamander

    Picky-Salamander

    Joined:
    Apr 26, 2013
    Posts:
    27
    When attempting to remote debug IL2CPP iOS builds with MonoDevelop I get the error: "
    Could not connect to the debugger." The error pops up almost instantly maybe a second or two after I try and connect to the iPad debugger. I've tried many things to get the script debugging to work, but the only way I can find is to switch "Scripting Backend" to "Mono (2.x)" in the build settings. Once it's switched to mono, there is no problem and the debugger connects quickly without incident.

    I checked the MonoDevelop log and it says the following when trying to connect to IL2CPP builds:

    [usbmuxd] Start listen thread
    [usbmuxd] Listen thread started
    [usbmuxd] Send listen message
    [usbmuxd] Attached: 1 ec6a9ab6737e712110b8f7b59a1750fb7dc4e794
    [iproxy] Start proxy: ec6a9ab6737e712110b8f7b59a1750fb7dc4e794 12000->56000
    [iproxy] Setup listen port
    [iproxy] Setup listen port succeeded
    [iproxy] Proxy thread started
    [iproxy] Accept incoming connection
    [iproxy] Got connection
    [iproxy] Connect to device
    [iproxy] Failed to connect to device: Connection refused
    [iproxy] Accept incoming connection
    ERROR [2015-08-25 10:37:29Z]: Error in debugger
    Mono.Debugging.Soft.ConnectionException: Could not connect to the debugger. ---> System.IO.IOException: DWP Handshake failed.
    at Mono.Debugger.Soft.Connection.Connect () [0x000f9] in /home/builduser/buildslave/monodevelop/build/monodevelop/main/contrib/Mono.Debugger.Soft/Mono.Debugger.Soft/Connection.cs:1099
    at Mono.Debugger.Soft.VirtualMachine.connect () [0x00000] in /home/builduser/buildslave/monodevelop/build/monodevelop/main/contrib/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachine.cs:329
    at Mono.Debugger.Soft.VirtualMachineManager.Connect (Mono.Debugger.Soft.Connection transport, System.IO.StreamReader standardOutput, System.IO.StreamReader standardError) [0x00022] in /home/builduser/buildslave/monodevelop/build/monodevelop/main/contrib/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachineManager.cs:334
    at Mono.Debugger.Soft.VirtualMachineManager.ConnectInternal (System.Net.Sockets.Socket dbg_sock, System.Net.Sockets.Socket con_sock, System.Net.IPEndPoint dbg_ep, System.Net.IPEndPoint con_ep) [0x00075] in /home/builduser/buildslave/monodevelop/build/monodevelop/main/contrib/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachineManager.cs:287
    --- End of inner exception stack trace ---
    [iproxy] Stop proxy: 12000
    [iproxy] Proxy thread exiting

    I've attached my build settings to this post as well. Is there something I'm doing wrong here? Is there some setting that I should turn on/off?

    Thanks!
     

    Attached Files:

    Picky-Salamander, Aug 25, 2015
    #1

  2. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    No, there is nothing wrong on your end. We don't support managed code debugging with IL2CPP yet. We have a alpha quality version of it internally, but it is not ready to be released. We are however looking getting this working.

    In the meantime, you can debug the C++ code generated by IL2CPP in Xcode. The debugging experience is not nearly as good as debugging managed code, but it is better than nothing. You can find some details here:

    http://blogs.unity3d.com/2015/05/20/il2cpp-internals-debugging-tips-for-generated-code/
     
    JoshPeterson, Aug 26, 2015
    #2

  3. Picky-Salamander

    Picky-Salamander

    Joined:
    Apr 26, 2013
    Posts:
    27
    Ah ok, in the meantime we'll compile with mono 2x when we need to do script debugging.
     
    Picky-Salamander, Aug 26, 2015
    #3

  4. Iamdain

    Iamdain

    Joined:
    Feb 3, 2010
    Posts:
    90
    @JoshPeterson any idea on timeline for script debugging of managed code in IL2CPP?
     
    Iamdain, Nov 25, 2015
    #4

  5. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    @Iamdain

    No, unfortunately I don't have a timeline yet. This is on our internal roadmap, but we're not ready to schedule it in a release yet. When we do, it will show up on the Unity public roadmap: https://unity3d.com/unity/roadmap
     
    JoshPeterson, Nov 25, 2015
    #5

  6. Iamdain

    Iamdain

    Joined:
    Feb 3, 2010
    Posts:
    90
    Ok, unless there's some other technique for gathering crash data on beta devices I'm not aware of I'd be surprised if this isn't a massive priority for most developers with live apps.

    We used to be able to script debug, send out to 30-40 beta testers, track and fix exceptions immediately with traces down to the line number in managed code. I've just spent three days straight trying to track down the cause of an exception logged from live il2cpp code, I have no idea if it's the signature for multiple causes, where they're coming from, no clues at all other than the trace started at Unity's Update or FixedUpdate loop and ended in an unhanded exception.

    We don't seem able to trigger the issue/s in editor nor when moving back to the MONO compiler - in any case I wouldn't be comfortable testing and declaring problems fixed on a different compiler to the one we're shipping with.

    Do you know any better approach to debugging such a problem @JoshPeterson ?

    Example trace:

    Hardware Model: iPhone6,1
    Process: **
    Path: **
    Identifier: **
    Version: 1.4.6.5
    Code Type: ARM-64
    Parent Process: ??? [1]

    Date/Time: 2015-11-29T10:41:45Z
    OS Version: iPhone OS 9.0.1 (13A404)
    Report Version: 104

    Exception Type: SIGTRAP
    Exception Codes: #0 at 0x100089ce8
    Crashed Thread: 0

    Thread 0 Crashed:
    0 [APPNAME] 0x0000000100089ce8 CrashedCheckBellowForHintsWhy() (CrashReporter.mm:106)
    1 [APPNAME] 0x00000001015df458 UnhandledExceptionHandler_NativeUnhandledExceptionHandler_m9_1829 (Bulk_UnityEngine_2.cpp:21600)
    2 [APPNAME] 0x00000001015df258 UnhandledExceptionHandler_HandleUnhandledException_m9_1827 (Bulk_UnityEngine_2.cpp:21549)
    3 [APPNAME] 0x00000001019d792c UnhandledExceptionEventHandler_Invoke_m3_7467 (Bulk_mscorlib_8.cpp:41018)
    4 [APPNAME] 0x00000001019d7870 UnhandledExceptionEventHandler_Invoke_m3_7467 (Bulk_mscorlib_8.cpp:41006)
    5 [APPNAME] 0x0000000101da5044 RuntimeInvoker_Void_t3_151_Object_t_Object_t(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:996)
    6 [APPNAME] 0x000000010255d710 il2cpp::vm::Runtime::CallUnhandledExceptionDelegate(Il2CppDomain*, Il2CppDelegate*, Il2CppObject*) (Runtime.cpp:345)
    7 [APPNAME] 0x000000010255d690 il2cpp::vm::Runtime::UnhandledException(Il2CppObject*) (Runtime.cpp:428)
    8 [APPNAME] 0x000000010209a2bc ScriptingInvocationNoArgs::Invoke(ScriptingException**) (ScriptingInvocationNoArgs.cpp:123)
    9 [APPNAME] 0x000000010209a180 ScriptingInvocationNoArgs::Invoke() (ScriptingInvocationNoArgs.cpp:87)
    10 [APPNAME] 0x0000000102091688 MonoBehaviour::CallUpdateMethod(int) (MonoBehaviour.cpp:560)
    11 [APPNAME] 0x0000000101fcec18 void BaseBehaviourManager::CommonUpdate<FixedBehaviourManager>() (Behaviour.cpp:198)
    12 [APPNAME] 0x0000000102058b6c PlayerLoop(bool, bool, IHookEvent*) (Player.cpp:1891)
    13 [APPNAME] 0x0000000101e4992c UnityPlayerLoop (LibEntryPoint.mm:253)
    14 [APPNAME] 0x000000010007db64 UnityRepaintImpl(bool) (UnityAppController+Rendering.mm:228)
    15 [APPNAME] 0x000000010007d2dc UnityRepaint (UnityAppController+Rendering.mm:240)
    16 [APPNAME] 0x000000010007d2bc -[UnityAppController(Rendering) repaint] (UnityAppController+Rendering.mm:70)
    17 [APPNAME] 0x000000010007d1ac __51-[UnityAppController(Rendering) repaintDisplayLink]_block_invoke (UnityAppController+Rendering.mm:55)
    18 libdispatch.dylib 0x0000000197ce97b0 _dispatch_call_block_and_release + 20
    19 libdispatch.dylib 0x0000000197ce9770 _dispatch_client_callout + 12
    20 libdispatch.dylib 0x0000000197ceee20 _dispatch_main_queue_callback_4CF + 1840
    21 CoreFoundation 0x00000001829a4258 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 8
    22 CoreFoundation 0x00000001829a20c0 __CFRunLoopRun + 1624
    23 CoreFoundation 0x00000001828d0dc0 CFRunLoopRunSpecific + 380
    24 GraphicsServices 0x000000018da24088 GSEventRunModal + 176
    25 UIKit 0x0000000187faaf60 UIApplicationMain + 200
    26 [APPNAME] 0x0000000100077318 main (main.mm:37)
    27 libdyld.dylib 0x0000000197d1a8b8 start + 0
     
    Last edited: Nov 30, 2015
    Iamdain, Nov 30, 2015
    #6

  7. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    @Iamdain

    I'm curious about what is different between Mono and IL2CPP in this regard, specifically with respect to managed stack traces. Previously with the Mono scripting backend, where did you see the managed stack traces?

    Currently Il2CPP does not provide source code line numbers for managed stack traces, but it should be able to still generate useful stack traces with method names.

    It looks like this call stack comes from the Unity crash report system on iOS: http://docs.unity3d.com/ScriptReference/CrashReport.html

    This is using the C# AppDomain.UnhandledException event (https://msdn.microsoft.com/en-us/library/system.appdomain.unhandledexception(v=vs.110).aspx). You should be able to hook up a custom listener to that event, then inspect the arguments passed to the event handler to see the managed stack trace.

    But again, if you didn't need to do any of this with the Mono scripting backend, it is probably better to track down the difference with IL2CPP so that we can make IL2CPP work similar to Mono in this regard.
     
    JoshPeterson, Nov 30, 2015
    #7

  8. Iamdain

    Iamdain

    Joined:
    Feb 3, 2010
    Posts:
    90
    @JoshPeterson

    We were using HockeyApp with script debugging enabled and it would debug with native methods and often line numbers in the trace with Mono2.x compiled builds.

    This C# hook looks like a solution, I just tried hooking into it and logging properties from a forced exception and it's telling me the managed script and method name. Booyah! Hopefully this will tell us about the gameplay crashes related to the the trace above, will need to rig up a system to log remotely. Thanks for the tip, this looks like it might help.

    No idea why HockeyApp isn't utilising this and collecting more meaningful data then, maybe they used to intercept exceptions at a different point when 'script debugging' was available. I'll email their support and note here for anyone interested.
     
    Iamdain, Nov 30, 2015
    #8

  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    @Iamdain

    Thanks for the information. I wonder if HockeyApp was catching signals to handle crashes. If so, we may need to work with them to get things working better with IL2CPP (as it does not use signals). Please post here if/when you hear back from them.
     
    JoshPeterson, Dec 1, 2015
    #9

  10. Iamdain

    Iamdain

    Joined:
    Feb 3, 2010
    Posts:
    90
    @JoshPeterson yeah with IL2CPP Hockey doesn't seem to give any useful information about the source of null reference exceptions, which in my experience is the vast majority of exceptions. We tend to switch back to Mono build a lot for fast build time in testing cycles anyway but I wonder if there aren't others getting quite stuck with debugging that haven't figured out / don't want to / are unable to do this.

    In any case the only way to debug a published / live IL2CPP build exhibiting problems would be to:
    a) Receive notice of a crash in the build
    b) Push out a Mono compiled version of the build to testers (assuming NO changes have been made since launch to guarantee testing identical code-base and peace of mind - highly unlikely)
    c) Wait / hope for someone to trigger a crash
    d) Fix the crash and try to convince ourselves that it was the one occurring in the live build

    Perhaps there's something I'm doing wrong with setup but I think that's where it's at. Would be awesome if you do have the ability to help Hockey-app work with the new compiler.

    For now we successfully made a system to catch exceptions using the method you suggested and log them to our Parse server on next game load. Still a bit of deduction needed as it resolves down to the originating method name but in most cases can find the issue in seconds and with relative certainty that we've fixed the correct bug. =)
     
    Iamdain, Dec 15, 2015
    #10

  11. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    @Iamdain

    This workflow for tracking down crashes seems like far too much hassle (as I'm sure you've seen). We really should make HockeyApp work with IL2CPP. Can you put together a sample project showing how to integrate with HockeyApp and submit a bug report? That is probably our best route to solving this issue.
     
    JoshPeterson, Dec 15, 2015
    #11

  12. alvaro-em

    alvaro-em

    Joined:
    Feb 23, 2012
    Posts:
    77
    @JoshPeterson, is this feature still not supported? I am unable to debug any IL2CPP project. Any deadlines already? I believe this is a "must" feature that has been delayed too much...
     
    alvaro-em, Feb 24, 2016
    #12

  13. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    @alvaro.em

    We don't have any support for a managed debugger with IL2CPP ready to release yet. Your best option is still to debug the generated C++ code, as mentioned earlier in this thread. We don't have a deadline yet for supporting a managed debugger. We would like do so, but it is a matter of a lack of development resources from our side at the moment. We're working on higher priority items.

    I don't see a feature request yet for this on feedback.unity3d.com. Can you add one?
     
    JoshPeterson, Feb 24, 2016
    #13

  14. alvaro-em

    alvaro-em

    Joined:
    Feb 23, 2012
    Posts:
    77
    Done!

    https://feedback.unity3d.com/suggestions/ios-ilcpp-debugging-support

    Thank you!
     
    alvaro-em, Mar 9, 2016
    #14

  15. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,938
    JoshPeterson, Mar 9, 2016
    #15