Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

No Line Numbers in CallStack for Development Builds

Discussion in 'Unity Cloud Diagnostics' started by alan-lawrance, Oct 22, 2019.

  1. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    We have the "Development Builds" option enabled for the Cloud Build, but the diagnostics from these builds does not give us line numbers:

    Managed Stack Trace:

    PolyPhysics.World.RemoveAndDestroyNode (Poly.Physics.NodeHandle node) (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    PolyPhysics.World.Clear () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    BridgePhysics.Reset () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    Bridge.StartSimulation () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    GameStateMainMenu.StartSimulation () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    GameStateMainMenu.FixedUpdateManual () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    GameStateManager.FixedUpdateManual () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    GameManager.FixedUpdateManual () (at <9234b57c782d4eab90802e4c8278dd7d>:0)
    Main.FixedUpdate () (at <9234b57c782d4eab90802e4c8278dd7d>:0)

    Is there something I'm missing that would allow line numbers to show up?
     
  2. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi @alan-lawrance, do you know if you're using IL2CPP or the Mono scripting backend when generating your builds? You can check in the metadata of your report in the Dashboard and looking for "scripting_backend" to see which one you're using. We do not currently support line numbers in IL2CPP for either debug or release builds. Could you try switching your scripting backend to Mono? That should work in debug builds.
     
  3. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    The scripting_backed is Mono.

    The "Development Builds" option is set to Yes in the Cloud Build Config. Anything else necessary for the build to be considered debug?
     
  4. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    That is odd. You should be seeing line numbers properly. Which platform are these reports coming from?
     
  5. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
  6. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    I I see. Cloud Diagnostics is just displaying the stack trace that is returned to us by the scripting backend team. I'll reach out to them to find out what could be going on here and get back to you as soon as I learn anything.
     
  7. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Not sure if it's relevant, but we are obtaining a zip of the build from the Cloud Build back-end. Since we have the Cloud Build configured as a Dev build it should be fine, but I guess it's one thing that may be out of the ordinary.
     
  8. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi Alan, sorry for the late response. I'm still trying to track down the root of this issue. Could you share with me a link to one of your reports that has this issue in it so that I can share it with our devs for debugging? Just giving me your report's URL will be fine.
     
    Last edited: Oct 31, 2019
  9. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
  10. MatL

    MatL

    Joined:
    Jan 29, 2014
    Posts:
    43
    I am having the same issue with unity cloud builds, the line number is not appearing on the report on Crash and Exceptions.
    ryanc-unity I will send you a link.
     
  11. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Ryan -- sent you a link.

    As far as I can tell the symbols uploaded successfully. The full log doesn't indicate any errors related to symbols, and we are getting the function names in the stack... it's just the line numbers are all 0.
     
  12. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Thank you both for submitting your reports, @alan-lawrance and @MatL! After looking through them we can't find any reason, with regards to the Cloud Diagnostics service, that you aren't getting line numbers in your debug builds. We can only guess that the symbols file is getting stripped unintentionally, either manually on your end or during the Cloud Build setup. I've pinged the Cloud Build team to see if they have any input on this issue and will update here when I have more info.
     
  13. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Thanks Ryan. Checking if there is any update. We are at the point in our project where crash reporting is going to be a vital part of our testing process and having line numbers for exceptions will be important to quickly narrow down issues.
     
  14. MatL

    MatL

    Joined:
    Jan 29, 2014
    Posts:
    43
    Thanks Ryan, we actually don't need to set the build to develop mode since is using the symbols that where created when building on unity cloud right?

    We are too working on some production exceptions that is going to be very helpful to have the line number.
     
  15. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    @alan-lawrance, I talked with the Cloud Build team and they wanted to know if you are able to reproduce this issue when building locally, as well as check which version of Unity you're building in, primarily in Cloud Build.

    @MatL, you do need to specify that you're using a development build if you want line numbers for your call stack. They're not currently supported yet in release builds. In the Config section of the Cloud Build Dashboard, look under the Advanced Options and make sure that Development Builds are enabled.
     
  16. MatL

    MatL

    Joined:
    Jan 29, 2014
    Posts:
    43
    @ryanc-unity Thanks for the clarification, if I use Development builds is working correctly, I thought that was also supported for release builds.
     
  17. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Apologies for the confusion. We're working to add line number support for release builds that use IL2CPP in the near future. (Mono will be a lot trickier so we don't have a projection for that yet)
     
  18. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Sorry for the delay on running the local build test --

    Creating a local standalone build with the "Copy PDB files" set in the Build Settings results in line numbers for the callstack in the debug log.

    I noticed that "Copy PDB files" doesn't appear to be an option anywhere in the Cloud Build settings, and setting it locally doesn't modify any project files. Is it possible this isn't set on the Cloud Build side for some reason?
     
  19. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    No worries on the delay. I double checked with the Cloud Build team and they recommended writing a post-export method script to enable that setting (which can be added under the Additional Options for the Build Target.
     
  20. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    OK, I'll do that. It's odd though that this setting is not exposed in Cloud Build, but seems to default to true for most people.
     
  21. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Unfortunately we are still seeing '0' for line numbers after adding export methods to turn on the CopyPDBFiles setting.

    The methods are in a the file Assets/Editor/ExportMethods.cs:

    Code (CSharp):
    1. public class ExportMethods
    2. {
    3.     public static void PreExport()
    4.     {
    5.         EnableCopyPDBFiles();
    6.     }
    7.  
    8.     public static void PostExport()
    9.     {
    10.         EnableCopyPDBFiles();
    11.     }
    12.  
    13.     public static void EnableCopyPDBFiles()
    14.     {
    15.         EditorUserBuildSettings.SetPlatformSettings("Standalone", "CopyPDBFiles", "true");
    16.     }
    17. }
    On the Build side we specify both the Pre and Post export methods as:

    ExportMethods.PreExport
    ExportMethods.PostExport
     
  22. hoangminh1789

    hoangminh1789

    Joined:
    May 5, 2017
    Posts:
    3
    I have the same issue when build project on my PC. I build with scriptbackend Mono
     
  23. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
  24. rmon222

    rmon222

    Joined:
    Oct 24, 2018
    Posts:
    77
    Hi @ryanc-unity, Are line numbers supported when building locally, IL2CPP, for development build? The exceptions I'm seeing in Player.log do not have line numbers.
     
  25. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Hi, I have a similar problem as well.

    I don't know exactly since when (but it's fairly recent), but I have now a Development Build on Steam, and I also don't seen any longer the line numbers in the stack trace. It has Mono backend and Script Debugging is active.

    Moreover, before every exception logged, it appears a line I never noticed before "Uploading Crash Report" (but the game didn't crash, it was just a minor exception)

    a Debug that it's in the same build, is instead now shown like this:

    : 35)

    I am using Unity 2019.2.8f1. Any idea on what I might be doing wrong? Could the Steam drm-wrapper be connected to the problem?

    EDIT: As a small update, this seems to only happen with the Steam builds. Either using only the Steamworks.NET framework or with the DRM wrapper as well. A build of the same without any of them, in local, works like a charm. Still for our Early Access having bug reports with line errors would be incredibly useful if there is a way to do it.
     
    Last edited: Jan 21, 2020
  26. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    [EDIT] Hi @rmon222, line numbers are not supported in IL2CPP builds of either debug or release builds, so what you're seeing is as expected. Sorry for any confusion.
     
  27. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    We are using Steamworks.NET as well, so that may very well be a factor.
     
    drHogan likes this.
  28. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    I did run some testings, and it is that indeed. I suspected the Steam DRM at first, but then i tried a build without DRM and with Steamworks.NET only, and i had the same problem. Then a build without both, same settings, everything identical, and used it locally, and it did print the error line.
     
    cchute likes this.
  29. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Thank you for verifying where the issue is, guys! I've passed this info along to the Scripting team and will let you know if I hear anything back from them.

    Would either of you be willing to share a sample project using the Unity Bug Reporter that includes the offending Steamworks.dll that I could pass along to the Scripting team?
     
    Last edited: Jan 23, 2020
    drHogan likes this.
  30. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    I can try to reproduce it on monday, but it's a bit hard to test because Steamworks.NET needs an app ID to run properly and I am not sure if it would do using a fake one. If it works anyway, gonna upload it! Thanks Ryan!
     
  31. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Hi @ryanc-unity , sorry for the delay, but I did submit a small project, Case 1215261
     
  32. m-guntrum

    m-guntrum

    Joined:
    Mar 14, 2013
    Posts:
    6
    Hi, same issue here. Can also confirm it's related to Steamworks.NET.
     
    drHogan likes this.
  33. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Is there by chance any update or news on this? we're going through early access, and bug finding without line number and file names is horror!
     
  34. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi @drHogan! I've passed this bug along to the Scripting Team and updated them about what the issue is. Here's the Issue Tracker link for the issue: https://issuetracker.unity3d.com/product/unity/issues/guid/1215261/. I can't make any guarantees about how soon they'll get to this issue, but I will post here when there are any updates to share. Thank you for your patience!
     
  35. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Hi @ryanc-unity thank you very much! (I can't seem to open the link with the tracker though!)
     
  36. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Apologies! The link should work now. It took a while for the webpage to be created.
     
    drHogan likes this.
  37. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Yes it does, perfect, thanks!
     
    Ryan-Unity likes this.
  38. Xtro

    Xtro

    Joined:
    Apr 17, 2013
    Posts:
    604
  39. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    I added a comment to the Issue Tracker ticket saying the problem only occurs on builds produced by Unity Cloud Build.

    But as the item is marked as Not Reproducible, it may not be looked at again without some extra prodding.
     
  40. Xtro

    Xtro

    Joined:
    Apr 17, 2013
    Posts:
    604
    I copied all project settings files and packages manifest file from empty project to my project.
    Deleted the library folder.
    Still the same problem.
    Empty project can show callstack filenames.
    My project can't show them.
    This is a real RAGE QUIT :(
     
  41. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Hi @alan-lawrance actually to me it happens also with classic builds as long as they do integrate Steamworks.NET, i guess we're probably dealing with two separate problems.
     
  42. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    Mhm I just noticed now the Non Reproducible, what the hell. Every Unity developer I did ask to, that has a game on Steam, has the same issue.
     
    valentin56610 and Ivan-Pestrikov like this.
  43. drHogan

    drHogan

    Joined:
    Sep 26, 2012
    Posts:
    201
    @ryanc-unity hi, I did run a few more tests, and it turns out it needs to run correctly on steam in order to reproduce the problem.
    If started locally without a valid appID from Steam, steamworks.NET shuts down and the problem is not testable. I tried to find a way to keep it persistent but i couldn't find any.
    I don't know if you guys have any testing steam account or anything like that, but it seems like it would be necessary in order to reproduce the problem.
    I tried using my demo app ID code on the testing, but it was not enough and it did anyway launch the steam client and the steam app.
     
  44. Ivan-Pestrikov

    Ivan-Pestrikov

    Joined:
    Aug 8, 2014
    Posts:
    21
    The same problem still exists in 2019.2.21f.
    Steam + development build (Mono) without using the Unity Cloud.
    A stack trace contains no line numbers:
    Systems.RobotDirectControlSystem.Tick (System.Single deltaTime) (at <33c6e530fbd146279f09809e6221487c>:0)
    GameManager.Update () (at <33c6e530fbd146279f09809e6221487c>:0)
     
  45. Ivan-Pestrikov

    Ivan-Pestrikov

    Joined:
    Aug 8, 2014
    Posts:
    21
    What is interesting, some lines on the Unity side do have numbers:

    Systems.TechTreeDespawn.Execute (TechTreeData techTreeData) (at <68f8fa128b634d8ea0bdc20bcfd70745>:0)
    GameManager.DestroyUniverse () (at <68f8fa128b634d8ea0bdc20bcfd70745>:0)
    UIMainMenu.OnLoadPanelLoadButtonPress () (at <68f8fa128b634d8ea0bdc20bcfd70745>:0)
    UnityEngine.Events.InvokableCall.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnityEvent/UnityEvent.cs:166)
    UnityEngine.Events.UnityEvent.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnityEvent/UnityEvent/UnityEvent_0.cs:58)
    UnityEngine.UI.Button.Press () (at <c69dc105a139406b995621d4538d83f5>:0)
    UnityEngine.UI.Button.OnPointerClick (UnityEngine.EventSystems.PointerEventData eventData) (at <c69dc105a139406b995621d4538d83f5>:0)
    UnityEngine.EventSystems.ExecuteEvents.Execute (UnityEngine.EventSystems.IPointerClickHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at <c69dc105a139406b995621d4538d83f5>:0)
    UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) (at <c69dc105a139406b995621d4538d83f5>:0)
    UnityEngine.EventSystems.EventSystem:Update()
     
  46. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi guys, sorry for the delayed response. I hope you all are staying safe during these crazy times.

    Unfortunately we haven't been successful in reproducing this issue on our end. We do have test Steam accounts to use at Unity, but I and the fellow QA that I've reached out to so far aren't familiar with setting up a Unity project with Steam. If someone would be willing to provide a repro project, with detailed steps on how to setup their project with a Steam account, then we'll be able to look into a fix right away.
     
  47. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Not sure what changed -- but I am seeing line numbers in my crash reports in the past day or two. As far as I know nothing changed in the game configuration or cloud build configuration on my end. I'll keep an eye on it, but did anything change that might have fixed this on Unity's side?

    *** EDIT ***
    Turns out I was looking at crash reports from a build that was made locally, not from Unity Cloud Build. The issues appears to still exist when the build is made by Unity Cloud Buid.
    *** EDIT ***
     
    Last edited: Apr 23, 2020
    Ivan-Pestrikov likes this.
  48. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Friendly reminder if this problem affects you (or may in the future) please vote up this issue so it gets fixed sooner than later:

    https://issuetracker.unity3d.com/is...3.1369987345.1589211507-1353259828.1572867000

    Unfortunately we are likely not going to see this fixed as we head into our final phase before shipping. This is a serious issue and it's like having one arm tied behind your back when tracking down crashes.
     
  49. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    386
    Is there any estimation when line numbers will be possible in IL2CPP for development builds? We need to use IL2CPP, and debugging this without line numbers is a nightmare.
     
  50. p0w1nd

    p0w1nd

    Joined:
    Oct 28, 2015
    Posts:
    140
    +1 for line number in stack trace.
     
    calgof likes this.