Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Utilities ░ ▒ ▓ █ Heap Explorer - Memory Profiler, Debugger and Analyzer for Unity

Discussion in 'Tools In Progress' started by Peter77, Apr 22, 2018.

  1. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Oh, Thanks for the reminder of that bug. I re-opend it and marked as fixed, pending the release of Memory Profiler package version 0.4.0 (to be released ~this month) but the underlying changes are already in the back-end for Unity versions 2021.2.0a12, 2021.1.9, 2020.3.12f1, 2019.4.29f1 or newer.

    (snapshot.managedHeapSections.startAddress & 1ul << 63) == 0
    signifies managed object heap sections going forward from these versions. Memory Profiler Package 0.4.0 is also going to report the same stats as the Memory Profiler Module is showing in a similar way as it is now doing in 2021.2, but with 2 further categories not available in the Memory Profiler Module, i.e. Executable and DLLs and Manage Virtual Machine memory when using Mono (IL2CPP memory isn't reporting these yet. And yep, those are the managedHeapSections with their highest bit set.) Also, that UI includes reporting the total amount used as reported by the OS (not yet implemented on all platforms) and where it is reported, show how much memory is used but not tracked by Unity's memory manager. Yet.

    So: Managed Heap Used maps to all reported Managed Heap sections, regardless of Object usage within these, as that is indeed the memory that is used in OS/Managed VM terms. Admittedly, a bit counter intuitive. Gonna have to see if we can get a per frame stat for actual object memory usage, if that isn't too expensive to get.

    Update: 0.4.0 is released
     
    Last edited: Sep 2, 2021
    Peter77 likes this.
  2. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Is there a way to try to find coroutines ongoing using Heap Explorer? We have found some nasty leaks here recently in our project, and I am curious if we could use your tool to potentially finding more of them.
     
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Good question. I just looked at it, but I don't think there is any useful information in the memory snapshot that would help. Perhaps some Unity staff can chime in and enlighten me if I miss anything.

    Below you can find an image what Heap Explorer knows from the snapshot (Windows Player) of the code below. There are no "Path to root" and no useful references as far as I can tell.

    How exactly does the leak look like that you found?

    upload_2021-9-2_14-10-4.png

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. public class NewBehaviourScript : MonoBehaviour
    6. {
    7.     static int s_Counter = 0;
    8.  
    9.     void Start()
    10.     {
    11.         StartCoroutine(MyStaticCoroutine());
    12.         StartCoroutine(MyWorkerCoroutine());
    13.     }
    14.  
    15.     void Update()
    16.     {
    17.         if ((s_Counter % 13371337) == 0)
    18.             Debug.Log(s_Counter);
    19.     }
    20.  
    21.     IEnumerator MyWorkerCoroutine()
    22.     {
    23.         while (enabled)
    24.         {
    25.             s_Counter++;
    26.             yield return null;
    27.         }
    28.     }
    29.  
    30.     static IEnumerator MyStaticCoroutine()
    31.     {
    32.         while (true)
    33.         {
    34.             s_Counter++;
    35.             yield return null;
    36.         }
    37.     }
    38. }
    39.  
     
    GloriaVictis likes this.
  4. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    We have found out that there was a coroutine that was supposed to start once, but it was starting on the update and never closing, so basically worst case scenario but that made us find that.

    I can imagine that on such a big project there could be an edge case where we would think of "there should be only one coroutine ongoing" but they stack, or they never end. Of course, that requires a pretty neat F***up on the code, but those will always happen.

    So all I need to know is "how many coroutines with the method name or origin script" are running. - but looking at your screen, it should be doable with Heap Explorer currently.

    Awesome, thanks!
     
    Last edited: Sep 2, 2021
  5. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    We hadn't yet considered debugging Coroutines via the Memory Profiler. Usually the CPU Profiler does the job.

    That said, I guess you could have a search filter for anything named
    *>d_*
    but I'm currently not 100% sure that naming scheme is guaranteed. Would have to double check in mono code.
     
    Last edited: Sep 5, 2021
    SugoiDev and Peter77 like this.
  6. Gotmachine

    Gotmachine

    Joined:
    Sep 26, 2019
    Posts:
    34
    Hi there.

    First : fantastic job ! This is the real tool we need to actually debug managed memory leaks.

    My main use case is finding destroyed MonoBehaviours that are kept around by delegates or static references. I saw that at some point you created an "empty-managed-shell-detection" branch for such an use case.

    I've tried using that branch, but the feature doesn't seem to work when using it against a built x64 Unity 2019.4 player, the "C# Empty Shell Objects" view is empty, despite such objects clearly existing in the regular "C# objects" view.

    Any chance we can get that (very useful) feature in an update ?
     
    Peter77 likes this.
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
  8. Gotmachine

    Gotmachine

    Joined:
    Sep 26, 2019
    Posts:
    34
    That was fast !

    Unfortunately, it doesn't work for my use case (a built player).

    Looking at your code, it seems to be checking for the
    m_InstanceID
    field value to exclude some built-in editor-only objects. But since the
    m_InstanceID
    field doesn't exists on built players, this piece of code will exclude all objects on built players.

    I would suggest changing
    ManagedEmptyShellObjectsControl.cs
    to proceed if the
    m_InstanceID
    field isn't found :
    Code (CSharp):
    1. // Try to get the m_InstanceID field (only exists in editor, not in built players)
    2. PackedManagedField packedField;
    3. if (richType.FindField("m_InstanceID", out packedField))
    4. {
    5.     // The editor contains various empty shell objects whose instanceID all contain 0.
    6.     // I guess it's some kind of special object? In this case we just ignore them.
    7.     var instanceID = memoryReader.ReadInt32(obj.address + (ulong)packedField.offset);
    8.     if (instanceID == 0)
    9.         continue;
    10. }
    This gives me expected results for my use case.

    Edit : also, small UI QoL issue, when using "Capture and Analyze..." while a previous capture is already shown in a view, the view isn't automatically updated with the new capture and can still be manipulated with the outdated capture data. You have to manually select another viewand come back to update it.
     
    Last edited: Jun 29, 2022
    SonicBloomEric likes this.
  9. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Thank you. I made these changes and published an update. It's included in version 4.1.1.

    Good catch. I'm not sure when I get to it. But if you want to fix it and open a Pull Request, I'm happy to merge it :)
     
  10. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    This is wonderful project, thank you!
     
    Peter77 likes this.
  11. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    970
    Amazing as per usual, @Peter77! I've been using this quite a lot, very handy!

    One little thing I'd love to see would be multi-selection in the object view. All I'd expect from that would be the info window to show any aggregate data that can be calculated, such as the sum of the sizes of the selected objects. I could look into doing it and submitting a pull request if you don't have the time.
     
    Peter77 likes this.
  12. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Thank you :)

    Yes, submitting a PR would be the fastest way to get something done on the project, I guess :) The code is quite complex I think, not sure how easy it would be if the code is new to you.
     
  13. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Hello,

    We are using your tool on daily basis and it's great! We found one thing which is potentially being missed. Leaks from interfaces seems to not be showed at all - we are using 2019.4 LTS.

    https://t.gyazo.com/teams/gloriavictis/e8252c885806d0de633dd19e92a1fc8a.png
    Leak from CraftingWorkshopCreator is visible, yet IContainerLootServer is not visible (classes implementing this interface leaked this way are also not being listed).

    Thanks in advance and once more, great tool!
     
  14. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Thank you for the kind words. Can you create a small example script that can be used for testing it? That would help to make sure to look into the correct issue :)
     
  15. FAAndreev

    FAAndreev

    Joined:
    Feb 14, 2019
    Posts:
    6
    @Peter77
    Oh my gosh, this is such an exceptional tool. Thank you so much for sharing it!
    Is there any way to see the full path from an object to the selected root? Having objects on depth 36 makes it a bit hard to track down the exact chain of references :D

    By the way, do you accept tips or donations?
     
    Peter77 and robrab2000-aa like this.
  16. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    Thank you for the kind words. At the moment there exists only the "Paths to Root" panel:

    upload_2022-12-10_12-45-10.png


    Do you have any ideas how the full path should be visualized to make it easier when the path is deep?
     
  17. FAAndreev

    FAAndreev

    Joined:
    Feb 14, 2019
    Posts:
    6
    Ahh, I just figured that this list is actually "a tree of nested references" and not "a list of siblings". After this realisation, I think it is fine as it is.
     
  18. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    In Unity 2022.2 the PackedMemorySnapshot has been deprecated and I can't find how to load a memory snapshot with the new API.

    In earlier versions I was able to load a memory snapshot with:
    UnityEditor.Profiling.Memory.Experimental.PackedMemorySnapshot.Load(path);


    The new namespace seems to be Unity.Profiling.Memory.MemoryProfiler, but I can't find any "load snapshot" API there or in any of its parent namespaces.

    How can I load a memory snapshot with the new API in lets say Unity 2022.3?
     
  19. binxxxx

    binxxxx

    Joined:
    Sep 18, 2020
    Posts:
    3
    @Peter77
    I found your tool and trying to use it in my project, but when I trying to capture a snapshot, the Heap Explore window shows nothing.

    When I look into the source code, it seems API :
    Code (CSharp):
    1. UnityEngine.Profiling.Memory.Experimental.MemoryProfiler.TakeSnapshot
    didn't return a snapshot and the captureResult is false, but it didn't show any reason. I want to know why the capture is failedo_O.

    My Unity Version: 2021.2.13f1
    Heap Explore: 4.1.1

    Hoping for your reply:)
     
  20. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Sorry for not getting back to you but just FYI: we held back from pulling the plug on the now deprecated API until we have better planing security on the replacement API. I had for some reason, from a previous look at HE, assumed it didn't need much of any of the APIs, similar to the Memory Profiler itself, which just straight up reads the file.