Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Assets ░ ▒ ▓ █ Heap Explorer - Memory Profiler, Debugger and Analyzer for Unity

Discussion in 'Works In Progress' started by Peter77, Apr 22, 2018.

  1. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Woohoo \o/

    Most of the stuff just worked fine with 2019.3. Very surprising to me, since Unity replaced the underlying memory profiling back-end as far as I know. Thumbs up towards @alexrvn , @MartinTilo and the rest of the gang who worked on that.

    The only things I noticed that did break in Heap Explorer were related to the overhauled UI in 2019.3, but also only in specific views.

    In the profiler forum I found a post where they discuss "empty managed shell objects". It's basically a native object that was destroyed, where its managed object representation is still hanging around, which seems to indicate some sort of memory leak. According Martins descriptions, I believe it would be fairly simple to implement detection for that in Heap Explorer, so I might implement that sooner or later :)
     
  2. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    I've tested the Heap Explorer package with Unity 2020.1.0b6 and it seems to work.

    I wasn't sure if it does, because of this comment:
    So you didn't change the existing API, I guess? Otherwise I'm wondering why Heap Explorer still works :)

    BTW, documentation seems to be missing for the "Connection" type:
    https://docs.unity3d.com/2020.1/Doc...ence/MemoryProfiler.PackedMemorySnapshot.html
     
  3. roberto_sc

    roberto_sc

    Joined:
    Dec 13, 2010
    Posts:
    143
    Are you doing it?? :):):)
     
  4. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Do you know how to produce such "empty shell object"? I would need that to have something to test against when I implement detection for it. Or do you have a Heap Explorer memory snapshot that contains such object, that I could use?
     
  5. QFSW

    QFSW

    Joined:
    Mar 24, 2015
    Posts:
    2,833
    I would imagine if you stored a reference to a GO then deleted said GO, you would end up with an empty shell that can't be GCd as it's still referenced
     
    Peter77 likes this.
  6. roberto_sc

    roberto_sc

    Joined:
    Dec 13, 2010
    Posts:
    143
    @Peter77 yeah it's like @QFSW said, but I'm happy to provide a project and/or snapshot if you need it.

    ...but it's something like this:

    Code (CSharp):
    1. public class SomeClass : MonoBehaviour {}
    2.  
    3. public class AnotherClass : MonoBehaviour {
    4. private SomeClass _someClass;
    5. private void Awake()
    6. {
    7.     GameObject go = new GameObject();
    8.     _someClass = go.AddComponent<SomeClass>();
    9. }
    10.  
    11. private void Update()
    12. {
    13.    // take a snapshot before pressing any key and one after
    14.     if (Input.anyKeyDown)
    15.     {
    16.         Destroy(_someClass);
    17.     }
    18. }
     
    Peter77 likes this.
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Thanks for the snippet.

    I've created a branch with the empty shell object detection. If you use Unity 2019.3 or newer, you can just add this git URL via the Unity Package Manager:
    https://github.com/pschraut/UnityHeapExplorer.git#empty-managed-shell-detection

    Otherwise please download and add to your project manually:
    https://github.com/pschraut/UnityHeapExplorer/tree/empty-managed-shell-detection

    Can you please let me know if this works for you? Because if it works, then I'm turning it into a proper update and merge it into master.

    You can find it under the "View" popup:
    upload_2020-5-25_17-27-58.png


    Opening this view will analyze the snapshot for empty shell objects and displays them like:
    cs_empty_shell_objects_01.png

    As shown in the image above, the C# MeshRenderer object has no native object anymore (m_CachedPtr = null) and is referenced by "AnotherClass".

    This was the code I used for testing:
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. public class AnotherClass : MonoBehaviour
    6. {
    7.     private UnityEngine.Object _someClass;
    8.  
    9.     private void Awake()
    10.     {
    11.         GameObject go = new GameObject();
    12.         _someClass = go.AddComponent<MeshRenderer>();
    13.     }
    14.  
    15.     private void Update()
    16.     {
    17.         // take a snapshot before pressing any key and one after
    18.         if (Input.anyKeyDown)
    19.         {
    20.             Debug.Log("Destroy");
    21.             Destroy(_someClass);
    22.         }
    23.     }
    24. }
    25.  
     
  8. CircuitLord

    CircuitLord

    Joined:
    Dec 15, 2016
    Posts:
    7
    Heyo! This is potentially a dumb question, but I've had issues using this and Unity's official memory profiler.
    Trying to capture a standalone player in either of them results in

    I've tried letting it run for over half an hour, I have a very high end PC so I don't really foresee that being an issue. The editor log file mentions no errors, it just stays on that progress bar forever. However, if I capture an editor player, it works completely fine and finishes quickly.

    My project isn't massively sized or anything either.

    Tried using 2019.3 and 2020.2.
     
  9. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Does the issue occur with both, "Capture and Save" and "Capture and Analyze"? I'm wondering if it's related to capturing or analyzing. From what platform do you try to capture a memory snapshot? Just from PC or some mobile device or console?

    If "Capture and Save" works, I could take a look at the snapshot why it's not loading. If "Capture and Save" and Unity's Memory Profiler don't work, it could be a problem on Unity's side.
     
  10. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    Does this happen in 2019.3.12f - 2019.3.14f1? If so, these versions (and 2020.x versions released around the same time frame) have been affected by this bug that is fixed in 2019.3.15f1.
     
    Peter77 likes this.
  11. CircuitLord

    CircuitLord

    Joined:
    Dec 15, 2016
    Posts:
    7
    Sorry about the delayed response :/

    Not sure if it was something I changed with my project, or if it was updating, but I went to Unity 2020.2a12 and everything seems to be working now, both in Heap Explorer and the Unity Memory Profiler. ^_^ Thanks guys!
     
    MartinTilo and Peter77 like this.
  12. roberto_sc

    roberto_sc

    Joined:
    Dec 13, 2010
    Posts:
    143
    Hi @Peter77
    I'm sorry I took so long to answer, I was busy with other things in the project.
    I tried to use the tool but I get an exception. On the other hand, my colleague was able to use it and said it is really helpful!

    I get this when I change the view to "Empty C# Shell Objects":

    Code (CSharp):
    1. NullReferenceException: Object reference not set to an instance of an object
    2. HeapExplorer.AbstractManagedObjectsView.OnGUI () (at Assets/Plugins/UnityHeapExplorer/Editor/Scripts/ManagedObjectsView/ManagedObjectsView.cs:303)
    3. HeapExplorer.ManagedEmptyShellObjectsView.OnGUI () (at Assets/Plugins/UnityHeapExplorer/Editor/Scripts/ManagedEmptyShellObjectsView/ManagedEmptyShellObjectsView.cs:54)
    4. HeapExplorer.HeapExplorerWindow.DrawView () (at Assets/Plugins/UnityHeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:486)
    5. HeapExplorer.HeapExplorerWindow.OnGUI () (at Assets/Plugins/UnityHeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:398)
    6. System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <567df3e0919241ba98db88bec4c6696f>:0)
    7. Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
    8. System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <567df3e0919241ba98db88bec4c6696f>:0)
    9. System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at <567df3e0919241ba98db88bec4c6696f>:0)
    10. UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at /Users/builduser/buildslave/unity/build/Editor/Mono/HostView.cs:359)
    11. UnityEditor.HostView.Invoke (System.String methodName) (at /Users/builduser/buildslave/unity/build/Editor/Mono/HostView.cs:353)
    12. UnityEditor.HostView.InvokeOnGUI (UnityEngine.Rect onGUIPosition, UnityEngine.Rect viewRect) (at /Users/builduser/buildslave/unity/build/Editor/Mono/HostView.cs:329)
    13. UnityEditor.MaximizedHostView.OldOnGUI () (at /Users/builduser/buildslave/unity/build/Editor/Mono/GUI/DockArea.cs:1092)
    14. UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize) (at /Users/builduser/buildslave/unity/build/Modules/UIElements/IMGUIContainer.cs:281)
    15. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr) (at /Users/builduser/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:179)
    When I was capturing the memory I also got a lot of this:

    Code (CSharp):
    1. Missing Profiler.EndSample (BeginSample and EndSample count must match): Texture.ReleaseTextureUploadMemory
    2. Previous 5 samples:
    3.     AsyncUploadManager.gScheduleAsyncRead
    4.     AsyncUploadManager.ScheduleAsyncCommand
    5.     AsyncUploadManager.gScheduleAsyncRead
    6.     AsyncUploadManager.ScheduleAsyncCommand
    7.     AsyncUploadManager.gScheduleAsyncRead
    8. In the scope:
    9.     Texture.ReleaseTextureUploadMemory
    10.     AsyncUploadManager.AsyncResourceUpload
    11.     Render Thread
    I'm attaching the snapshot that makes the problem happen, my colleague also gets the exception when loading this:
    https://drive.google.com/file/d/1KVwIPTRqBEZg-K64XBNCF2yhEKidEArt/view?usp=sharing

    I'm using 2019.2.21f1.

    ----
    EDIT:
    Now I found out that when I profile the editor it works. I was profiling an iPad Mini 2 before.
     
    Peter77 likes this.
  13. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    @MartinTilo @alexrvn

    It seems the
    m_InstanceID
    field in
    UnityEngine.Object
    does not exist in a Player.

    How can I get the instanceID of an object in a Player?

    The UnityEngine.Object type captured from an iPad Mini 2 with 2019.2.21f1 looks like this (notice no m_InstanceID):
    upload_2020-6-9_6-45-8.png


    The UnityEngine.Object type captured from the Editor with 2019.3.7f1 looks like this:
    upload_2020-6-9_6-45-44.png

    I use the m_InstanceID field during the "empty managed shell objects detection" (link) and only consider objects where the m_InstanceID value isn't 0 (link).

    I implemented it like this, because the editor contains various empty shell objects whose instanceID all contain 0. I assumed it would be some kind of special object or so.

    But because the m_InstanceID field does not exist in the Player snapshot, it skips them all :)

    @roberto_sc Thanks for the snapshot! I fixed the null-ref issue, but I'm now stuck with the issue described above. I'll post if I have an update.
     
    roberto_sc likes this.
  14. QFSW

    QFSW

    Joined:
    Mar 24, 2015
    Posts:
    2,833
    Does GetInstanceID() work? At least in the editor it's equivalent to m_InstanceID looking at the decompilation
     
  15. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    yeah that's the wrong field to check here, mostly because (as you found out) this m_InstanceID is stripped in a build. Checking
    m_CachedPtr = 0
    is a valid check for leaked shells that also works with builds. Alternatively, any managed object inheriting fro UnityEngine.Object that isn't referenced by a native object via their gchandle target is a leaked shell.

    In both of these cases, the shell could be leaked or waiting for the next GC.Collect, so to be sure it is a leaked shell, it still needs to have some references pointing to it that have a valid root.
     
    Peter77 likes this.
  16. roberto_sc

    roberto_sc

    Joined:
    Dec 13, 2010
    Posts:
    143
    @Peter77 do you usually get snapshots from playing in the Editor? I thought they were not reliable, I thought they would contain not only game memory but stuff from the Editor itself, since Editor snapshots are much bigger in size in the Memory Profiler. I always get from the device.
     
    Peter77 likes this.
  17. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    It totally depends and I'm sorry in advance to point out the obvious :)
    • If I want to check memory usage in a build, capturing a build is required.
    • If I want to check memory usage in the editor, capturing the editor is required.
    • If I want to test something in Heap Explorer, capturing a snapshot in the editor is most of the time sufficient.
     
  18. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    So only checking
    m_CachedPtr
    would be sufficient?

    This was my first implementation, but then found there are a lot, what seems to be Unity internal, objects in the editor that all have
    m_CachedPtr = 0
    , so I added the additional filter to check for rm_InstanceID as well, because those objects all have m_InstanceID=0.
     
  19. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    Yeah, at least in theory. Then again no one keeps anyone from building types that have a field of that name so some filtering might be reasonable to sanitize this, like checking it is indeed a managed UnityEngine.Object inheriting type.
     
  20. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    380
    This tools looks exactly what I needed, but unfortunately Editor freezes on capturing the snapshot from iOS dev build.
    Tried different Unity versions: 2019.3, 2019.4, same result.
    Furthermore Memory Profiler doesn't work as well, same freeze.
    I am on Mac, developing for iOS/Android

    The log says:
    Code (CSharp):
    1. Creating memory snapshot file: {path}
    Creating memory snapshot file: {path}

    UPDATE
    Seems that relates to Memory Profiler bug in 2019.3.13 and 2013.4.4
    Migrated to 2019.3.15 and that works fine
     
    Last edited: Jul 16, 2020
    Peter77 likes this.
  21. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    I guess those version numbers where typos? There was indeed bug, introduced in 2019.3.12f0 was fixed in 2019.3.15f0 (or .16?) but 2019.4.X should be wholy unaffected because all of those versions are just newer versions on the same branch. Anyways, just in case it weren't typos, if this re-occures in any 2019.4 version, please file a bug :)
     
  22. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    380
    you are right, just tested it again on 2019.4.4 and that worked
     
    MartinTilo likes this.
  23. Sohaib_techverx

    Sohaib_techverx

    Joined:
    Aug 15, 2019
    Posts:
    16
    hi,
    I just downloaded the package using package manager with the git URL and I got this error:

    Library/PackageCache/com.oddworm.heapexplorer@e29645c02f1e04d53ce91202330bc0d0d79caa0c/Editor/Scripts/TestVariables.cs(9,7): error CS0246: The type or namespace name 'NUnit' could not be found (are you missing a using directive or an assembly reference?)

    Apparently no-one else have this issue. I am using Unity 2019.3.06f. Is there anything I am missing?

    Thanks,
    Sohaib
     
    Peter77 likes this.
  24. andre_unity573

    andre_unity573

    Joined:
    Oct 17, 2018
    Posts:
    34
    @Sohaib_techverx you need to include manually the "Test Framework" package as well if you removed from your project. @Peter77 could not this be marked as a dependency when installing your package via git package?

    @Peter77 one thing that I did not understood and it is not in the documentation is what GCHandles indicates and how can I solve any data that is remaining and referenced in it. It would help me a lot understand this.

    Also, I'm trying to find here some memory that is remaining between scenes and it is strange that several delegates of mine appears as static when in theory they are not static.. I mean I'm creating lambda expressions and anonymous functions and placing them in some functions and referencing this delegates into actions/events in classes, is this expected? Am I understading something wrong?

    I'm not used to handle memory leaks, so I'm not sure also if the issue is that the class which contains this event/action is not being cleared due to being referenced by someone which is not being cleared and so on.. Is it how it usually goes?
     
    Sohaib_techverx and Peter77 like this.
  25. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Thanks for letting me know. I believe that should be a simple fix, will post an update here when I fixed it.

    To be honest, I don't know how useful the gchandles view actually is. As far as I remember, gchandles are used to prevent the garbage collector from collecting a managed object, which is just referenced on the native side. It's a while ago, my memory sucks, I'd have to look it up to provide a better explanation, but perhaps @MartinTilo could help me out here :)

    I've never looked at how lambda expressions and anonymous functions appear in a memory snapshot and whether it's correct. Good question :) Do you know if they appear as static in Unity's memory profiler as well?
    I'm asking, because I'm wondering:
    • Perhaps that is just correct
    • Perhaps it's a bug in heap explorer, in this case unity's memory profiler would not display it
    • Perhaps it's a bug in the memory profiler api, in which case heap explorer and unity's memory profiler would be affected
    @MartinTilo you probably know if lambda expressions and anonymous functions should appear as static delegates?
     
    andre_unity573 likes this.
  26. andre_unity573

    andre_unity573

    Joined:
    Oct 17, 2018
    Posts:
    34
    @Peter77, have not checked the delegates in Memory Profiler in Unity. I'm using Memory Profiler mostly for the TreeMap and such, but I was finding it easier to use Heat Explorer to find references and filter between the different data types, but I will take a look later and let you know! And thanks for the quick answer :)
     
    Peter77 likes this.
  27. andre_unity573

    andre_unity573

    Joined:
    Oct 17, 2018
    Posts:
    34
    Honestly I could not find myself in Unity Memory profiler to find as easily the same anonymous functions and lambda expressions as I have found in Heap explorer hehe If you want I can send via direct message a snapshot from both heap and memory profiler if you want to check later.
     
  28. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Heap Explorer 3.3.0 is available on github now :)
    https://github.com/pschraut/UnityHeapExplorer

    FIXED
    • "The type or namespace name 'NUnit' could not be found" (thanks Sohaib and andre)
    • "You need Unity 2017.4 or newer" dialog when running Heap Explorer in Unity 2020.2 (thanks Martin)
     
  29. jonagill_rr

    jonagill_rr

    Joined:
    Jun 21, 2017
    Posts:
    54
    Thank you for keeping Heap Explorer up to date! In my experience it's a hell of a lot faster and easier to use than Unity's Memory Profiler (although that has some useful features on its own, especially when looking at individual assets in the Map view). Most importantly, I can't find any function in Memory Profiler that compares to your Paths to Root feature.

    Speaking of, I've found that Paths to Root sometimes throws an exception and freezes mid-way through analyzing objects in my captures. I get an IndexOutOfRangeException thrown on line 340 of RootPathControl:

    Code (CSharp):
    1. value.Add(new ObjectProxy(obj.snapshot, obj.snapshot.gcHandles[c.from]));
    Have you encountered this before?
     
    Peter77 likes this.
  30. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Yes, I encountered it just recently when I was testing Unity 2020.2.0a17. What version did you use?

    I believe it's related to a change in the Memory Profiling API they did a while ago and Unity staff were actually kind enough to tell me I should take a look at it, because they expected it would cause issues with Heap Explorer (they actually made me aware of it twice, so shame on me for not listening):
    https://forum.unity.com/threads/hea...analyzer-for-unity.527949/page-5#post-5821303

    When I checked with 2019.3 and 2020.1, I didn't see an issue. The documentation doesn't seem to be up to date (1, 2).

    Sooo... not really sure what to change exactly, but I'll take a look at it at the weekend and hopefully figure out what to do.
     
  31. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    Whenever you create an allocation on the managed heap, you also create a GCHandle to it. The Garbage collector uses that GCHandle to keep track of the allocation. The memory snapshots contain a list of all GCHandles and the entire contents of the managed heap. That managed heap includes data for memory that has since been garbage collected but not overwritten yet. Memory Profiler tools then need to use the GCHandles to figure out what of the data in the managed Heap is actually still kept alive and what is just dead and ready to be overwritten (if that particular heap section is still the heap section at the highest virtual memory address and therefore the active heap section) or flushed out (as soon as the entire heap section it is in is empty and mono/IL2CPP decided it is time to say goodby and return the memory to the OS).

    There's also other ways of creating GCHandles than just allocating heap memory. You could pin something to memory, box some value, or manually call GCHandle.Alloc() on an object (e.g. a struct). The Memory snapshots currently don't report the type of the GCHandle but we're looking into adding that.

    An Example:
    Code (CSharp):
    1. var x = new MyObject();
    2. var y = x;
    3. var z = GCHandle.Alloc(x);
    4. var w = GCHandle.Alloc(x, GCHandleType.Weak); // or new WeakReference(x)
    5. var wr = GCHandle.Alloc(x, GCHandleType.WeakTrackResurrection); // or new WeakReference(x, true);
    6. var p = GCHandle.Alloc(x) // or ulong p; void * ptr = UnsafeUtility.PinGCObjectAndGetAddress(x, out p);
    7.  
    This results in 5 GCHandles out of which 3 (x, z and p, weak ones are dropped and assignment doesn't create one) get considered to get reported by the memory profiler snapshot API. However, the API is currently only reporting memory addresses for these GCHandles (i.e. just the GCHandle.Target value), and while reporting these, we're pruning out duplicated addresses. End result: 1 GCHandle for all the code above is reported.

    We will eventually change this to report all GCHanldes, together with their type and without pruning them but that change should therefore then be quite obvious and hopefully self explanatory / well documented ;)

    Now as to what the value is in showing all the GCHandles specifically in a Memory Profiler... Currently I guess it is a bit debatable. I mean, I guess there is value in recognizing that the GCHandles take up some memory (but it isn't even showing the duplicates so...), but beyond that it's very much just a list of what type of things are still alive. With the full reporting of GCHandles, the value would be in seeing how many handles are referencing the same thing and what type they are.

    There is multiple parts to the static-ness of lambdas/anonymous functions and delegates. First of, lambdas/anonymous need to live somewhere from a compiler perspective, so they get added to a static class.
    Next and possibly relevant to this bit:
    referencing this delegates into actions/events in classes
    is that if these events are static, then the delegate/action formed implicitly on assigning to these events (if it is already of the correct type then it won't be converted but just referenced directly) will be referenced by a static field. So the field would be static and that is then a classic risk of accidentally keeping it around forever. This is particularly tricky with anonymous functions because unless you keep the original Action of it around, you can't de-register them, because you'd invariably be creating a new action that you'd try to de-register that then doesn't exist in the event list, so nothing is de-registered.

    A bit of code to illustrate the potential issues with lambdas and event subscriptions:
    Code (CSharp):
    1. event Action<int> test;
    2. int x = 0;
    3. void Update()
    4. {
    5.     var y = x++;
    6.     // with a closure
    7.     test += (x1) => { y += x1; };
    8.     test(x);
    9.     test -= (x1) => { y += x1; }; // This won't unsubscribe because it is not the same delegate!
    10.     test = null; // This is the only way to get rid of the subscriber again.
    11.  
    12.     // and without one
    13.     test += (x1) => { x1++; };
    14.     test(x);
    15.     test -= (x1) => { x1++; }; // This won't unsubscribe because it is not the same delegate!
    16.     test = null; // This is the only way to get rid of the subscriber again.
    17.  
    18.     Action<int> actWithClosure = (x1) => { y += x1; };
    19.  
    20.     test += actWithClosure;
    21.     test(x);
    22.     test -= actWithClosure; // unsubscribes successfully
    23.  
    24.     Action<int> actWithoutClosure = (x1) => { x1++; };
    25.  
    26.     test += actWithoutClosure;
    27.     test(x);
    28.     test -= actWithoutClosure; // unsubscribes successfully
    29.  
    30.  
    31.     test += TestMethod; // implicitly allocates a new delegate of type Action<int>
    32.     test(x);
    33.     test -= TestMethod; // implicitly allocates a new delegate of type Action<int>, but it matches the old one, so it unsubscribes successfully
    34.  
    35.     x = y;
    36. }
    37.  
    38. void TestMethod(int x1)
    39. {
    40.     x += x1;
    41. }
    Does this help? Otherwise I think I'll need more context on this.

    We are very aware of this gap, as it is a feature missing in the Memory Profiler package that was in the Bitbucket one before. We are working on this.

    Yes, we're in the process of fixing that. Somehow the only page that got updated with the new info is the one on NativeObjectEntries.gcHandleIndex. I hope the info I send you in the pm helps though.

    Note that right now, Native to Manage object connections and maybe even GCHandles in HeapExplorer might be showing faulty data for 2019.3 and up. (I didn't look to deep into the code of HeapExplorer though).
     
    Peter77 and SugoiDev like this.
  32. andre_unity573

    andre_unity573

    Joined:
    Oct 17, 2018
    Posts:
    34
    Hi @MartinTilo! Thanks for answering :)

    Hmm, interesting, not sure I understood everything about the lambd and anonymous methods yet.
    Let me give a few simple examples to understand if there is any risk in these:

    Code (CSharp):
    1.  
    2. public class MyClass : MonoBehaviour
    3. {
    4.     private List<Texture2D> textures;
    5.     private Action myAction;
    6.  
    7.     private void Function1()
    8.     {
    9.         // Does this where usage function will be considered static?
    10.         // If so, will these sprites stay in the memory? Even after my mono is destroyed? Considering that my function is not static neither my class.
    11.         textures = GetComponentsInChildren<Texture2D>().ToList().Where(__texture => __texture.name.StartsWith("filter")).ToList();
    12.     }
    13.  
    14.     private void Function2()
    15.     {
    16.         // Does this where usage function will be considered static?
    17.         // If so, will these sprites stay in the memory? Even after my mono is destroyed? Considering that my function is not static neither my class.
    18.         CustomFunctionWithCallback(delegate (Texture2D p_texture)
    19.         {
    20.             textures.Add(p_texture);
    21.         });
    22.     }
    23.  
    24.     private void Function3()
    25.     {
    26.         Texture2D __texture = new Texture2D(1, 1);
    27.         // Does this where usage function will be considered static?
    28.         // Is there any change in the behaviour considering this delegate is using a local variable?
    29.         CustomFunctionWithCallback(delegate ()
    30.         {
    31.             textures.Add(__texture);
    32.         });
    33.     }
    34.  
    35.     private void Function4()
    36.     {
    37.         // Is this delegate also considered static?
    38.         // Will I need to manually set it to null when my mono is destroied?
    39.         // Or considering the action is not static this is not necessary?
    40.         myAction += delegate ()
    41.         {
    42.             textures.Add(new Texture2D(1, 1));
    43.         };
    44.     }
    45.  
    46.     private void Function5()
    47.     {
    48.         // Is there any difference in this delegate where no private variable from the script is used?
    49.         // Will this be static and require some null setting to clear the texture created inside?
    50.         myAction += delegate ()
    51.         {
    52.             Texture2D __texture = new Texture2D(1, 1);
    53.             Debug.Log("Creating the texture for nothing");
    54.         };
    55.     }
    56. }
    57.  
    Consider that myclass can also not be mono necessarily but being referenced my someone that is monobehaviour which is also cleared..

    I'm trying to understand if there is something wrong with these lambda and anonymous functions since they appear to be remaining in memory or if they are remaining in memory because of another issue in my game logic which can be holding a reference which I still not found that can lead to these stay in the memory later..
     
  33. jonagill_rr

    jonagill_rr

    Joined:
    Jun 21, 2017
    Posts:
    54
    We're using 2019.4.2f1, so perhaps the change came downstream to both the 2019 LTS and 2020 branches?
     
  34. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,314
    No, it originated in 2019.3

    @andre_unity573 to your examples:
    I believe all the delegates here would be considered static methods on a static class but I'd have to inspect the IL to be sure.
    Function 1:
    It's a bit wasteful in terms of GC allocs because you Alloc two lists and two arrays, only to keep the last list but it shouldn't keep the textures in memory. The where delegate would end up on a static class but that shouldn't be an issue, it's just we're the method is kept for compiler purposes. Do you see this show up in the Profiler?

    Function 2:
    The function would I guess be static, though I'm unsure about it's reference to the closure (the inclusion of a local variable in the lambda). I'd have to inspect the IL. So not sure if it would keep a reference to the list after the delegate is GCed. I'd check it with the Memory Profiler to see if the list survives the instance of the class and it's usage in the function it is passed to, because it depends on what CustomFunctionWithCallback does with the callback. As long as that method is somehow keeping a reference to the passed delegate, the delegate instance will be kept in memory and anything that is part of it via that closure I'd assume.

    In case the closure keeps a reference to the Textures list, you can call Clear on the list in this class, then the only thing it would hold a closure on though would be a now empty list taking up it's capacity * pointer size plus the size of all list fields. The textures could unload.

    Function 3:
    The method local Texture that gets the closure should behave pretty similar to the Textures list, except the reference to it will survive the scope (instead of the textures list GCHandle potential surviving the instance of the class) if CustomFunctionWithCallback keeps the reference around for longer than the calling scope exists (e.g. by passing it to a coroutine, async call or adding the reference to some field.).


    Function 4 and 5:
    Since myAction isn't static, the delegates would be collected together with the class instance, and so would be what they reference. In case of Function 4, that means the reference to the textures list. But that list would be kept around as long as the class instance and therefore the event is kept around anyways.

    Now same as cases 2&3, case 4 might be different to case 5 if the static code would somehow keep a reference to the textures list in a way that it'll keep it alive beyond the class instance. As I mentioned, I'd need to inspect the IL to be sure (or double check with the Memory Profiler if it gets unloaded properly) but I'd guess it wouldn't?

    I'd double check these assumptions but, not at a computer right now.
     
  35. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Heap Explorer 3.4.0 is available on github now :)
    https://github.com/pschraut/UnityHeapExplorer

    Please thank @MartinTilo for explaining to me the new
    connections
    format in a memory snapshot. Without his help, this update would not have been possible. Thank you Martin!

    I've updated Heap Explorer to use the new Memory Profiling API, that was introduced in Unity 2019.3 and is also the back-end of Unity's own Memory Profiler. This will hopefully get rid of a few issues.

    The migration wasn't too bad, because I was lucky enough to had abstracted the memory snapshot already and only needed to rewrite the "Unity to HeapExplorer snapshot conversion", rather than every location where it's used. It's this commit if anybody is interested.

    However, while migrating to the new API, I also found this bug:
    (Case 1269293) 2019.4: PackedMemorySnapshot: nativeObjects.gcHandleIndex contains -1 always

    Which means Heap Explorer will not display "native object to/from gchandle" connections. Martin told me via PM how I can workaround it myself, but I don't have time to work on it anymore this weekend. And to be honest, I hope Unity can just fix it. If I find time next weekend, I might look into the workaround.

    I've also added checks to detect invalid array-indices during "Searching Root Paths", it will now handle this gracefully with an error message rather than an exception.

    Please let me know whether 3.4.0 works for you.
     
  36. alexrvn

    alexrvn

    Unity Technologies

    Joined:
    May 16, 2017
    Posts:
    44
    Heya Peter, the the -1 handle index bug has been fixed up ^^, and is currently in flight to 19.4. I'll also be fixing up the issue with the old api where connections data is being reported as InstanceID instead of indices, that seems to have been an unintended side-effect of the backend overhaul in 19.3.
     
    Last edited: Aug 14, 2020
    Peter77 likes this.
  37. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    That's excellent news, thank you!

    LOL, so I didn't even have to update Heap Explorer to use the new API. I guess it was the peer pressure that made me act fast. ;)
     
  38. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    435
    Hey @Peter77
    CaputureAndSaveHeap fails in 3.4.0 due to sharing violation. I worked around it by replacing
    Code (csharp):
    1. UnityEngine.Profiling.Memory.Experimental.MemoryProfiler.TakeSnapshot(path, OnHeapReceivedSaveOnly);
    with
    Code (csharp):
    1. UnityEngine.Profiling.Memory.Experimental.MemoryProfiler.TakeSnapshot(FileUtil.GetUniqueTempPathInProject(), OnHeapReceivedSaveOnly);
     
    chrismarch and Peter77 like this.
  39. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Thank you for the info! I'm going to take a look at the issue at the weekend.
     
    chrismarch likes this.
  40. chrismarch

    chrismarch

    Joined:
    Jul 24, 2013
    Posts:
    430
    I've created a pull request with changes inspired by this, but slightly different to allow for the file name authoring in the original code, and to fix a follow up violation when trying to write to the heap file to the snapshot file:
    https://github.com/pschraut/UnityHeapExplorer/pull/1

    Thanks for this tool! I have to get the diff working, as the diff of MemoryProfiler is missing hundreds of MB that I was hoping HeapExplorer would catch, but even if not, being able to walk references to the asset bundles is invaluable
     
    Last edited: Oct 22, 2020
    Peter77 likes this.
  41. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    That's great, thank you!
     
  42. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
  43. kukuruzza2

    kukuruzza2

    Joined:
    Aug 28, 2017
    Posts:
    3
    Hey Peter77 !
    Thank you very much for sharing this tool, really saved my day. Still much better than Unity's own mem profiler.
     
    Peter77 likes this.
  44. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    Thank you for the kind words. I'm glad it still proves useful :)
     
  45. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,589
    I released Heap Explorer 3.7.0:
    • Fixed "Count" display in Compare Snapshots not updating correctly after swapping two snapshots. Thanks to niqibiao for the fix. See PR#3 for details.
    • Fixed slow performance and high memory usage in rendering large lists, in particular when resizing a column or the window. Thanks to niqibiao for the fix. See PR#3 for details.
    Download or install package from github:
    https://github.com/pschraut/UnityHeapExplorer
     
unityunity