Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Utilities ░ ▒ ▓ █ Heap Explorer - Memory Profiler, Debugger and Analyzer for Unity

Discussion in 'Tools In Progress' started by Peter77, Apr 22, 2018.

  1. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Thanks a lot of the encouragement!

    I've a pretty cool new feature in the works, that I'll show in a few days. Make sure to watch this forum thread, to get a notification when I post an update. :)
     
  2. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    970
    Having played with the new MemoryProfiler at Unite and having a good chat with both devs currently working on it (the backend dev and UI dev), it is very worth having a chat with them with any of the issues you have uncovered in the API directly. PM me if you would like to get in contact with them.
     
    Peter77 likes this.
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Uploaded an update, same link as before.

    Changelog
    • C++ Objects View: Added asset preview. Memory snapshots do not contain asset memory, thus the preview is generated from the asset in the project instead rather than the memory snapshot!
    • C++ Objects View: Reworked list filtering. It allows to change filter settings without the list being recreated every time you change a single option. You have to apply those changes now. This helps to save time when filtering big memory snapshots.
    • C++ Objects View: If any list filtering is active, the "Filter" button in the toolbar uses a different color, to make it obvious filtering is in place.
    • C++ Objects View: Fixed "Type" display of MonoBehaviour, which now displays the actual derived type rather than just MonoBehaviour always.
    • C++ Objects View: Fixed "Type" column sometimes showing the object name rather than type.
    • Brief Overview: Changing wording of the Memory Section description.


    Please let me know what you think.
     
  4. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    That's a nice offer, thank you.

    Several months ago, I contacted Unity Technologies and asked whether they're interested to help me with some memory profiling API issues.

    At that time, they gave me two or three contacts, unfortunately this didn't lead to more than just one email with no helpful information.

    Thus, I doubt it would be helpful trying to contact another UT person. If they're interested to make the Unity development world a better place by helping me helping everyone else being able to profile memory in large projects, they would have shown some interest to do this by now.

    PS: I documented most issues/questions in Heap Explorer's documentation. They did fix various of those bugs, which is really great. But there are still a few things and questions I would like to get answered.
     
    Last edited: Jul 7, 2018
  5. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,424
    Hi Peter, sorry our communication was a bit spotty so far. There is a lot still in flux with the backend as well as the UI for the new Memory Profiler and we're still a rather small team for the amount of work that needs to still go into this. I've been monitoring this thread sporadically and we're fixing things, but we're sort of split between the old backend (which your stuff is based on) and the new one (with better performance and stability which will replace the old one under the hood while keeping the API) and getting that balance right when there is a huge push to get this to our users is tricky.

    We do however greatly appreciate you pushing us on this and finding things we missed. Sometimes, half of the bugs for our team where reported by you and we always had to weight fixing them vs moving forward with the new stuff making sure it's fixed there. Also, just getting the lowlevel code that facilitates all of this into our codebase while making sure we don't regress is a very ... involved process.

    Also, pre-Unite, nothing was announced yet so it was a bit tricky to talk about it too ;)
    I can't promise anything but the backend and the new (additional) API has to solidify soonish and hopefully we'll find the time to get back to you then :)

    Also, while this might sound scrappy, as Greg mentioned in his talk, all of this is still going to be experimental and potentially not covering all use cases. However, we believe that anything of what we have coming is better than what we have now so ... we hope you can hold on for a bit more and a bit of a bumpy ride before things smoothen out :)

    Once again, thanks for all the valuable input so far :)
     
    SugoiDev and Peter77 like this.
  6. EthanHunt

    EthanHunt

    Joined:
    Oct 9, 2012
    Posts:
    14
    Is there a chance that I can get to try it?
     
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Big news today! I decided to provide Heap Explorer for free to everyone. I might change my mind in the future, I don't know yet.

    You can find a download link to a fully working Heap Explorer build in this thread, very first post.

    Why did I decide to do that? After I watched the Unite 2018 Memory Profiler talk, where Unity Technologies showed what they worked on behind closed doors, I assume Heap Explorer isn't going to be the financial success I believed, because there is now Unity's free memory profiler, that also seems really good and they have more man-power and possibilities to add great stuff, because they provide the memory profiling API in the first place.

    I assume it's going to be difficult to compete with the new memory profiler Unity is going to provide and in this case I decided to spend my time on other things instead.
     
    pahe, Its4u, Noisecrime and 2 others like this.
  8. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I just released Heap Explorer 2.4 beta, you can find the download link here.

    Changelog
    • Overview View: Fixed negative size display if object group exceeded the maximum int limit.
    • C++ Objects View: Fixed negative size display, if an object group exceeded the maximum int limit.
    • C# Objects View: Fixed negative size display, if an object group exceeded the maximum int limit.
    • Memory Sections: Fixed saving memory sections as file which are larger than 2gb.
    • Loading memory snapshots displays more information about the progress now
     
    SugoiDev likes this.
  9. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    In this video I show you my first iteration on providing an API that allows you do add custom code to Heap Explorer, to create your own Heap Explorer Views and functionality.

    Please let me know what you think.

     
    SugoiDev likes this.
  10. JJJohan

    JJJohan

    Joined:
    Mar 18, 2016
    Posts:
    214
    Thanks for providing this tool Peter, the alpha builds have been a fantastic help for me - I do apologise for not providing feedback. It is a bit of a shame that the memory profiler initially really struggles with most footprints above a gigabytes or so (in terms of load time during 'receiving memory...' dialog) but I realise that's more due to how Unity provides the data since their memory profiler has the same issue.
     
    Last edited: Aug 13, 2018
  11. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Hello,

    I have backported our project from latest Unity 2017.4.9f1 LTS into Unity 2017.4.6f1 LTS but it still crashes, have you seen a issue like that and have any potential solutions?

    0x000007FEE5A63AC6 (mono) [c:\buildslave\mono\build\unity\unity_memory_info.c:31] ContextInsertClass
    0x000007FEE5A63BB2 (mono) [c:\buildslave\mono\build\unity\unity_memory_info.c:75] CollectAssemblyMetaData
    0x000007FEE5917985 (mono) [c:\buildslave\mono\build\eglib\src\glist.c:255] g_list_foreach
    0x000007FEE5929D94 (mono) [c:\buildslave\mono\build\mono\metadata\assembly.c:2622] mono_assembly_foreach
    0x000007FEE5A63E6A (mono) [c:\buildslave\mono\build\unity\unity_memory_info.c:179] CollectMetadata
    0x000007FEE5A64364 (mono) [c:\buildslave\mono\build\unity\unity_memory_info.c:432] mono_unity_capture_memory_snapshot
    0x00000001409371F2 (Unity) MemorySnapshots::CaptureMemorySnapshot
    0x000000014120B20F (Unity) EditorProfilerConnection::SendGetMemorySnapshot
    0x0000000022F3377A (Mono JIT Code) (wrapper managed-to-native) UnityEditorInternal.ProfilerDriver:RequestMemorySnapshot ()
    0x0000000022F3368A (Mono JIT Code) [C:\buildslave\unity\build\Modules\Profiler\Editor\ProfilerWindow\MemorySnapshot.cs:16] UnityEditor.MemoryProfiler.MemorySnapshot:RequestNewSnapshot ()
    0x0000000022F32C92 (Mono JIT Code) [C:\Users\crash\Documents\unityheapexplorer\Unity\Assets\HeapExplorer\Editor\Scripts\HeapExplorerWindow.cs:779] HeapExplorer.HeapExplorerWindow:CaptureAndSaveHeap ()
    0x0000000022F328C8 (Mono JIT Code) [C:\buildslave\unity\build\artifacts\generated\common\editor\GenericMenuBindings.gen.cs:121] UnityEditor.GenericMenu:CatchMenu (object,string[],int)
    0x0000000022F325A4 (Mono JIT Code) (wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_object_int (object,intptr,intptr,intptr)
    0x000007FEE5A364D3 (mono) [c:\buildslave\mono\build\mono\mini\mini.c:4937] mono_jit_runtime_invoke
    0x000007FEE5988A2D (mono) [c:\buildslave\mono\build\mono\metadata\object.c:2623] mono_runtime_invoke
    0x0000000140A2F89C (Unity) scripting_method_invoke
    0x0000000140A2311A (Unity) ScriptingInvocation::Invoke
    0x0000000141183473 (Unity) CustomPopupMenuInterface::Execute
    0x00000001411610C9 (Unity) MenuController::ExecuteMenuItem
    0x0000000141407149 (Unity) ShowDelayedContextMenu
    0x00000001413FB95B (Unity) GUIView::OnInputEvent
    0x000000014112C65C (Unity) GUIView::processInputEvent
    0x00000001413F3665 (Unity) GUIView::processEventMessages
    0x00000001413FD65A (Unity) GUIView::GUIViewWndProc
    0x00000000770D9BBD (USER32) UserCallWinProcCheckWow
    0x00000000770D98C2 (USER32) DispatchMessageWorker
    0x0000000141410FC0 (Unity) MainMessageLoop
    0x00000001414128DC (Unity) WinMain
    0x0000000141E6A578 (Unity) __tmainCRTStartup
    0x00000000771D59CD (kernel32) BaseThreadInitThunk
    0x000000007733383D (ntdll) RtlUserThreadStart
     
  12. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I haven't seen this crash before. From what platform did you try to capture a memory snapshot, I would like to give it a try? Have you tried to capture a snapshot from different platforms, did it work there?

    It looks like it's crashing deep in Unity's code, perhaps @MartinTilo can provide help here. I believe tt would be from advantage if you can submit a bug-report to Unity Technologies with a test/project that they can use to reproduce the issue and test their fix against.
     
  13. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
  14. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I doubt I can help, because the crash occurs in Unity's code. According to the call-stack you posted earlier, it looks to me only Unity Technologies would be able to fix that issue. It's very unfortunate, because the issue exists in their LTS release, which stays around for almost two further years.

    From my experience, if Unity Technologies don't have a project to reproduce the issue, they're most likely not going to take a look at the problem. Which is why I asked if you can submit a bug-report, but I understand that it might not be possible due to size or NDA.

    Sorry, can't help :(
     
  15. alexrvn

    alexrvn

    Unity Technologies

    Joined:
    May 16, 2017
    Posts:
    53
    @GloriaVictis that is a crash inside mono during the class metadata collection step of a memory capture, can you submit a bug report about this ? :D Just to have it tracked...I believe I have some ideas as to how that can be reproduced even without a repo case :) (if NDA prevents you from submitting one)


    if you can't submit a repo from your own project, another way would be creating a script that mimics the memory footprint of your project (eg: stored data, gameObject present at runtime etc)
     
    sharkapps, GloriaVictis and Peter77 like this.
  16. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    I am a CEO of company, so lets say NDA might not be a big issue on my case, just please let me know, since we have not been assigning almost entire project (I cannot strip it down and keep the issue) - its gonna be available to download only for Unity developers and not for random people right?
     
  17. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Here is how I submitted big projects, that were under NDA, to Unity Technologies.
    1. Let Unity sign a NDA first.
    2. Submit a bug-report without the actual project attached first and ask their QA for a password protected owncloud link. Tell them the actual size of the file you need to upload. The bug-report needs to have full information on how to reproduce the issue already, just not the project attached.
    3. Ask them not to reply with the password in the bug-report, but to send it directly to your email address.
    4. Upload your project to the password protected owncloud location.
    5. Add a note to the bug-report, that you uploaded the project.
    6. Post the bug-report case number in this thread for alexrvn to pick up (post the number only, not the actual link) or send him a private message.
    7. Wait for them to fix the issue.
     
    GloriaVictis and MartinTilo like this.
  18. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Thank you, sent them a bug-report without project. Fingers crossed for a fast reply.
    Also I have tried stripping our scene from all scripts - profiler works after that so is seems to be somehow scripts or references related.
     
  19. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I recommend to post the Case number from the bug-report confirmation (the number, not the link). This allows Unity staff, who are watching this thread, to pick it up earlier or perhaps poke QA ;)
     
    MartinTilo likes this.
  20. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,424
    Thanks for submitting it and thanks Peter, you where faster than me :D
    While that's getting processed: have you tried building your server as IL2CPP build and capturing that?
     
    Peter77 likes this.
  21. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Would it be possible for you to leave a message in this thread, if this issue has been fixed? Thanks in advance!
     
  22. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,424
    @Peter77 I've managed to find it even without the case number (which is 1071466 for anyone who cares) and added a comment to update in this thread once solved. I'm subscribed to the case but won't be the one fixing it (it's not UI but backend) and I've pinged @alexrvn
     
    Peter77 likes this.
  23. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Just wanted to say that we were finally been able to take a snapshot after doing a custom scene with like 10% of our terrains, spawners etc. and take a snapshot using your tool. It led us to many issues we have been able to track down - we still have some to track down but I have to admit that your tool is absolutelly amazing.

    What I would suggest, is to put it on Asset Store because when 2017.3 new memory profiller will happen, not everyone will switch to that (we won't) and your tool is a no-brainer.

    Saw ILOptimizer - it sounds very interresting, but first we need to finish tracking down the bugs we found by your tool.

    Cheers!
     
    Peter77 likes this.
  24. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Hi!

    For some reason while getting a heap it started to throwing those errors, I have checked and I have 30gb of disk space and ram left, so I have no idea whats happening. Have you seen a issue like that ?

    OutOfMemoryException: Out of memory
    HeapExplorer.PackedConnection.FromMemoryProfiler (UnityEditor.MemoryProfiler.Connection[] source) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedConnection.cs:76)
    HeapExplorer.PackedMemorySnapshot.FromMemoryProfiler (UnityEditor.MemoryProfiler.PackedMemorySnapshot source) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshot.cs:58)
    UnityEngine.Debug:LogException(Exception)
    HeapExplorer.PackedMemorySnapshot:FromMemoryProfiler(PackedMemorySnapshot) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshot.cs:71)
    HeapExplorer.HeapExplorerWindow:OnHeapReceivedSaveOnly(PackedMemorySnapshot) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:795)
    UnityEditor.MemoryProfiler.MemorySnapshot:DispatchSnapshot(PackedMemorySnapshot)



    NullReferenceException: Object reference not set to an instance of an object
    HeapExplorer.HeapExplorerWindow.OnHeapReceivedSaveOnly (UnityEditor.MemoryProfiler.PackedMemorySnapshot snapshot) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:796)
    UnityEditor.MemoryProfiler.MemorySnapshot.DispatchSnapshot (UnityEditor.MemoryProfiler.PackedMemorySnapshot snapshot) (at C:/buildslave/unity/build/Modules/Profiler/Editor/ProfilerWindow/MemorySnapshot.cs:24)
     
    Peter77 likes this.
  25. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    That's excellent, thank you!

    I'm glad it proves useful. Thanks for the positive feedback!

    Nope, the OutOfMemoryException is new to me.

    The code for that is rather simple though, it's just a "new PackedConnection[size]" call, where "size" is the actual size of the connections array as found in the captured Unity MemorySnapshot. My best guess would be that the connections array is either really huge or contains a garbage length.

    It's probably huge, because I still believe there is a major bug in Unity's MemoryProfiling API that affects the connections array, which I reported as:
    (Case 987839) PackedMemorySnapshot: Unexpected connections between native objects

    ... which was unfortunately closed as "By Design". The issue is that Unity collects in real-world projects tens-of-millions of redundant connections. This is really just reproducible in an actual project, so not an easy reproduce for Unity.

    I uploaded a new Heap Explorer build (same link as before), where I log the array length in case of an exception now:
    Code (CSharp):
    1. public static PackedConnection[] FromMemoryProfiler(UnityEditor.MemoryProfiler.Connection[] source)
    2. {
    3.    PackedConnection[] value = null;
    4.    try
    5.    {
    6.        value = new PackedConnection[source.Length];
    7.    }
    8.    catch
    9.    {
    10.        Debug.LogErrorFormat("Failed to allocate PackedConnection[{0}].", source.Length);
    11.        throw;
    12.    }
    13.  
    If you could try this version and run in an out of memory, we can at least see the array length. If you have an .exe file that I could use to reproduce the issue, that would probably help me too.

    If you don't see the error in the Console, please disable in Heap Explorer "File > Settings > Use Multi-Threading".

    Thanks for the feedback!
     
  26. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Failed to allocate PackedConnection[509117918] - it indeed seems huge, what actually bothers me that it started to happen lately which makes me feel like something bad happened.

    So basicly, we cannot do a memory sample from our project, and we need to strip it about 80% size everytime we want to test it and now, even that is not working due to those connections overlap. Damn, I really feel like we are wasting half of our developement time on stupid bugs like that.

    Sadly I have never got any reply about my raported bug we have been speaking earlier from Unity.

    Btw any chance your tool might give some informations about those connections, of course from sample which is actually working ?
     
    Last edited: Aug 29, 2018
  27. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    That's what I expected. The problem is that Unity starts to connect "everything with everything" in more complex projects, causing an insane amount of (redundant) connections, in your case 509.117.918 :confused:

    @MartinTilo If you capture memory of a rather complex real-world project with HeapExplorer or MemoryProfiler, you should be able to observe that a lot of native objects have the same number of references.

    For example, native mesh objects have 700 references to all sorts of different unrelated Components in one memory snapshot an user sent me. This really does not seem right to me, if a Mesh has a reference to a AudioSource for example.

    I tried to provide a tiny test case that should explain the problem, which I reported as:
    (Case 987839) PackedMemorySnapshot: Unexpected connections between native objects

    Perhaps you could have look at the ticket? Heap Explorer is now also available for free, you could give it a try with a real project as well. This connection issue is really the only major problem left in the old MemoryProfiling API and I would really like to see this issue sorted out.

    The connections are displayed as "References" in the C++ Objects view. I guess, if you go through those native objects list, at one point you observe a lot of objects have the same number of references (like hundreds of refs).
     
  28. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I uploaded a new build, same link as before.

    In order to workaround the major connections bug, where Unity provides more than five hundred million connections, I implemented a feature in Heap Explorer to exclude native object connections. You can activate it from Heap Explorers "File > Settings" menu. It's called "Exclude NativeObject connections when capturing a memory snapshot".

    Unity still captures those connections, but Heap Explorer will not process most of them. While this means Heap Explorer is unable to show which native objects are connected, it’s perhaps still better than not being able to capture a snapshot at all.

    I hope this allows you to inspect a memory snapshot again.
     
  29. MildaGenius2

    MildaGenius2

    Joined:
    May 6, 2013
    Posts:
    19
    Hello,
    I want to test your HeapExplorer 2.6.

    But after Capture and Analyze Editor or Captute and save and then Open profiler, I got this exception.

    Code (CSharp):
    1. IndexOutOfRangeException: Index was outside the bounds of the array.
    2. HeapExplorer.HeEditorUtility..cctor () (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/HeEditorUtility.cs:19)
    3. Rethrow as TypeInitializationException: The type initializer for 'HeapExplorer.HeEditorUtility' threw an exception.
    4. HeapExplorer.HeapExplorerWindow+<>c.<DrawToolbar>b__61_8 () (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:587)
    5. UnityEditor.GenericMenu.CatchMenu (System.Object userData, System.String[] options, System.Int32 selected) (at /Users/builduser/buildslave/unity/build/Editor/Mono/GUI/GenericMenu.cs:121)
    6.  
     
  30. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Interesting, this happens while parsing the Unity version. Here is the relevant code.
    Code (CSharp):
    1. public static class HeEditorUtility
    2. {
    3.     static int s_Major;
    4.     static int s_Minor;
    5.  
    6.     static HeEditorUtility()
    7.     {
    8.         var splits = Application.version.Split(new[] { '.' });
    9.         int.TryParse(splits[0], out s_Major);
    10.         int.TryParse(splits[1], out s_Minor); // this causes the IndexOutOfRangeException
    11.     }
    Could you please post which Unity version you're using. You can see this in "Help > About Unity...". A screenshot of the About dialog would be best.

    Additionally, what language is your operating system? Perhaps Application.version value isn't delimited with a point, but with a comma in your language?

    Do you use Windows or Mac?
     
  31. jmedveckyh1

    jmedveckyh1

    Joined:
    Jan 15, 2018
    Posts:
    5
    Hello, I got something similar when trying to Capture & Analyze in Unity 2018.2.6f1 in an existing project. The tool works OK if i create a new project. Maybe these stacktraces help you identify the problem:
    Code (CSharp):
    1. IndexOutOfRangeException: Index was outside the bounds of the array.
    2. HeapExplorer.PackedManagedObjectCrawler.InitializeCachedPtr () (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedManagedObjectCrawler.cs:68)
    3. HeapExplorer.PackedManagedObjectCrawler.Crawl (HeapExplorer.PackedMemorySnapshot snapshot) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedManagedObjectCrawler.cs:48)
    4. HeapExplorer.PackedMemorySnapshot.Initialize () (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshotEx.cs:1031)
    5.  
    and this one:
    Code (CSharp):
    1. System.IndexOutOfRangeException: Index was outside the bounds of the array.
    2.   at HeapExplorer.PackedManagedObjectCrawler.InitializeCachedPtr () [0x00000] in C:\Users\crash\Documents\unityheapexplorer\Unity\Assets\HeapExplorer\Editor\Scripts\PackedTypes\PackedManagedObjectCrawler.cs:68
    3.   at HeapExplorer.PackedManagedObjectCrawler.Crawl (HeapExplorer.PackedMemorySnapshot snapshot) [0x00020] in C:\Users\crash\Documents\unityheapexplorer\Unity\Assets\HeapExplorer\Editor\Scripts\PackedTypes\PackedManagedObjectCrawler.cs:48
    4.   at HeapExplorer.PackedMemorySnapshot.Initialize () [0x00035] in C:\Users\crash\Documents\unityheapexplorer\Unity\Assets\HeapExplorer\Editor\Scripts\PackedTypes\PackedMemorySnapshotEx.cs:1019
    5. UnityEngine.Debug:LogError(Object)
    6. HeapExplorer.PackedMemorySnapshot:Error(String, Object[]) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshotEx.cs:78)
    7. HeapExplorer.PackedMemorySnapshot:Initialize() (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshotEx.cs:1030)
    8.  
    but if I capture and save and then try to open the snapshot i get this:

    Code (CSharp):
    1. Exception: snapshot.managedHeapSections array mus not be empty.
    2. HeapExplorer.PackedMemorySnapshot.LoadFromFile (System.String filePath) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshot.cs:157)
    3. UnityEngine.Debug:LogException(Exception)
    4. HeapExplorer.PackedMemorySnapshot:LoadFromFile(String) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/PackedTypes/PackedMemorySnapshot.cs:167)
    5. HeapExplorer.HeapExplorerWindow:LoadFromFileThreaded(Object) (at C:/Users/crash/Documents/unityheapexplorer/Unity/Assets/HeapExplorer/Editor/Scripts/HeapExplorerWindow.cs:733)
     
  32. PsyKaw

    PsyKaw

    Joined:
    Aug 16, 2012
    Posts:
    102
    Yes, same errors here on existing project :(
     
  33. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I tested the Survival Shooter Tutorial with 2018.2.6f1, but I'm unable to reproduce the error. Perhaps the issue is related to a specified project setting, or to the project itself.

    A few questions about the project, which could help to reproduce the issue. If you go to the Player Settings and expand "Other Settings", what do you use for the following options:
    • Scripting Runtime Version
    • Scripting Backend
    • Api Compatibility Level
    • C++ Compiler Configuration
    • Allow 'unsafe' code
    What target platform do you try to capture a memory snapshot from and on what OS runs the editor?
     
  34. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I was able to reproduce this issue when using the ".NET 4.x Equivalent" Scripting Runtime Version using 2018.2.6f1. I'm looking into the issue now.
     
    Last edited: Sep 9, 2018
  35. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    It turns out, this is an issue in Unity's MemoryProfiling API unfortunately, rather than in Heap Explorer.

    I've submitted a bug-report to Unity Technologies, to allow them to take a look at the issue and hopefully come up with a fix. In the meantime, I'm integrating better error messages in Heap Explorer.

    The problem is related to the Scripting Runtime Version. .NET 3.5 seems to work fine, but .NET 4.x causes problems with various Unity versions. I reproduced the issue with Unity 2017.4.10f1, 2018.2.6f1 and 2018.2.7f1.

    (Case 1079363) PackedMemorySnapshot: .NET 4.x Scripting Runtime breaks memory snapshot

    Using the .NET 4.x Scripting Runtime, causes empty typeDescriptions, managedHeapSections and gcHandles arrays in a PackedMemorySnapshot.

    This issue exists in 2017.4.10f1 where .NET 4.x is marked as experimental, as well as on Unity 2018.2.6f1 where .NET 4.x is considered stable.

    Reproduce
    • Open user attached project
    • Click "Mainmenu > Edit > Project Settings > Player" and expand the "Other Settings" tab
    • Notice the "Scripting Runtime Version" is set to .NET 4.6
    • Click "Mainmenu > Bug Report > Open TestCode Window"
    • Click "Capture" button in TestCode Window
    Observe it outputs typeDescriptions = 0, managedHeapSections = 0, gcHandles = 0.
    • Click "Mainmenu > Edit > Project Settings > Player" and expand the "Other Settings" tab
    • Change the "Scripting Runtime Version" to .NET 3.5
    • Click "Mainmenu > Bug Report > Open TestCode Window"
    • Click "Capture" button in TestCode Window
    Observe typeDescriptions, managedHeapSections, gcHandles were found.


    Expected
    Using the .NET 4.x Scripting Runtime Version should not break the PackedMemorySnapshot.

     
    Last edited: Sep 11, 2018
  36. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I uploaded version 2.7, same link as before. The download link can be found here.

    Changelog
    • Added warning if project uses .NET 4.x Scripting Runtime, because Unity’s MemoryProfiling API does not work with that at the time of writing (more).
    • Added better error messages for known memory snapshot issues, like empty typeDescriptions array. (more)
    • Added error message dialog and then quit Heap Explorer if an unrecoverable error occurs.
    • Added error message if Heap Explorer is unable to parse the editor version, to get at the bottom of this problem. (more)
     
  37. jmedveckyh1

    jmedveckyh1

    Joined:
    Jan 15, 2018
    Posts:
    5
    hey, just wanted to thank you for your work and mention that it works now in the new beta (2018.3.0b1)
     
    Peter77 likes this.
  38. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Thanks for letting me know!
     
  39. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Unity QA replied to the bug-report:
    Thus, if you're using .NET 4.x Scripting Runtime, Unity 2017.4 LTS cannot be used to capture a memory snapshot. You have to use 2018.3 or newer instead.
     
  40. z3nth10n

    z3nth10n

    Joined:
    Nov 23, 2013
    Posts:
    55
    This asset is amazing, I'm currently learning the differencies between assign variables on the heap vs stack, to know what should I attack on my project to trying optimize it.

    https://gitlab.com/uta-gi/gta-mapper-tests

    And this is what I get:



    I'm reading a PNG that has 7000x5000 pixels and casting it into a custom Color struct (it's type length is about 4 bytes). So, 7000x5000=35M*4=105M, I'm cloning it two times to 210M of bytes=200,3MB

    Knowing what actually I'm doing it's easier to optimize my app.

    Maybe somebody is interested on the Benchmark snippet I did:


    Code (CSharp):
    1. Getting report from GTAMapperTests.Utilities.Benchmarks.TextureBenchmark
    2.  
    3. ------------------------------
    4.  
    5. Ordered by AZ:
    6.  
    7. ToArrayStack: 4.98 s * 1 = 4.98 s (~48.685 %)
    8. ToArrayHeap: 5.01 s * 1 = 5.01 s (~48.969 %)
    9. LoadingTexture: 95.91 ms * 1 = 95.91 ms (~0.937 %)
    10. LoadingResource: 51.66 ms * 1 = 51.66 ms (~0.505 %)
    11. ElementAt: 0.73 ms * 1 = 0.73 ms (~0.007 %)
    12. Destroying: 7.45 ms * 1 = (7 ticks) >7.45 ms (~0.073 %)
    13. CloningStack: 41.91 ms * 1 = 41.91 ms (~0.410 %)
    14. CloningHeap: 41.81 ms * 1 = 41.81 ms (~0.409 %)
    15. Casting: 0.56 ms * 1 = 0.56 ms (~0.005 %)
    16.  
    17. ------------------------------
    18.  
    19. Ordered by Desdescing:
    20.  
    21. Casting: 0.56 ms * 1 = 0.56 ms (~0.005 %)
    22. ElementAt: 0.73 ms * 1 = 0.73 ms (~0.007 %)
    23. Destroying: 7.45 ms * 1 = (7 ticks) >7.45 ms (~0.073 %)
    24. CloningHeap: 41.81 ms * 1 = 41.81 ms (~0.409 %)
    25. CloningStack: 41.91 ms * 1 = 41.91 ms (~0.410 %)
    26. LoadingResource: 51.66 ms * 1 = 51.66 ms (~0.505 %)
    27. LoadingTexture: 95.91 ms * 1 = 95.91 ms (~0.937 %)
    28. ToArrayStack: 4.98 s * 1 = 4.98 s (~48.685 %)
    29. ToArrayHeap: 5.01 s * 1 = 5.01 s (~48.969 %)
    30.  
    31. ------------------------------
    32.  
    33. Total time: 10.23 s of 16.59 s (~61.66%) Avg: 1.14 s | Calls: 9 | Avg / per call: 126.31 ms
    Look BenchmarkData class. :p

    Thanks!
     
    Peter77 likes this.
  41. MildaGenius2

    MildaGenius2

    Joined:
    May 6, 2013
    Posts:
    19
    Tested in Unity 2018.1.0f2 on Mac 10.13.6 with preferred language English.
     
  42. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    If you download the latest Heap Explorer version, do you still get this error message? I added some code around this to hopefully provide additional information if an error occurs. Can you test this again? Thank you!
     
  43. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Thanks for the kind words. Glad you find it useful!
     
  44. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    We recently started having a issues that Unity simply closes after taking a sample from our big scene, but it seems to be happening on various Unity and Heap Explorer versions, anyone had a issue like this before? Sadly nothing on log file.
     
  45. rob_bravoco

    rob_bravoco

    Joined:
    Dec 19, 2017
    Posts:
    2
    This is extremely good.
     
    Peter77 likes this.
  46. Its4u

    Its4u

    Joined:
    Oct 11, 2013
    Posts:
    14
    Thanks for awesome asset. It worked perfectly with iOS device, but I got Unity crashed when try to capture in editor play mode (Unity version 2017.4.13f1 and Heap Explorer 2.7).
     
  47. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    If you can reproduce the issue, I think it would be extremely helpful if you would submit a bug-report to allow Unity Technologies to take a look at this problem.

    Thanks for the nice words, great that the tool proves to be useful! :)

    If you can reproduce the issue, could you please submit a bug-report to allow Unity Technologies to take a look at this problem.

    Thanks a lot for the feedback!
     
    Its4u likes this.
  48. GloriaVictis

    GloriaVictis

    Joined:
    Sep 1, 2016
    Posts:
    133
    Actually I have not got any reply for a bug on that matter (overall crashing on big scene) for like two months already when I asked them for NDA stuff sadly.
     
  49. MildaGenius2

    MildaGenius2

    Joined:
    May 6, 2013
    Posts:
    19
    We switched to .NET 4.x so I am not able test it again in the same project :(.
     
  50. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    Which Case number is that? Is it Case 1071466? Can you post the number here, because then we can poke Martin and kindly ask him to poke someone from QA for us.