Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. The 2023.1 beta is now available for testing. To find out what's new, have a look at our 2023.1 feature highlights.
    Dismiss Notice

Learning to use the Memory Profiler package

Discussion in 'Profiler Previews' started by Livealot, Nov 11, 2021.

  1. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    I'm an instant fan of the experimental Memory Profiler package (0.4.1 with 2020.3 LTS). The first time I tried it I found a memory leak that was crashing our game. Hurrah!

    Now I'm back looking for more and getting confused. It looks like the docs are lagging the package so wanted to ask some basic questions as a new user. Here's a screenshot from comparing two snapshots, before and after running some save game logic.

    Screen Shot 2021-11-05 at 8.55.22 AM.png

    1. From the top right, I see a total increase from 1.02 to 1.05 GB, so I'm expecting to find 30 MB of detailed differences somewhere. The Managed Heap grows 10MB, but I don't see anything else to account for the other 20MB.
    2. In the Table View, it would be great to get an explanation of Managed vs Native Size columns and how to use them, especially since they are so different.
    3. In the bottom right, I thought it made sense until I noticed the units. 285MB of Native Memory was deleted but only 365KB of Native Memory was added. That seems like a huge net decrease that would show up in the summary above somewhere
    4. (Not shown) I do lots of group filtering, but have got confused several times when comparing references to managed size. When expanding a group (clicking a down arrow) sometimes the sub-totals add up and sometime they don't, which is a real head scratcher.
     
    codestage likes this.
  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    happy to hear that! :D

    Actually, the managed heap usage grew, the heap itself (reserved size) is still the same size so that wouldn't affect the Total amount of Memory occupied by Unity... Sadly that makes it even less clear where that increase came from

    Also, technically only something like 21 MB difference would be needed depending on the exact numbers and rounding. Also, I just checked and there is a minor of precision loss going on in EditorUtility.FormatBytes(), which is used to display all Byte sizes in the Profiling tools. Basically it does int divisions by 1024 until the value is in range of the right order of magnitude and only then does a division as double and passes it on to formatting with rounding, using IEEE 754 rounding rules.

    That all said, this does look rather off. If you could file a bug report with these two snapshots attached and ping me the issue number, I can check out if there is some further imprecision going on that we can fix or how to show that difference better.

    Yeah, you're not the first one to be confused by those size columns. We really do need to clarify them. We had already renamed them a bit but will simplify the size display and clarify the columns as we overhaul the tables in general.

    But basically: you are looking at the "All Objects Table", so that's a mixture of Managed Objects (all objects on the Scripting Heap) and Native Unity Objects (All C++ Objects inheriting from UnityEngine.Object, or rather, it's C++ equivalent).
    All Unity Objects (i.e. originating from types inheriting from UnityEngine.Object) can have a Managed Shell Object, i.e. a Managed Object that allows C# code to interact with them.

    So Unity Objects can consist of:
    • 0 or 1 Native Object
    • 0 or 1 Managed Shell Object
    Thus, they can have a Managed Size and a Native Size. Also, all pure C# Type objects (not inheriting from UnityEngine.Object) will obviously also only have a Managed Size.

    The Field Target size only applies to fields on Managed Objects, i.e. if you fold out a Managed Objects, you can see for each field the Managed Size that is inherent to this object. If it is a reference field to an Object, the Managed Size of that field is the size of the pointer, and the Field Target Size the size of the referenced Object. Since that Other Object will appear elsewhere in the table (at least in an unfiltered All Objects or All Managed Objects) table, this avoids double counting the memory. We want to simplify this though, especially for cases where the referenced object has nothing else referencing it, so it can technically be counted as "owned" by this object.

    Unfortunately, the current All Object Table logic only partially stitches the Managed Shell Object and the Native Objects together. The result is that a Unity Object that has both will show the Native Size on the Managed Shell AND on the Native Object, as both will appear in the table.

    This means that the Native Size summary show for Group headers like your Diff table groups of "Same" "New" "Deleted" may be inflated by double counting the native size for any Object that has a Managed Shell.

    That logic error needs fixing but that goes a bit deeper into the entire Table logic and is therefore scheduled to happen with the general table overhaul. So this should get simpler in the future.

    The double counting may only account for, at max, a doubling, so I guess something else is funky with that math... I can't tell what from this high level view. If I had the snapshots, e.g. via that bug report, I could try digging a bit deeper ...


    Is this happening while comparing snapshots or also while looking at just one? I think we need to take a long hard look at the tables and grouping logic but specifically with Diffing, I think that a group containing rows with mixed Diff column values might be funky on the group header summary with the sizes?
     
  3. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    That detail is very helpful @MartinTilo . I submitted the snapshots as a bug report, Case # 1381034
     
  4. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    Thanks for the report, I've taken a look and figured out where those missing 27MB went. The rounding was all accurate enough but the Executable And Dlls part was bigger than the otherwise untracked memory.

    Attributing all of that executable memory as considered-in-use-by-the-os (the Total value) is wrong and something we're looking into fixing. It's quite the messy situation across platforms and not always possible to get good numbers, and some platforms don't include the Executable And DLL size in the total in use memory stat at all. I was already working on a UI fix/temporary workaround that would at least trim the usage down to what is maximally possible, but that needs some Highlighting and explanation in the UI as in e.g. your case it could then look like the Executable And DLL size grew while it was really either that more of it was considered dirty, or that the Untracked amount grew.

    So for your two snapshots, the usage grew by 27 MB in either the amount of executable memory that was considered dirty, or in Untracked (or a mix of both). But it's impossible to tell for the memory Profiler. You'd need to check this out with xCode Instruments
     
  5. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    I've made the bug public and found another issue somewhat related to the mismatch in New and Deleted, possibly causing the discrepancy there, or at least to a degree. Somehow the Snapshot Crawler in the package fails to properly link up the some of the Managed Shell Objects with the Native Objects in the second snapshot, so they don't have a Name or Native Size.

    Thanks again for bringing this up and reporting these and please ignore the part of my answer email where I forgot to change "Custumer QA" to "Developer in the Profiler Team" :D
     
    Last edited: Nov 23, 2021
  6. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    Happy to help! I'm glad the snapshots proved useful in finding bugs.

    When the documentation for the package gets updated, it probably would help to include a description of "Untracked Memory" so folks know what that includes and where they could explore further if that is the category that grows in the snapshot. Said another way, when looking for causes of memory crashes, it sounds like the Memory Profiler can identify most but not all potential sources. So it would be nice to have an overview of the types of causes that Memory Profiler can't identify.

    Thanks again for the response.
     
  7. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    Aye, we need to surface that info better. It currently lives in the Known Limitations section of the package's documentation index landing page. I'll also look at getting that info directly into the tool.
     
  8. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    I think the same doc suggestion applies to the "Other" category as well. What's in it? What should I do if I observe it increasing? etc.

    Ironically, in my latest memory crash hunt, the snapshots I just analyzed showed a 80 MB increase that was:
    1. 50 MB untracked
    2. 27 MB other
    3. 3 MB Managed Heap
    So diving in to just examine the 3MB portion seems silly.
     
  9. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    A separate issue I forgot to mention. At the bottom of the Summary View, there's a Top Issues section.

    I have an issue flagged there, but I'm not sure how to fix or implement the recommendation.

    I can't cut and paste, so paraphrasing, the warning is
    System Allocator is used. It is generally advised to use the Dynamic Heap Allocator instead.

    I'd love to. How? :)
     
  10. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    Yeah it applies to Others too, though since that bit is included in the data, it's a little less nebulous, though with the current snapshot data hard to specifically attribute to single objects and allocations. It's the Native Heap excluding Audio, Profiler and GPU side graphics parts. I.e. it includes the native sizes of Scene Objects, Assets (CPU side only for stuff like Textures), Subsystems etc. So generally looking at Native Objects should show that, but the other bug I found via your snapshots may make pinpointing a bit more tricky.

    The System Allocator warning currently doesn't work correctly on all platforms as there is an issue with the reporting of the memory regions. A fix for that is in review. Unless you went out of your way to enable the System Allocator, you can ignore that one for now.
     
    EirikWahl and Livealot like this.
  11. Livealot

    Livealot

    Joined:
    Sep 2, 2013
    Posts:
    228
    In ongoing tests, I'd like to clarify how the Garbage Collector comes into play.

    When testing in the Editor, there's a toggle under the Capture dropdown to "Run Garbage Collection before capturing Editor".

    I assumed that was the case when testing on device too, which is the suggested approach. But now I notice that the toggle is NOT there under the Capture dropdown when a Device is selected.

    @MartinTilo When taking snapshots from a device, is there a way to know whether the GC has run or not?

    Given the variability in our results, I'm wondering whether GC is the difference.
     
  12. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    Taking a snapshot on the device doesn't trigger the GC. If you want to be sure, you'd need to turn off incremental GC, call GC.Collect and GC.WaitForPendingFinalizers before triggering the capture via API.

    You won't see any managed memory in the snapshot that is about to be collected by the GC, because that memory has no rooted reference chains pointing at it that the snapshot crawler could follow to find it. So the only difference you'd see between before and after GC.Collect is that sometimes you might see less empty heap space.

    What variations are you seeing though?
     
    Last edited: Dec 16, 2021
  13. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    Ah yes and btw, 0.4.3-Preview.1 contains the fixes for the UI Total numbers not matching up to the totals of the bar's elements and for the managed and native objects not getting stitched together properly in that second snapshot you submitted. Thanks again for the report :)
     
  14. ZenTeapot

    ZenTeapot

    Joined:
    Oct 19, 2014
    Posts:
    60
    Is the situation changed with System Allocator warning? I still get those on 2021.3.15f, even though I've never tinkered with the default allocator.
     
  15. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,910
    As per the issue tracker item for that, it's fixed in 2022.2.0a1, the fix is not getting backported,so it wont get fixed on 2021.3. As before, just ignore the warning on those versions.