Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Help needed to find out what my memory leak is

Discussion in 'Editor & General Support' started by marcrem, Jan 17, 2021.

  1. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Hi,

    I'm using the amazing asset "Graphy" to monitor my performances in-game. There is 3 things to watch as far as memory is concerned.
    - Reserved
    - Allocated
    - Mono

    While my Reserved and Allocated values are all fine over the course of the runtime session, my Mono keeps going up, sometimes clearing a couble MB, but not enough to go back down to what it was when the game opened.

    This makes me think I have a memory leak or a problem with Garbage Collection in my project.

    What are the steps you would recommend for me to find the culprit? Where should I start? What would you do?

    I have to be honest, I have taken snapshots in the memory profiler, but I don't really understand how to analyze them.

    I really hope you guys can help :)
    Cheers
     
  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Which unity version are you using?

    For 2020.2 and up, you can use the ProfilerRecorder API to track the same values as shown in the Memory Profiler module of the Profiler Window (find the counter name on the Module's manual page I linked to)

    For earlier versions you have
    - Reserved
    - Allocated
    - Mono
    Beyond that you should probably dig deeper than these high-level numbers using the Memory Profiler Package (which hasmayor UI UX overhaul pending which should make this all easier to gorg) or the HeapExplorer.
     
  3. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    I should've mentionned, I'm on 2019.4 (LTS). I'll dig in and try to show more details as to what could cause memory to constantly increase. Heap Explorer seems nice!

    Reading the page you linked for GetMonoUsedSizeLong, I noticed the following:
    "Note that this will return an ever increasing value until GC.Collect() is called."

    I enabled the incremental GC in my project, maybe I shouldn't have?

    Thanks!
     
  4. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    incremental GC will also lower the used size by freeing up resources, just not immediately but over a couple of frames
     
    Joe-Censored likes this.
  5. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Thanks I'll get back to you with some data!
     
  6. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Here is a memory snapshot after about 15 minutes of playing, when it reached about 10GB in mono
    memoryprofiler.png

    This first value down there has 1.5 million references? Huh...
     
  7. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    That looks like a Transform named Group which probably has quite a few child elements?

    I'd start by looking at the biggest items there though. It's quite some textures and a somewhat odd total of 2d int32 arrays ( int32 [,]) and other multidimensional arrays, which would more directly tie to the mono memory growing, unless all hose textures re basically just managed shell objects without a native object underneath them.
     
  8. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Here, I used the profiler to see what was piling up in my memory: memory.png
     
  9. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    The Profiler Window's detailed snapshot does not capture any details of the Managed Heap. To see those details you have to use the Memory Profiler Package. In there, if you want to focus on the Managed Heap, you can select the view: Tables > All Managed objects.

    As per your previous screenshot, it looks like it might be down to quite a few multidimensional arrays. Check if all of those need to be in memory, who is referencing them and if that all looks correct to you.
     
  10. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Thanks for your help.

    I went to tables -> all managed objects.

    I can't seem to find anything helpful in there?


    upload_2021-2-28_20-42-49.png
     
  11. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    You can click the column headers to filter them. It's too hidden for most and that will be addressed in a future release of the package :)
     
  12. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Alright, I'm really trying and I can't seem to understand anything about this memory profiler :(

    The only place where I found a clue (that still doesn't make sense) is in the regular profiler.
    upload_2021-3-1_14-4-51.png

    Here, I have 1300-something references to each terrain texture, and it's referenced by objects that have nothing to do with it and no reference to either a terrain, or a texture, in its code. Why is my "OBK_Destructible" component mentionned in the "referenced by" tab referencing to a terrain texture? It is absolutely black magic for me. Doesn't make any sense.

    Now I still am trying to use the memory profiler. I went to "All managed objects" and sorted them using filters according to what you said in your previous post (I don't know what I'm looking for though, so pretty hard to even know if I'm looking at the right place).

    Tree:
    upload_2021-3-1_14-11-0.png

    Here's the result:
    upload_2021-3-1_14-7-31.png

    I'm sorry for my ignorance, I'm really trying!!
     
  13. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    No worries, the UI is really suboptimal and something I have to apologize for. Have you tried clicking the Type column and searching for the classes of your multidimensional arrays? Also, you can click on that RefCount in there to see the connections there, even if only one layer at a time instead of as a tree as in the right column of the memory Profiler module.
     
    Last edited: Mar 1, 2021
  14. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Btw, the references to the terrain might be just down to the Hierarchy structure or collision detection?
     
  15. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Also 107 Render Textures sounds like a bit unusually high unless you are doing something unusual with them I guess?

    If your memory usage count is continuously rising, try taking two snapshots in quick succession (but enough time passing for the numbers to rise) then open both snapshots and diff them. In the diff mode, check the All Managed Objects table and group the Diff column and check the new objects. Chances are whatever you are leaking is something you are continuously creating anew and not loosing all references to them afterwards.
     
  16. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    You also have some Texture2D Objects without a name, so if you are creating new textures from code, make sure to assign a helpful value to their
    .name
    field to make tracking them easier.

    Do you have any Texture2D Objects in the Managed Objects table that have
    m_CachedPtr = 0
    ? That would indicate leaked managed shell objects, i.e. you called Destroy() on them, deleting the native texture memory, but are still holding a reference to their scripting reference wrapper. Might be some references need to be set to null? Their blue RefCount link text should be clickable and tell you what is holding on to them.
     
  17. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Alright, so here's what it looks like a couple seconds between the 2 snapshots:
    upload_2021-3-6_20-38-52.png

    I do have a bunch of those:
    upload_2021-3-6_20-39-35.png

    When I click on them though, it doesn't really help me to find what holds what in memory:
    upload_2021-3-6_20-41-7.png

    I can't even know what the texture is, so it's pretty hard to find the source of the memory "leak".
     
  18. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Click on the blue 1 in the RefCount column. That should show you what is holding the reference to this empty managed shell, and therefore point to the culprit of the leak.
     
  19. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Keep in mind that empty managed shell objects appear as fake null because UnityEngine.Object overrides == operator and implicit cast to bool to show that it the Object is null or false respectively, when it is only an Empty shell that had it's native part Destroyed.
     
  20. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Hi,

    Thanks for your help, it's very much appreciated. My case is very hard to deal with so I'm gonna give some more details on what's going on.

    I'm not sure if it's normal to see this, but when I'm not moving, my "mono" (using graphy for real time stats) is constantly going up but it goes back down every now and then, so it actually never really stack memory, it looks like it's doing garbage collection after a couple seconds, which brings it down to what it was, or close to it. So unless you tell me it should be absolutely stable, I don't think it's a problem. That's also why comparing snapshots can be very misleading.

    My problem is when streaming new scenes, aka when my player moves around the world, something is not unloading from the unloaded scenes. What would you suggest to find out what's keeping stuff in memory? I have dozens of scenes being unloaded and loaded when I move in the world, so I need to find a way to see an unloaded object still in memory.

    Again, thank you so much!
     
  21. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Continuously allocating managed memory frame over frame when nothing is technically happening is not ideal, can contribute towards memory fragmentation, but unless you are on a very resource constrained device like a phone, is likely not going to be too important to focus on. There might be some interesting performance gains to be had with optimizing these out but if the GC can fully reclaim that memory, that's less of an issue. (The Profiler Window can help finding the sources for these allocations better. Clearing these out also might make finding the leaks here easier because there is less noise in the data)

    However, from those previous screenshots, it very much looks like you have some Managed Texture2D wrapper objects that something is holding on to while there is no native object underneath that anymore. The Managed wrappers here are only taking up 40 B so if Graphy is showing you numbers in rounded KB numbers or bigger, it might be hard to tell if the amount fully resets? If not, then as you mentioned, the issue is more likely to be happening around steaming in scenes.

    In that case, take a snapshot, before a new scene is streamed in and after it is streamed in. Filter the Managed Objects Table for Texture2D objects and check out which ones are the same in both, but where m_SharedPtr==0 in the second one. Btw, at least the address would be the same for them if you need to switch between the snapshots.

    Check the chain of references leading to these and see what's holding them in memory.