Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We are updating our Terms of Service for all Unity subscription plans, effective October 13, 2022, to create a more streamlined, user-friendly set of terms. Please review them here: unity.com/legal/terms-of-service.
    Dismiss Notice
  3. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice
  4. Join us on Thursday, September 29, for a day with Unity's SRP teams here on the forum or on Reddit, and discuss topics around URP, HDRP, and the Scriptable Render Pipeline in general.
    Dismiss Notice

Very odd Huge Memory Usage 40GB ManagedHeap.UsedSize !!!

Discussion in 'Scripting' started by tinyant, May 17, 2022.

  1. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    117
    Unity2020.3.20f1
    Running My app on WindowServer 2019,
    Every frame only 138Byte alloc, Everything goes right unity wil trigger gc after a few minutes,
    BUT After several Hours,The total memory increased So large,


    In Profiler I Just Find Only 136B memory alloc, But GC Used Memory just increase 4GB,I can't find where these big memory alloc.



    微信截图_20220517100057.png _20220517100057.png _20220517100222.png
     
  2. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    117
    My application start at
    [2022-05-16 17:24:36.754 +08:00]


    2022-05-16 23:01:45.072 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 9776KB 9.546875MB 5084KB 4.964844MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133149KB 130.0283MB -572.6602KB -0.5592384MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 195347KB 190.7685MB 572.6602KB 0.5592384MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 11432KB 11.16406MB 3628KB 3.542969MB

    2022-05-16 23:02:45.080 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 44840KB 43.78906MB 35064KB 34.24219MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133285.6KB 130.1618MB 136.6221KB 0.13342MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 195210.4KB 190.6351MB -136.6221KB -0.13342MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 84164KB 82.19141MB 72732KB 71.02734MB

    2022-05-16 23:03:45.108 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 99784KB 97.44531MB 54944KB 53.65625MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133285.6KB 130.1618MB 0KB 0MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 195210.4KB 190.6351MB 0KB 0MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 150216KB 146.6953MB 66052KB 64.50391MB

    2022-05-16 23:04:45.108 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 109968KB 107.3906MB 10184KB 9.945313MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133720KB 130.5859MB 434.3506KB 0.4241705MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 194776KB 190.211MB -434.3506KB -0.4241705MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 150216KB 146.6953MB 0KB 0MB



    2022-05-16 23:05:45.140 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 154524KB 150.9023MB 44556KB 43.51172MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133149KB 130.0283MB -570.9727KB -0.5575905MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 195347KB 190.7685MB 570.9727KB 0.5575905MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 281804KB 275.1992MB 131588KB 128.5039MB



    2022-05-17 04:03:49.415 +08:00 [INF] GCManager:MEM-----------------------
    l1 GetMonoUsedSizeLong: 4198112KB 4099.719MB 4KB 0.00390625MB
    l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
    l3 GetTotalAllocatedMemoryLong: 133866KB 130.7285MB 434.0928KB 0.4239187MB
    l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
    l5 GetTotalUnusedReservedMemoryLong: 194630KB 190.0684MB -434.0928KB -0.4239187MB
    l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
    l7 GetMonoHeapSizeLong: 4251432KB 4151.789MB 4KB 0.00390625MB



    Just MonoHeapSizeLong and GetMonoUsedSizeLong Increased Huge but GetTotalAllocatedMemoryLong GetTotalReservedMemoryLong GetTotalUnusedReservedMemoryLong seems normals


    I can't find why GetMonoHeapSizeLong GetMonoUsedSizeLong become so odd :(
     
    Last edited: May 17, 2022
  3. Bunny83

    Bunny83

    Joined:
    Oct 18, 2010
    Posts:
    2,212
    Your last screenshot doesn't seem to show a sample at a point where your memory usage actually makes those huge jumps up. You know that you can use the arrow keys to move one sample to the left / right? Turn on deep profiling and actually select one of the frames where a spike happens. You clearly have a huge increase at larger time intervals. Maybe a coroutine or invokeRepeating call.

    So you need to properly analyse the profiler results
     
    tinyant likes this.
  4. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    117

    I can't find so large memory usage in profiler.


    _20220519111927.png
     
  5. Bunny83

    Bunny83

    Joined:
    Oct 18, 2010
    Posts:
    2,212
    I don't get your new screenshot. It's still a frame you're showing which allocates 139kb. Here, I made a screenshot from your first screenshot.
    Profiler_Example.png

    The "red dot" is the memory usage of the frame you have selected. As you can see the line is almost straight (I overpainted the yellow line since it was below the tooltips). What you should select is a frame where the memory usage increases heavily. Like the pink line I drawn in to the left. Here the usage makes a huge jump. So have a look at those frames and see how much is actually allocated in those frames and what is allocating it. I don't know how you made the screenshot, but you can simply pause the game in order to analyse the profiler results.

    As I said above you can use the arrow keys to go one frame forwards / backwards in order to fine select a certain frame. Again, we can not do that for you. You have the project and profiler data.
     
    tinyant and Kurt-Dekker like this.
  6. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    117
    Thanks so much , i will select a certain frame try find where memeory alloc.
    I will check if this big memory alloc in other thread, unity's profiler can't show other thread's gc status
     
  7. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    1,852
    It can, but the Hierarchy view usually only shows the Main Tread, though in newer Unity versions it can be switched to show other threads as well.

    The Memory Module shows GC.Alloc in frame across threads and the CPU Usage Module's Timeline view shows Magenta Samples for GC.Alloc across all frames too

    Instead of looking for the frame where the memory is getting allocated, you might also get somewhere by looking at what is in memory once it is using this much by using the Memory Profiler package. A combination of these two approaches is likely to help you the most in understanding what's going on and how to fix it.
     
    tinyant likes this.
  8. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    117

    Thanks

    Finally with look up other thread gc in Unity3d's Profiler, I find that the big gc was happened in other thread not in mainthread "while(true)" I used https://github.com/simonwittber/uniwebserver for create a http server in unity3d.

    Code (CSharp):
    1.         string ReadLine(NetworkStream stream)
    2.         {
    3.             var s = new List<byte>();
    4.             while (true)
    5.             {
    6.                 var b = (byte)stream.ReadByte();
    7.                 if (b < 0) break;
    8.                 if (b == '\n')
    9.                 {
    10.                     break;
    11.                 }
    12.  
    13.                 s.Add(b);
    14.             }
    15.  
    16.             return System.Text.Encoding.UTF8.GetString(s.ToArray()).Trim();
    17.         }
     

    Attached Files:

    MartinTilo likes this.
  9. polemical

    polemical

    Joined:
    Jun 17, 2019
    Posts:
    1,241
    I recently dealt with something similar. The heap was just continuing to grow in builds only (Win 10 64), despite some barely-significant alloc outside of the heavy lifting of scene inits and specific stuff at other times I knew the reason for, nothing could explain why a few hundred MB was just not being freed on every scene unload. It should never have been going higher than 2-3 gb or so (based on Play Mode and WebGL builds), but definitely came off leaky and was easily on the way to consuming all available resources / crashing out unless I'd close it before then.

    I learned the Addressables Event Viewer stuff (amazing btw), found no problems there, probably tried every which combination of builds with Mono and IL2CPP and incremental GC and Burst. Things were working fine in the Editor and WebGL.. well, after a dozen builds, I found some code that had gone grey - literally and figuratively, I sort of just kept skimming right over it each time without noticing what it was / figuring it wasn't relevant.

    Turns out, a long while back I was messing with runtime GC settings, experimenting with tuning how and when to perform it, and forgot a line when I removed the related stuff later. I hadn't profiled Windows builds for a while since then due to not being focused on a release of a build for it, and just never noticed what I had done until the next time I wanted to spend time with related optimization (yesterday).

    Well sure enough, after finding and removing this, things went back to working as expected.
    Code (CSharp):
    1. #if !UNITY_EDITOR && !UNITY_WEBGL
    2.   GarbageCollector.GCMode = GarbageCollector.Mode.Disabled;
    3. #endif

    The moral is - while unrelated to your specific problem (but definitely on topic) - it's worth searching for any code with "GC." or "GarbageCollector." that might be in the project and commenting it out while trying to track down a problem like this :rolleyes:
     
    tinyant likes this.
unityunity