Very odd Huge Memory Usage 40GB ManagedHeap.UsedSize !!!

Unity2020.3.20f1
Running My app on WindowServer 2019，
Every frame only 138Byte alloc, Everything goes right unity wil trigger gc after a few minutes,
BUT After several Hours，The total memory increased So large，


In Profiler I Just Find Only 136B memory alloc, But GC Used Memory just increase 4GB，I can't find where these big memory alloc.

 

My application start at
[2022-05-16 17:24:36.754 +08:00]


2022-05-16 23:01:45.072 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 9776KB 9.546875MB 5084KB 4.964844MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133149KB 130.0283MB -572.6602KB -0.5592384MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 195347KB 190.7685MB 572.6602KB 0.5592384MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 11432KB 11.16406MB 3628KB 3.542969MB

2022-05-16 23:02:45.080 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 44840KB 43.78906MB 35064KB 34.24219MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133285.6KB 130.1618MB 136.6221KB 0.13342MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 195210.4KB 190.6351MB -136.6221KB -0.13342MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 84164KB 82.19141MB 72732KB 71.02734MB

2022-05-16 23:03:45.108 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 99784KB 97.44531MB 54944KB 53.65625MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133285.6KB 130.1618MB 0KB 0MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 195210.4KB 190.6351MB 0KB 0MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 150216KB 146.6953MB 66052KB 64.50391MB

2022-05-16 23:04:45.108 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 109968KB 107.3906MB 10184KB 9.945313MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133720KB 130.5859MB 434.3506KB 0.4241705MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 194776KB 190.211MB -434.3506KB -0.4241705MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 150216KB 146.6953MB 0KB 0MB



2022-05-16 23:05:45.140 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 154524KB 150.9023MB 44556KB 43.51172MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133149KB 130.0283MB -570.9727KB -0.5575905MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 195347KB 190.7685MB 570.9727KB 0.5575905MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 281804KB 275.1992MB 131588KB 128.5039MB



2022-05-17 04:03:49.415 +08:00 [INF] GCManager:MEM-----------------------
l1 GetMonoUsedSizeLong: 4198112KB 4099.719MB 4KB 0.00390625MB
l2 GetTempAllocatorSize: 4096KB 4MB 0KB 0MB
l3 GetTotalAllocatedMemoryLong: 133866KB 130.7285MB 434.0928KB 0.4239187MB
l4 GetTotalReservedMemoryLong: 328496KB 320.7969MB 0KB 0MB
l5 GetTotalUnusedReservedMemoryLong: 194630KB 190.0684MB -434.0928KB -0.4239187MB
l6 maxUsedMemory: 131072KB 128MB 0KB 0MB
l7 GetMonoHeapSizeLong: 4251432KB 4151.789MB 4KB 0.00390625MB



Just MonoHeapSizeLong and GetMonoUsedSizeLong Increased Huge but GetTotalAllocatedMemoryLong GetTotalReservedMemoryLong GetTotalUnusedReservedMemoryLong seems normals


I can't find why GetMonoHeapSizeLong GetMonoUsedSizeLong become so odd

 

Your last screenshot doesn't seem to show a sample at a point where your memory usage actually makes those huge jumps up. You know that you can use the arrow keys to move one sample to the left / right? Turn on deep profiling and actually select one of the frames where a spike happens. You clearly have a huge increase at larger time intervals. Maybe a coroutine or invokeRepeating call.

So you need to properly analyse the profiler results

 

I can't find so large memory usage in profiler.

 

I don't get your new screenshot. It's still a frame you're showing which allocates 139kb. Here, I made a screenshot from your first screenshot.


The "red dot" is the memory usage of the frame you have selected. As you can see the line is almost straight (I overpainted the yellow line since it was below the tooltips). What you should select is a frame where the memory usage increases heavily. Like the pink line I drawn in to the left. Here the usage makes a huge jump. So have a look at those frames and see how much is actually allocated in those frames and what is allocating it. I don't know how you made the screenshot, but you can simply pause the game in order to analyse the profiler results.

As I said above you can use the arrow keys to go one frame forwards / backwards in order to fine select a certain frame. Again, we can not do that for you. You have the project and profiler data.

 

Thanks so much , i will select a certain frame try find where memeory alloc.
I will check if this big memory alloc in other thread, unity's profiler can't show other thread's gc status

 

Thanks

Finally with look up other thread gc in Unity3d's Profiler, I find that the big gc was happened in other thread not in mainthread "while(true)" I used https://github.com/simonwittber/uniwebserver for create a http server in unity3d.

Code (CSharp):

1.         string ReadLine(NetworkStream stream)
2.         {
3.             var s = new List<byte>();
4.             while (true)
5.             {
6.                 var b = (byte)stream.ReadByte();
7.                 if (b < 0) break;
8.                 if (b == '\n')
9.                 {
10.                     break;
11.                 }
12.  
13.                 s.Add(b);
14.             }
15.  
16.             return System.Text.Encoding.UTF8.GetString(s.ToArray()).Trim();
17.         }

 

I recently dealt with something similar. The heap was just continuing to grow in builds only (Win 10 64), despite some barely-significant alloc outside of the heavy lifting of scene inits and specific stuff at other times I knew the reason for, nothing could explain why a few hundred MB was just not being freed on every scene unload. It should never have been going higher than 2-3 gb or so (based on Play Mode and WebGL builds), but definitely came off leaky and was easily on the way to consuming all available resources / crashing out unless I'd close it before then.

I learned the Addressables Event Viewer stuff (amazing btw), found no problems there, probably tried every which combination of builds with Mono and IL2CPP and incremental GC and Burst. Things were working fine in the Editor and WebGL.. well, after a dozen builds, I found some code that had gone grey - literally and figuratively, I sort of just kept skimming right over it each time without noticing what it was / figuring it wasn't relevant.

Turns out, a long while back I was messing with runtime GC settings, experimenting with tuning how and when to perform it, and forgot a line when I removed the related stuff later. I hadn't profiled Windows builds for a while since then due to not being focused on a release of a build for it, and just never noticed what I had done until the next time I wanted to spend time with related optimization (yesterday).

Well sure enough, after finding and removing this, things went back to working as expected.

Spoiler

Code (CSharp):

1. #if !UNITY_EDITOR && !UNITY_WEBGL
2.   GarbageCollector.GCMode = GarbageCollector.Mode.Disabled;
3. #endif

The moral is - while unrelated to your specific problem (but definitely on topic) - it's worth searching for any code with "GC." or "GarbageCollector." that might be in the project and commenting it out while trying to track down a problem like this