How to investigate and improve memory fragmentation?

Hi there,

This looks like you are running on IL2CPP and the Memory Map resolution set to 6.4 MB per line?

This means that we can't see the Virtual Machine Memory Allocated by IL2CPP because it is not going through Unity's Native Memory Manager Allocators, so there might be an unaccounted for amount of memory here that is tied to Type Metadata, Generics and Reflection.

Beyond that, you look to be loosing some 10.24 MB to Managed Memory Fragmentation. Analyzing the Native Memory like this is a bit trickier from a screenshot.

As for how I did that analysis: Managed Memory is allocated in Sections that are either VM Memory (not yet visible in IL2CPP, in Mono it's dark blue and currently (Update: as of version 0.4.0 of the package it is visible for snapshots taken with from builds made with Unity Editor versions 2021.2.0a12, 2021.1.9f1, 2020.3.12f1, and 2019.4.29f1 or newer) undistinguishable from empty Heap Sections, but those are rare as they would get returned to the OS after some GC.Collect cycles) or Heap Sections used for Managed Object allocations. The Memory in these sections is either used for Managed Object memory (light blue) or unused (dark blue). Generally speaking, the Heap Section which has the highest Virtual Address Values of all Heap sections is the Active Heap Section.

New Object Allocation will be first placed in the Active Heap section. If that is not possible, the allocator will scan the free block list ( think of these as reclaimed chunks of memory of specific sizes in old sections) and if a block that can fit the allocation is found we'll allocate in that. Otherwise a new heap section will be created (gc heap expansion) at a higher virtual address value than the highest one that was ever used for an allocation in this run, i.e. at the very bottom of the map.

All Inactive Heap sections are abandoned for new block allocations and only monitored by the GC, which will check if they still have uncollected Objects in them during each GC. When objects are collected and the Block they were allocated in is thereby emptied, that Block gets added to the free block list. If a section only consists of free blocks, the section will be returned to the OS, freeing up the Virtual Address space (i.e. that will then show the black background color in its place).
Therefore, all dark blue space in heap sections with lower virtual address values is (potentially, but likely) wasted to fragmentation. If all the Objects within these sections would get collected, that memory could be returned to the OS.

Now obviously, the next run might result in entirely different memory distribution so it might not be as simple as: "if this object(s) would be more short lived, I'll get back x amount of RAM". But they can provide evidence to the moment in which this heap section was created and what other short lived allocations might have shared that space and been collected already. If these objects are purposefully long lived, they can also show which kind of objects might be good to allocated at the same time as some other long lived allocations in order to have them packed together more tightly and make it less likely that they share a heap section with a short lived allocation.

Now that is a bit much to take in I guess but, try and see if that helps you making any sense of what is going on in your memory there, because there's only so much I can tell you from screenshots.

P.s. you can tell the bottom area of the Memory Map to show you the objects in a selected Virtual address range.

P.P.S: Oh and, Native Memory is often allocated as pools or buckets and then used and reused for objects of a certain size, in an attempt to avoid fragmenting them, by mixing allocations of varying sizes. Still, you might have too many large size pools which are only sporadically filled to their full size, or too many small ones that can not be used for bigger ones then. Native Memory usage is a bit trickier to control though.

[May 6th 2021 Update]: I had previously glossed over the free block list a bit, that should technically help reuse some of that wasted space. The memory profiler doesn't yet have any info over managed Blocks or the free Block list (we're looking into adding that).

 

Right, so that's 12.8MB lost to fragmentation then.

Right, so, I stitched your screenshots together, then copied together pieces of managed memory with bigger chunks of unused memory in them together and lined them up. While lining them up, I tired to compensate for actual used memory in these sections by eyeballing them and aligning them somewhat parallel. Stitched together like that, I continued until I had one full line, i.e. 8 MB of wasted memory. Between the last line I had taken a snippet from and it's next line, I split the screenshots, added a black background and left that cobbled together line there, starting with a new one until all pieces where lined up, totaling 1.6 lines x 8 MB per line: ~12.8MB eyeball measurement. I'm planning to adjust the memory Profiler to give a fragmentation analysis like that in a future version, so that this manual process shouldn't be needed but thought I'd prototype it like that, in a way, and use it to explain managed fragmentation while at it.

Nope it doesn't, so the issue is likely elsewhere, or at least bigger gains are to be had when looking at other areas of optimization.

Which tool are you getting these numbers from, specifically the Allocatable?

That sounds like the issue is in memory not tracked by Unity, e.g. native plugin memory or IL2CPP? What platform are you running on and have you tried using a native platform profiler to see what that memory could be?

Yep

Yes but just to clarify: the GC does monitor it and collect unused objects in there, but that's little more than deleting the GC.Handle pointing at them. I.e. it's just deleting the thing used to find the object in the haystack. Once no GC.Handles point at that section anymore, this section of memory is just freed and returned to the OS.

Not sure I'll have some time to look at the snapshots but if it's, as suspected, in memory untracked by unity, that wouldn't lead far...