Question How can I identify where a native memory leak comes from?

In short: how can I identify where a native memory leak comes from?

Long story:

We are using Unity 2021.3.5f1.

In our project we found that memory keeps increasing in our builds and in editor. There is clearly a leak going on, which is big enough to become a problem quickly. However even after going through obvious places, memory was still increasing, and Unity isn't logging any particular warning.

I tried `Jobs -> Leak Detection -> Full Stack Traces`, but it didn't find anything.

I tried the memory profiler, which shows that it's "other native memory", but I could not find useful information in the tool to identify where all these allocations come from. All I see is most of them being done with `UnsafeUtility.Malloc`.

Part of our project uses a framework that uses `UnsafeUtility.Malloc` and `UnsafeUtility.Free`, so I wrapped them up with my own version of Malloc and Free which captures the stack of Persistent allocs, with Burst turned off. This allowed me to dump a lot of useful data, which I could analyze myself (notably finding memory allocations that keep going up due to a bad design, even if they are freed at the end) and fix some small leaks. This was immensely useful, But memory is still increasing badly.

So my next idea was to track `Malloc` and `Free` in `Unity.Collections` as well, because our game and a lot of other packages are using it. Unfortunately, collections like `NativeArray` are not source-available in the package version we are using (`0.15.0-preview.21`), so I am not able to modify them to use my own version of `Malloc` and `Free`, therefore I can't identify allocations by giving them a stack.

In newer versions it seems we can define custom allocators, however it appears too complicated for this use case, and it wouldn't work because using a custom allocator requires to explicitely state it everywhere, including packages, some of which i can't modify either.

So unless I can stumble on a solution by trying random things, I'm short of ideas for now... guesswork is wasting a lot of time

 

I was able to track some extra allocations by modifying `Memory.cs` in the source-available part of Unity Collections. Unsure if it will be accurate since in theory nothing prevents Malloc to be paired with a Free that's not in this file, but so far it seems to work.
An important kind of leak I was able to track by doing that is `BlobAssetReference<T>`, which has to be disposed manually if created from code.

I still wish there was a way to simply hook Malloc and Free in UnsafeUtility somehow.

 

One thing to keep in mind is that modifying the source code of third-party libraries or packages can create complications when upgrading to a new version of the package.

Regarding hooking Malloc and Free in UnsafeUtility, one potential approach is to use a memory profiler tool that can intercept and track all memory allocations and deallocations in your project, including those done through UnsafeUtility.Malloc and UnsafeUtility.Free. There are a number of memory profiler tools available for Unity, such as Unity's own Memory Profiler or third-party tools like JetBrains dotMemory and Redgate ANTS Memory Profiler.

By using a memory profiler, you can get a more complete picture of the memory usage in your project and identify any memory leaks or excessive allocations. Additionally, some memory profiler tools allow you to track memory allocations to specific code locations, such as by setting markers or capturing call stacks, which can help you pinpoint where in your code the allocations are occurring.

 

Thanks for the good suggestion. This helped me find my issue.

In case anyone has the same: In my game, we run the simulation in a custom world and after upgrading Entities version, we missed an important change: In our custom world, we need: WorldUpdateAllocatorResetSystem as well as myCustomWorld.Update(). Otherwise, the "WorldUpdate" allocators don't clear and they grow indefinitely.