Question Creating a unit test that verifies if memory was allocated

I would like to create a simple unit test that can verify whether some invoked method actually causes any memory allocations.

Is this possible? I saw the the Profiler class exposes some information about allocated Mono memory, but i am not entirely sure if it works when running edit mode tests.

Can anyone guide me how to create such a test?

 

Just get the size of the heap before and after the method call and compare them... you have access to C# GC classes from unity like you would in any other C# program.

 

I tried doing that, it gave the wrong output. Running the same code in the profiler shows a 1kb GC alloc.

 

Have you profiled it while in a build or the editor? because the editor could have GC allocations that would not show up in build.
You can always use Profiler.BeginSample / EndSample for this.
profile it in a dev build with the profiler attached using the begin sample and end sample of the profiler.

Also might be that the first frame is too quick, wait one or more frames before you actually execute your code.

 

This is written as a unit test (edit mode). It doesn't run in a build (only in the editor).

 

Yeah that is the difference between playmode tests and Editmode tests.
surround your code in the test with Profiler.BeginSample / EndSample
Use the profiler -> turn on Profile Editor -> turn on Record -> run the test -> turn off Record -> use the search bar for your named sample.

I know string interpolation causes GC allocations because under the hood it is just string.Format
This allocates 160 bytes according to the profiler.

Code (CSharp):

1. [UnityTest]
2. public IEnumerator GCAllocationTestWithEnumeratorPasses()
3. {
4.     yield return null;
5.     yield return null;
6.  
7.     Profiler.BeginSample("GC Test");
8.     var stringInterpolation = $"This test is of type: {GetType()}";
9.     Profiler.EndSample();
10. }
11.  

 

I can't see your code but its not the wrong output. There may be other sources of allocation (the editor, the profiler, internal unity code), but there's no magic.

Its also worth noting this test wont be that useful:
- you would need to run it against every method (or at least every entry point)
- there are several common methods that allocate in the editor but not during a build (GetComponent if the component isn't found for example).
- there are allocations that occur outside of the method as a side effect of your code (i.e. code that makes Unity's systems do something)

If you want to solve a specific problem in a method you already know to be an issue just deep profile and solve it.

If you wanted to automate something its better to just have a method that tracks heap growth while the app runs, and raises an error (both in editor and in dev mode builds) if the heap growth over time is larger than some desired parameter (possible zero during gameplay).

Whenever the alarm goes off you can trigger investigation. Because it always runs you will generally know exactly what caused the issue as it is likely to be something you changed since the last run.

Its not unreasonable to create a performance test suite, but these should be system tests against builds running on a device, not unit tests.

 

you can actually run unit tests on the target device (Run all in player). But it has to be a play mode test.
But I don't think Unity has implemented the test runner just so that you can profile GC allocations. That's what the profiler is for.

Personally I use Resharper with the Heap Allocations Viewer extension just to see where allocations could take place in my own code. Its a good thing to have as an indication for your own code. However it does not cover methods calls.
I kind of wish Resharper was part of visual studio but sadly it is a paid third party extension.

@liortal if you are uncertain about pieces of code why they allocate memory, maybe you could put them here on the forum. I'm sure someone can explain to you why allocations happen.

 

Unity has a very loose interpretation of what a unit test is. Generally the term unit test is used to refer to tests that test a small unit in isolation such as a class or method and as such rarely useful for performance testing.*

To be honest I know little about Unity's testing framework as I wrote my own testing framework based on CIL injection many years ago and it has served me well to this point.

---
* I'm sure you know this, just trying to make clear that what I mean when I say Unit test is not just any test that uses Unity's test runner.

 

I just want to create a test that will verify that a certain method doesn't allocate any memory as it's part of the game infra and i'd like to always be able to verify that.

 

Just keep in mind this will not find quirks in Mono since it uses .NET CLR not Mono CLR. If you use 4.5 in unity 2018.2 its alot more close to .NET

 

https://github.com/controlflow/resharper-heapview

.net 2.0 framework (.net 3.5) vs .net 4.x heap allocations don't really change.
Like I said before, the extension helps. The extension is a good thing to have as an indication.
Because of the extension I am a lot more aware of allocations than I was before. Personally I like to strive to write optimal code so I don't have to optimize projects later.
Some may call it premature optimizations, I call it stop being lazy for easy optimizations.
For example, some people like to overly use LINQ because it is easy to use. I agree it is easy, but it generates GC.
Resharper can convert LINQ to Code doing the exact same thing just in code that does not allocate GC.

For my own projects I already use .net 4.x in Unity 2017.4.x as it is quite stable already. You don't necessarily need 2018.x for that. I also profile my projects with the Unity profiler just using the Profiler API to check GC allocations and performance.

 

The problem is Mono 2 that 3.5 in Unity uses has several bugs that .NET implementation does not have. For example

Dictionary.Values allocate, among others, only way to find those is to benchmark under Mono

 

Fair enough, Dictionary.Values is a property of the Dictionary class. The heap allocation viewer does not cover property calls as they have their own code inside the getter / setter that could cause GC.
.net 4.x does indeed have a more optimized version that does not allocate GC.

The heap allocation viewer is a nice indication but in the end you still have to use the profiler to track down allocations.

 

.NET have never allocated for that property. Mono has

 

This is a pretty old thread, but for those folks arriving here from search, it turns out that Unity has since evolved the ability to write tests on allocations.

First, TestFramework contains an assertion called AllocatingGCMemory that you can use for unit tests.

Second, if you don't use TestFramework or just want to have this functionality outside of tests, here is a utility class that uses the same technique:

https://gist.github.com/jeffomatic/cfa560bad7a1163730d1b5f75f549f93