NativeArrays complain about being leaked right after disposing them

Hello!

Someone having any experience with a bit more advanced usages of the job system? I'm having some trouble with disposed NativeArrays still complaining about being leaked and I don't understand why.

Basically I have a procedural generation system, that queues up a lot of generators that use jobs to quickly generate a map with dependencies on each other. So one generator can create a map that another generator will modify (think noise + erosion afterwards).

I use a Decorator pattern for the generators, each generator will queue up the job of the one it depends on first, then queues up it's own job and gives his jobhandle to the next one, until I reach the main generator that will ultimately run all the jobs to completion.

Some of the generators need to create temporary NativeArrays for their jobs. They store them in their class as members. When all generators are done and all jobs have been completed, I ask every generator to dispose of their own NativeArrays. This succeeds fine. But a frame later all of the already disposed arrays still complain about being leaked and I don't understand how I can fight this.

Here is some code to put it into perspective:

Code (CSharp):

1.  
2. NativeArray<float> GeneratedData = new NativeArray<float>(Map.Size.x * Map.Size.y, Allocator.Persistent);
3.  
4. JobHandle jobHandle = Generator.Evaluate(ref GeneratedData, Map.Size);
5. jobHandle.Complete();
6.                
7. Generator.CleanUp();
8.  

Here's the interface of the generators:

Code (CSharp):

1.  
2. public interface IGenerator<T> where T : struct
3. {
4.     JobHandle Evaluate(ref NativeArray<T> outData, int2 size);
5.  
6.     void CleanUp();
7. }
8.  

And here is one example of a simple generator that simply translates between int arrays to float arrays:

Code (CSharp):

1.  
2. public class IntToFloatGenerator : IGenerator<float>
3.     {
4.         [BurstCompile]
5.         struct IntToFloatJob : IJobParallelFor
6.         {
7.             public NativeArray<float> output;
8.             [ReadOnly] public NativeArray<int> input;
9.            
10.             public void Execute(int index)
11.             {
12.                 output[index] = input[index];
13.             }
14.         }
15.  
16.         readonly IGenerator<int> _intGen;
17.         NativeArray<int> _intData;
18.  
19.         public IntToFloatGenerator(IGenerator<int> intGen)
20.         {
21.             _intGen = intGen;
22.         }
23.        
24.         public override JobHandle Evaluate(ref NativeArray<float> outData, int2 size)
25.         {
26.             //Create temporary array
27.             _intData = new NativeArray<int>(outData.Length, Allocator.TempJob);
28.             //Evaluate input generator first with temporary array as output array
29.             JobHandle inputJobHandle = _intGen.Evaluate(ref _intData, size);
30.            
31.             IntToFloatJob job = new IntToFloatJob
32.                                     {
33.                                         input = _intData,
34.                                         output = outData
35.                                     };
36.  
37.             JobHandle outJobHandle = job.Schedule(outData.Length, size.x, inputJobHandle);
38.             JobHandle.ScheduleBatchedJobs();
39.             return outJobHandle;
40.         }
41.  
42.         public override void CleanUp()
43.         {
44.             _intGen.CleanUp();
45.  
46.             if(_intData.IsCreated)
47.             {
48.                 _intData.Dispose();
49.             }
50.         }
51.     }
52.  

As you can see I dispose of the temporary arrays as soon as the jobs are completed and as each generator calls the cleanup of their dependent generators, all of the temporary arrays should be disposed before the frame is over. So I really don't understand why the engine is complaining about them leaking.

Anyone having any idea about this? I'm glad for any help on this, as I already spend quite some time trying to debug this issue but with no solution yet.

Loofou

 

I've experimented even further with this over the last month, but the problem still appears. It's hard to tell from memory profilers (as they don't seem to track unmanaged memory) if the NativeArrays are really fully disposed, but from all I know they should. Yet I still get the error messages. I am assuming my use case is either too complicated or there is a bug in the system somewhere. Some help would really be greatly appreciated.

 

1. You did check that the temp arrays actually are the leaking arrays using the stack trace leak detection? I ask because in your example GeneratedData isn't disposed but I expect that is only the example code.
2. Put an Assert.IsFalse(<tempData>.IsCreated) everywhere you create a new NativeArray and check if it triggers anywhere. Maybe Evaluate is called multiple times which will result in the previous call leaking. Maybe start your Evaluate method with a Cleanup to prevent this?

 

GeneratedData is a permanent NativeArray that is disposed at the end of the game. And yes, Stack Trace Leak Detection reports every single one of the temporary arrays when I run my code.

I will try your second recommendation, though. Thank you for the tip. I don't know why Evaluation would be called multiple times, but it is indeed a possibility I haven't considered. Thank you for the idea!