[BUG] NativeQueue does not work properly. Proofs.

EDIT:

Spoiler: Problem

Problem is in NativeQueue.Enqueue() method.
Here is a test code that you can easily run and see memory increasing:

Code (CSharp):

1.  
2. using Unity.Collections;
3. using UnityEngine;
4.  
5. public class Runner : MonoBehaviour
6. {
7.     private NativeQueue<int> queue;
8.  
9.     private void OnEnable()
10.     {
11.         //Allocate
12.         queue = new NativeQueue<int>(Allocator.Persistent);
13.  
14.     }
15.  
16.     private void OnDisable()
17.     {
18.         //Deallocate
19.         queue.Dispose();
20.     }
21.  
22.     private void Update()
23.     {
24.         Do();
25.     }
26.  
27.     void Do()
28.     {
29.         for (int i = 0; i < 1000000; i++)
30.         {
31.             queue.Enqueue(0);
32.             queue.Dequeue();
33.         }
34.         if (queue.Count > 1)
35.             Debug.LogError("This should never happens.");
36.     }
37. }
38.  

If you run this code for a while you will see something similar:

Spoiler: Why this happens

And now why this memory leak happens...
Here is the source code of Enqueue method:

Code (CSharp):

1.         unsafe public void Enqueue(T entry)
2.         {
3. #if ENABLE_UNITY_COLLECTIONS_CHECKS
4.             AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
5. #endif
6.             NativeQueueBlockHeader* writeBlock = NativeQueueData.AllocateWriteBlockMT<T>(m_Buffer, m_QueuePool, 0);
7.             UnsafeUtility.WriteArrayElement(writeBlock + 1, writeBlock->m_NumItems, entry);
8.             ++writeBlock->m_NumItems;
9.         }
10.  
11.         unsafe public T Dequeue()
12.         {
13.             T item;
14.  
15.             if (!TryDequeue(out item))
16.                 throw new InvalidOperationException("Trying to dequeue from an empty queue");
17.  
18.             return item;
19.         }

Most important line is here:

NativeQueueBlockHeader* writeBlock = NativeQueueData.AllocateWriteBlockMT<T>(m_Buffer, m_QueuePool, 0);

NativeQueue allocates new memory every Enqueue call. Ok.

Here is the source code of Dequeue method:

Code (CSharp):

1.         unsafe public bool TryDequeue(out T item)
2.         {
3. #if ENABLE_UNITY_COLLECTIONS_CHECKS
4.             AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
5. #endif
6.             NativeQueueBlockHeader* firstBlock = NativeQueueData.GetFirstBlock(m_Buffer, m_QueuePool);
7.  
8.             if (firstBlock == null)
9.             {
10.                 item = default(T);
11.                 return false;
12.             }
13.  
14.             var currentRead = m_Buffer->m_CurrentRead++;
15.             item = UnsafeUtility.ReadArrayElement<T>(firstBlock + 1, currentRead);
16.  
17.             NativeQueueData.Release(firstBlock, m_QueuePool); // here must be memory deallocation
18.  
19.             return true;
20.         }

So, NativeQueue designed such way that Enqueue adds to the end and Dequeue frees first block.
But it doesn't work.

Also, I tried to do like this:

Code (CSharp):

1.  
2. // it is a modification of Runner scripts
3. private void Update()
4.     {
5.         if (Time.time < 10)
6.             Do();
7.         else
8.         {
9.             queue.Dispose()
10.             enabled = false;
11.         }
12.     }

After 10 seconds, when NativeQueue must be disposed, memory usage didn't change. So, memory freed by NativeQueue.Dequeue() are lost forever. (until unity restart)

Spoiler: Original post

Hi

In my project I use NativeCollections almost everywhere.
I use NativeArrays, NativeLists and NativeQueues.

When I was profiling my windows build I noticed that 'Reserved Total' memory was constantly increasing.
So I made a stress test and let my systems run for about 15 minutes in windows build.
That time was enough to crush my game with memory access error.

Here is the last frame in profiler before crush:

(fun fact, I have only 8GB of RAM, but Unity uses 16GB, lol)

I spent several hours trying to find what causes this issue.

AND I FOUND!!!

I have a simple job:

Code (CSharp):

1.  
2.         [BurstCompile]
3.         private struct MyJob : IJobParallelFor
4.         {
5.             public NativeArray<Data> datas;
6.             [WriteOnly]
7.             public NativeQueue<int>.ParallelWriter queue;
8.  
9.             public void Execute(int index)
10.             {
11.                 var data = datas[index];
12.                 if (data.NotNeed)
13.                 {
14.                     queue.Enqueue(index); // commenting this line fixes the issue
15.                 }
16.             }
17.         }
18.  

And the problem is in parallel writing to NativeQueue. If I don't Enqueue() to NativeQueue then the problem disappears.
I don't know how it was working in previous versions of NativeContainers with Concurrent instead of ParallelWriter, but now in 'Collections version 0.1.1' this problem occurs.

When should I expect a fix for this bug? Or maybe I made something wrong?

 

For NativeContainers you need to manage memory. See https://docs.unity3d.com/Manual/JobSystemNativeContainer.html and check which Allocator you use and if it is not managed for you, make sure that you free the memory yourself.

 

Thanks for the answer.
My bad, I didn't mention this in the original post, but my NativeQueue uses Allocator.Permanent.
I have some understanding about memory managing with NativeCollection and I'm 99% sure that this is a bug.

Here is video where I showing really strange behavior with NativeCollection.AsParallelWriter().
The problem just in one line where 'NativeCollection.AsParallelWriter()' called. In the video I commented this line and memory stops leaking.

 

Also, if I would really mess up with NativeCollections' memory disposing, then I would get errors in the console.
Jobs watching for memory leak in the editor.

 

Finally, I figure it out...
Problem is in NativeQueue.Enqueue() method.
Here is a test code that you can easily run and see memory increasing:

Code (CSharp):

1.  
2. using Unity.Collections;
3. using UnityEngine;
4.  
5. public class Runner : MonoBehaviour
6. {
7.     private NativeQueue<int> queue;
8.    
9.     private void OnEnable()
10.     {
11.         //Allocate
12.         queue = new NativeQueue<int>(Allocator.Persistent);
13.  
14.     }
15.  
16.     private void OnDisable()
17.     {
18.         //Deallocate
19.         queue.Dispose();
20.     }
21.  
22.     private void Update()
23.     {
24.         Do();
25.     }
26.  
27.     void Do()
28.     {
29.         for (int i = 0; i < 1000000; i++)
30.         {
31.             queue.Enqueue(0);
32.             queue.Dequeue();
33.         }
34.         if (queue.Count > 1)
35.             Debug.LogError("This should never happens.");
36.     }
37. }
38.  

If you run this code for a while you will see something similar:

 

I slightly changed the script to speed up this process.

Code (CSharp):

1.  
2. using Unity.Collections;
3. using Unity.Mathematics;
4. using UnityEngine;
5.  
6. public class RunnerBigData : MonoBehaviour
7. {
8.     private struct Data
9.     {
10.         public float4x4 data0;
11.         public float4x4 data1;
12.         public float4x4 data2;
13.         public float4x4 data3;
14.         public float4x4 data4;
15.         public float4x4 data5;
16.         public float4x4 data6;
17.         public float4x4 data7;
18.         public float4x4 data8;
19.         public float4x4 data9;
20.         public float4x4 data10;
21.         public float4x4 data11;
22.         public float4x4 data12;
23.         public float4x4 data13;
24.         public float4x4 data14;
25.         public float4x4 data15;
26.         public float4x4 data16;
27.         public float4x4 data17;
28.         public float4x4 data18;
29.         public float4x4 data19;
30.  
31.     }
32.  
33.     private NativeQueue<Data> queue;
34.  
35.     private void OnEnable()
36.     {
37.         //Allocate
38.         queue = new NativeQueue<Data>(Allocator.Persistent);
39.  
40.     }
41.  
42.     private void OnDisable()
43.     {
44.         //Deallocate
45.         queue.Dispose();
46.     }
47.  
48.     private void Update()
49.     {
50.         Do();
51.     }
52.  
53.     void Do()
54.     {
55.         for (int i = 0; i < 10000; i++)
56.         {
57.             queue.Enqueue(default);
58.             queue.Dequeue();
59.         }
60.         if (queue.Count > 1)
61.             Debug.LogError("This should never happens.");
62.     }
63. }
64.  

 

And now why this memory leak happens...
Here is the source code of Enqueue method:

Code (CSharp):

1.         unsafe public void Enqueue(T entry)
2.         {
3. #if ENABLE_UNITY_COLLECTIONS_CHECKS
4.             AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
5. #endif
6.             NativeQueueBlockHeader* writeBlock = NativeQueueData.AllocateWriteBlockMT<T>(m_Buffer, m_QueuePool, 0);
7.             UnsafeUtility.WriteArrayElement(writeBlock + 1, writeBlock->m_NumItems, entry);
8.             ++writeBlock->m_NumItems;
9.         }
10.  
11.         unsafe public T Dequeue()
12.         {
13.             T item;
14.  
15.             if (!TryDequeue(out item))
16.                 throw new InvalidOperationException("Trying to dequeue from an empty queue");
17.  
18.             return item;
19.         }

Most important line is here:

NativeQueueBlockHeader* writeBlock = NativeQueueData.AllocateWriteBlockMT<T>(m_Buffer, m_QueuePool, 0);

NativeQueue allocates new memory every Enqueue call. Ok.

Here is the source code of Dequeue method:

Code (CSharp):

1.         unsafe public bool TryDequeue(out T item)
2.         {
3. #if ENABLE_UNITY_COLLECTIONS_CHECKS
4.             AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
5. #endif
6.             NativeQueueBlockHeader* firstBlock = NativeQueueData.GetFirstBlock(m_Buffer, m_QueuePool);
7.  
8.             if (firstBlock == null)
9.             {
10.                 item = default(T);
11.                 return false;
12.             }
13.  
14.             var currentRead = m_Buffer->m_CurrentRead++;
15.             item = UnsafeUtility.ReadArrayElement<T>(firstBlock + 1, currentRead);
16.  
17.             NativeQueueData.Release(firstBlock, m_QueuePool); // here must be memory deallocation
18.  
19.             return true;
20.         }

So, NativeQueue designed such way that Enqueue adds to the end and Dequeue frees first block.
But it doesn't work.

Also, I tried to do like this:

Code (CSharp):

1.  
2. // it is a modification of Runner scripts
3. private void Update()
4.     {
5.         if (Time.time < 10)
6.             Do();
7.         else
8.         {
9.             queue.Dispose()
10.             enabled = false;
11.         }
12.     }

After 10 seconds, when NativeQueue must be disposed, memory usage didn't change. So, memory freed by NativeQueue.Dequeue() are lost forever. (until unity restart)

 

产品经理吃粑粑！。。。。。。。。。

 

This is quite horrible and seeing nobody from the team in this topic is quite concerning.

There is a related bug with TryDequeue. This code causes a leak as well:

Code (CSharp):

1. using Unity.Collections;
2. using UnityEngine;
3.  
4. public class NativeQueueMemLeak : MonoBehaviour
5. {
6.     NativeQueue<int> datas;
7.  
8.     void Awake() => datas = new NativeQueue<int>(Allocator.Persistent);
9.     void OnDestroy() => datas.Dispose();
10.  
11.     void Update()
12.     {
13.         datas.Enqueue(0);
14.         while (datas.TryDequeue(out int d)) { }
15.     }
16. }

At least for this TryDequeue problem one can work around it by using .Count and .Dequeue. But seeing the leak happening with only .Enqueue and .Dequeue, as described in this thread, it does not solve anything.

Maybe this topic should be moved/recreated in the DOTS forum, there might be more people involved with native collections looking at it?

 

Hi,

Run into this same issue myself. Enqueue allocates memory that does not seem to be freed by Dequeue.

It is easily reproducible using MUGIK's test code.

 

The DOTS team doesn't read the general scripting forums. Create a bug report with the official bug reporter, or post in the dedicated DOTS forum if you want this looked at.