Crash when sorting NativeList.

Hi,

Any idea of what can be causing this error:
NullReferenceException: Object reference not set to an instance of an object
System.Buffer.memcpy4 (System.Byte* dest, System.Byte* src, System.Int32 size) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.Buffer.Memcpy (System.Byte* dest, System.Byte* src, System.Int32 size) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.String.memcpy (System.Byte* dest, System.Byte* src, System.Int32 size) (at <437ba245d8404784b9fbab9b439ac908>:0)
Unity.Collections.LowLevel.Unsafe.UnsafeUtility.ReadArrayElement[T] (System.Void* source, System.Int32 index) (at <7d22f8e71133418c87c7b26ea181f3e3>:0)
Unity.Collections.NativeSortExtension.Partition[T,U] (System.Void* array, System.Int32 lo, System.Int32 hi, U comp) (at Library/PackageCache/com.unity.collections@0.1.1-preview/Unity.Collections/NativeSort.cs:142)

It's weird because the code runs up to the first of these lines but crashes on the secons:
...
while (comp.Compare(pivot, UnsafeUtility.ReadArrayElement<T>(array, ++left)) > 0) ;
while (comp.Compare(pivot, UnsafeUtility.ReadArrayElement<T>(array, --right)) < 0) ;
...

The function where it crashes is
unsafe static int Partition<T, U>(void* array, int lo, int hi, U comp) where T : struct where U : IComparer<T>
of class NativeSortExtension.

I'm sorting a NativeList:

Code (CSharp):

1. var array = new NativeList<SpatialRender>(Allocator.TempJob);

And sorting by calling sort function on the array:

Code (CSharp):

1. array.Sort(comparer);

Where SpatialRender is a struct:

Code (CSharp):

1. public struct SpatialRender
2. {
3.     public float3 location;
4.     public int renderDataIndex;
5.     public Matrix4x4 matrix;
6.     public Vector4 frame;
7. }

and comparer is a struct of type:

Code (CSharp):

1. protected struct LocationComparer : IComparer<SpatialRender>
2. {
3.     public int Compare(SpatialRender a, SpatialRender b)
4.     {
5.         if (a.location.y < b.location.y)
6.             return 1;
7.         else
8.             return -1;
9.     }
10. }
11.  

On Unity 2019.3.0f5
Entities preview.17 - 0.5.0
Collections preview.9 - 0.5.0
Burst 1.2.1

Thanks.

 

Hello,

I made a test like this:

Code (CSharp):

1.  
2.         var list = new NativeList<SpatialRender>(Allocator.TempJob)
3.         {
4.             new SpatialRender{location = new float3(0, 7, 0)},
5.             new SpatialRender{location = new float3(0, 2, 0)},
6.             new SpatialRender{location = new float3(0, 5, 0)},
7.             new SpatialRender{location = new float3(0, 4, 0)},
8.             new SpatialRender{location = new float3(0, 8, 0)},
9.             new SpatialRender{location = new float3(0, 3, 0)},
10.             new SpatialRender{location = new float3(0, 6, 0)},
11.             new SpatialRender{location = new float3(0, 12, 0)},
12.             new SpatialRender{location = new float3(0, 11, 0)},
13.             new SpatialRender{location = new float3(0, 11, 0)}
14.         };
15.  
16.         new TestJob { list = list }.Schedule().Complete();
17.         for (int i = 0; i < list.Length; i++)
18.         {
19.             var item = list[i];
20.             Debug.Log(item.location.y);
21.         }
22.         list.Dispose();
23.  

Code (CSharp):

1.     [Unity.Burst.BurstCompile]
2.     public struct TestJob : IJob
3.     {
4.         public NativeList<SpatialRender> list;
5.  
6.         public void Execute()
7.         {
8.             list.Sort(new LocationComparer());
9.         }
10.     }

and it works fine (i am running this code in a monobehaviour), it properly sorts on descending order, maybe if you can give us more information about your code perhaps people can help you better.

 

Thanks.

The code is a bit long.

1. I generate a tile map with a Diamond-Square algorithm, with only two types of terrain.
2. I generate data for trees where the value of the tile is less that 0.5 with some ramdomness. Only one tree per tile.
3. I inject the trees data into the static NativeHashMap m_spatialPartitionRenderStatics of the system JSystemSpatialPartition.
4. In the system JSystemSpatialPartition I fill the NativeMultiHashMap m_spatialPartitionRender with the render data of all entities in screen, including the trees.
5. In the system JSystemPrerender I sort the data of m_spatialPartitionRender and prepare it for render:
6. Get all the unique keys of JSystemSpatialPartition.m_spatialPartitionRender and sort them
7. Loop those keys, get the values of those keys and add them to as many NativeList as I need to mantain the sort time low.
8. Sort each NativeList (here is when randomly crashes).
9. Fill a NativeArray<Matrix4x4>, a NativeArray<Vector4> and a NativeList<Vector2Int> with limits up to 1023, that are going to be used for render with Graphics.DrawMeshInstanced by the system SystemSpritesRenderer.

The crash is random, always sorting a NativeList (step 8), and I have not been able to reproduce it when I don't add the trees render data to m_spatialPartitionRender in the system JSystemSpatialPartition. This does not mean that the trees are the cause, only, for now, that they increase the probability of the crash.

This is my first experment with DOTS so it's a bit difficult for me to trace the cause of this bug. Maybe is a memory leak. maybe some data is being overwritten.

Any advise?

Thanks again.

 

Change

Code (CSharp):

1. protected struct LocationComparer : IComparer<SpatialRender>
2. {
3.     public int Compare(SpatialRender a, SpatialRender b)
4.     {
5.         if (a.location.y < b.location.y)
6.             return 1;
7.         else
8.             return -1;
9.     }
10. }
11.  

to

Code (CSharp):

1. protected struct LocationComparer : IComparer<SpatialRender>
2. {
3.     public int Compare(SpatialRender a, SpatialRender b)
4.     {
5.         if (a.location.y < b.location.y)
6.             return 1;
7.         else if (a.location.y > b.location.y)
8.             return -1;
9.         else
10.             return 0;
11.     }
12. }
13.  

I had the same issue.

Cheers.

 

Wow, I missed it, I'll try it.

thanks!!!

 

This fixed my issue.

Thanks

 

This thread saved my day!
Thanks a lot.

 

Cheers

 

recently had same issue

just using IComparer that have logic mistake in condition makes Sort() going out of array bounds without any warning/error at all, even with ENABLE_UNITY_COLLECTIONS_CHECKS

debugging such thing is a nightmare, Sort() should perform array bounds check at least in editor with define above