EntityQuery.CopyFromComponentDataArray not updating entities

Hi guys,

I'm trying to batch instantiate a bunch of entities (testing with 10k, 100k, 1m). I'm using this excellent post as a guide: https://gametorrahod.com/batched-operation-on-entitymanager/ as well as various forum threads.

My entities get instantied but it seems like the calls I'm making on an entity query to CopyFromComponentDataArray do nothing. Code below:

Code (CSharp):

1.  
2.         EntityManager entityManager = World.DefaultGameObjectInjectionWorld.EntityManager;
3.  
4.         var types = new ComponentType[]{
5.             typeof(Translation),
6.             typeof(RenderMesh),
7.             typeof(Rotation),
8.         };
9.  
10.         var renderMesh = ... get a mesh
11.  
12.         var arc = entityManager.CreateArchetype(types);
13.  
14.         var prefab = entityManager.CreateEntity(arc);
15.         entityManager.SetSharedComponentData(prefab, renderMesh);
16.  
17.         var cellCount = rows * columns;
18.  
19.         // Instantite entities with prefab, then destroy
20.         var entityArray = new NativeArray<Entity>(cellCount, Allocator.Persistent);
21.         entityManager.Instantiate(prefab, entityArray);
22.         entityManager.DestroyEntity(prefab);
23.  
24.         // Set up component data arrays
25.         var translationArray = new NativeArray<Translation>(hexCellCount, Allocator.Persistent);
26.         var rotationArray = new NativeArray<Rotation>(hexCellCount, Allocator.Persistent);
27.  
28.         for (int i = 0; i < hexCellCount; i++) {
29.             translationArray[i] = new Translation { Value = translation };
30.             rotationArray[i] = new Rotation { Value = rotation };
31.         }
32.  
33.         var entityQuery = entityManager.CreateEntityQuery(
34.             ComponentType.ReadWrite<Translation>(),
35.             ComponentType.ReadWrite<Rotation>()
36.             );
37.  
38.         entityQuery.CopyFromComponentDataArray(translationArray);
39.         entityQuery.CopyFromComponentDataArray(rotationArray);
40.  
41.         entityArray.Dispose();
42.         translationArray.Dispose();
43.         rotationArray.Dispose();
44.  

Things I've tried:

Different allocators
Getting the job handle back from CopyFromComponentDataArrayAsync and ensuring it's completed
Calling entityQuery.CompleteDependency
Logging data at various points - the translationArray is filled correctly; calling GetComponentData before/after the calls to CopyFromComponentDataArray/Async just gives me 0/0/0
Checking entityQuery.CalculateEntityCount() - matches array length
Calling entityQuery.ToComponentDataArray, setting the values in that array, sending it back with CopyFrom
Changing the EntityQuery definition to using typeof(), ReadWrite etc.

Nothing seems to make a difference to the entities. Am I missing something here?
This code is running in Start() on a MonoBehaviour at the moment.

Unity @ 2019.3.12f1
Entities @ 0.10.0 - preview.6
Hybrid Renderer @ 0.5.0 - preview.6
Unity Physics @ 0.3.2

 

Same problem

 

@jonwah00 could you try with Entities 0.9.1-preview.15 ?
I think I also have an instance where it is brakes for me after updating to Entities 0.10

 

Hey @Srokaaa ; tried with the following:

Entities @ 0.9.1 - preview.15
Hybrid Render @ 0.4.2 - preview.16

Can confirm it does actually work!! Thank you, I was tearing my hair out. Is there somewhere I should report this to the devs?

Now nothing is rendering at all, but at least the entities have the right data, now I can figure out what's going on with the rendering..

 

You should add RenderBounds to archetype and set it

 

@jonwah00 You could try reporting a bug with minimal repro sample from within Unity Help->Report a bug
Last time I did that @Fabrice_Lete was able to find the culprit right away

 

@eizenhorn thanks; I already have the RenderBounds component on the archetype and set - I had to do that when I updated the Hybrid Render package originally.. The code I put above was just cut down to show the problem, the real archetype looks like this

Code (CSharp):

1. var types = new ComponentType[]{
2.             typeof(Translation),
3.             typeof(RenderMesh),
4.             typeof(LocalToWorld),
5.             typeof(Rotation),
6.             typeof(PhysicsCollider),
7.             typeof(HexNeighbors),
8.             typeof(Static),
9.             typeof(FrozenRenderSceneTag),
10.         };

@Srokaa cheers, I'll do that

 

Removing the FrozenRenderSceneTag component did the trick; I think that is only for Hybrid Render @ 0.5.0

 

Same Problem here. After some debugging I found out that he writes from the query to the NativeArray again....

Maybe just maybe Destination and Source were exchanged when converting to a Job

Code (CSharp):

1.    var src = chunk.m_Chunk->Buffer + typeOffset;
2.             var dst = ComponentData + (entityOffset * typeSize);
3.             var copySize = typeSize * chunk.Count;
4.  
5.             UnsafeUtility.MemCpy(dst, src, copySize);

*** ChunkDataGatherJobs.cs : 186 *** ->
GatherComponentDataJob

 

@roadis you're right; I created a minimum repro and confirmed that on Unity.Entities @ 0.10.0 the source array is overwritten with the target array.

I've submitted the bug to Unity, hopefully they sort it out.

 

Hm, weird, it reads fine.
GatherComponentDataJob copies chunk data to comp and CopyComponentArrayToChunks copies comp data to the chunk.
public static void MemCpy(void* destination, void* source, long size);
It should read as dest = source, so the code should be correct.

The jobs have been rewritten in 0.10 but if there's no deeper bug, everything should work exactly the same.

@jonwah00 You've tested this? Did it work when switching src and dest?

 

Hi @Enzi yep I've tested it; but only at the level of calling EntityQuery.CopyFromComponentDataArray - I haven't tested UnsafeUtility.MemCpy

In Entities @ 0.9.1 the values from the array get copied across as expected to the entities
In Entities @ 0.10.0 the values don't get copied across and the source array gets filled with the copy (in my case zero'ed out)

 

Yeah its exchanged in the code now

Old Code

Code (CSharp):

1. var destinationPtr = chunk.GetNativeArray(ComponentType).GetUnsafePtr();
2. var srcPtr = (byte*) ComponentData.GetUnsafeReadOnlyPtr() + UnsafeUtility.SizeOf<T>() * entityOffset;

New Code

Code (CSharp):

1. var src = chunk.m_Chunk->Buffer + typeOffset;
2. var dst = ComponentData + (entityOffset * typeSize);

QED

 

dang I was to slow :-D when will 0.11 come out?

 

@roadis I've read in some other thread that it is planned to come out at the end of this week

 

@roadis Nice catch on the bug!

@AriaBonczek I thought you have automated tests in plac?. This bug shouldn't have gone through.

 

Wait. Those are code blocks from 2 different jobs.

0.10

Code (CSharp):

1. [BurstCompile]
2.     unsafe struct GatherComponentDataJob : IJobChunk
3.     {
4.         [NativeDisableUnsafePtrRestriction] public byte* ComponentData;
5.         public int TypeIndex;
6.  
7.         public void Execute(ArchetypeChunk chunk, int chunkIndex, int entityOffset)
8.         {
9.             var archetype = chunk.Archetype.Archetype;
10.             var indexInTypeArray = ChunkDataUtility.GetIndexInTypeArray(archetype, TypeIndex);
11.             var typeOffset = archetype->Offsets[indexInTypeArray];
12.             var typeSize = archetype->SizeOfs[indexInTypeArray];
13.  
14.             var src = chunk.m_Chunk->Buffer + typeOffset;
15.             var dst = ComponentData + (entityOffset * typeSize);
16.             var copySize = typeSize * chunk.Count;
17.  
18.             UnsafeUtility.MemCpy(dst, src, copySize);
19.         }
20.     }
21.  
22.     [BurstCompile]
23.     unsafe struct CopyComponentArrayToChunks : IJobChunk
24.     {
25.         [NativeDisableUnsafePtrRestriction] public byte* ComponentData;
26.         public int TypeIndex;
27.  
28.         public void Execute(ArchetypeChunk chunk, int chunkIndex, int entityOffset)
29.         {
30.             var archetype = chunk.Archetype.Archetype;
31.             var indexInTypeArray = ChunkDataUtility.GetIndexInTypeArray(archetype, TypeIndex);
32.             var typeOffset = archetype->Offsets[indexInTypeArray];
33.             var typeSize = archetype->SizeOfs[indexInTypeArray];
34.  
35.             var dst = chunk.m_Chunk->Buffer + typeOffset;
36.             var src = ComponentData + (entityOffset * typeSize);
37.             var copySize = typeSize * chunk.Count;
38.  
39.             UnsafeUtility.MemCpy(dst, src, copySize);
40.         }
41.     }

0.9.1

Code (CSharp):

1. [BurstCompile]
2.     unsafe struct GatherComponentDataJob<T> : IJobChunk
3.         where T : struct,IComponentData
4.     {
5.         public NativeArray<T> ComponentData;
6.         [ReadOnly]public ArchetypeChunkComponentType<T> ComponentType;
7.  
8.         public void Execute(ArchetypeChunk chunk, int chunkIndex, int entityOffset)
9.         {
10.             var sourcePtr = chunk.GetNativeArray(ComponentType).GetUnsafeReadOnlyPtr();
11.             var destinationPtr = (byte*) ComponentData.GetUnsafePtr() + UnsafeUtility.SizeOf<T>() * entityOffset;
12.             var copySizeInBytes = UnsafeUtility.SizeOf<T>() * chunk.Count;
13.  
14.             UnsafeUtility.MemCpy(destinationPtr, sourcePtr, copySizeInBytes);
15.         }
16.     }
17.  
18.     [BurstCompile]
19.     unsafe struct CopyComponentArrayToChunks<T> : IJobChunk
20.         where T : struct,IComponentData
21.     {
22.         [ReadOnly]
23.         public NativeArray<T> ComponentData;
24.         public ArchetypeChunkComponentType<T> ComponentType;
25.  
26.         public void Execute(ArchetypeChunk chunk, int chunkIndex, int entityOffset)
27.         {
28.             var destinationPtr = chunk.GetNativeArray(ComponentType).GetUnsafePtr();
29.             var srcPtr = (byte*) ComponentData.GetUnsafeReadOnlyPtr() + UnsafeUtility.SizeOf<T>() * entityOffset;
30.             var copySizeInBytes = UnsafeUtility.SizeOf<T>() * chunk.Count;
31.  
32.             UnsafeUtility.MemCpy(destinationPtr, srcPtr, copySizeInBytes);
33.         }
34.     }

 

No worries @AriaBonczek !