Determinism in CopyEntitiesFrom and MoveEntitiesFrom

I've been trying to get to the bottom of some non-determinism in the initial state of a custom simulation world. Using a SceneSystem in the custom world seems to load subscene contents deterministically, but copying the same entities out of e.g. the default world leads to non-deterministic results.

Are these methods supposed to be deterministic? Is there a better way to additively copy entities into a world with deterministic order?

 

You should never rely on entities determinism.
Most likely you may get chance, to get same orders of entities indexes few times in a session.
Then after restart game / Unity, orders may change.
That specifically is true, if you use jobs and entities to create them.
Also, entities orders may change at runtime.
Plus, CPU architecture may also affect orders of entities.

If you change something in the scenes, your loading orders of entities will most likely change.
Likely entities will keep order, if you use for example EntityManager, at loading time.
But safest is, to use Native Collection, to track entities, to keep them in order as you desire.

 

I have a similar use case and I tracked down the divergence to this method call:

For some reason, this call will not always order the new chunks in the same way. This causes the following "var instantiated" query to get the chunks in a different order and MoveEntitiesFromInternalQuery to move entities in a different order (so the Entities in the destination world will have different indices).

I found the divergence by diff'ing all the Entity debug infos like this:

and using WinMerge to compare the files between two runs.

Only the _postAddComp.txt files had a divergence. The two files before were identical.

I'll try to dig a bit more. Would you know what could cause AddComponent to produce different results? In both runs, the world is pretty much brand new and empty (except for system entities).

 

Ok, I found the issue.

EntityManager.CopyEntitiesFrom uses
EntityManager.AddComponent which uses
EntityComponentStore.CreateEntityBatchList which uses
EntityComponentStore.SortEntityInChunk which is not deterministic because ptr values are used to sort.

 

I think making EntityManager.AddComponent deterministic would be great, since it's a very commonly used method.

 

CopyEntitiesFrom sister function CopyAndReplaceEntitiesFrom says it is deterministic and during my quick look at I did not see AddComponent. I was never able to test CopyAndReplaceEntitiesFrom because of other bugs. If AddComponent is nondeterministic then that really breaks determinism for a lot of code.

 

Unfortunately, since each scene load adds new entities, I believe I cannot use CopyAndReplace in my use case. Every call would erase the existing entities.

 

For anyone who wants EntityManager.AddComponent to be deterministic, here's what worked for me:
In EntityInChunk.cs, change this:

Code (CSharp):

1. public int CompareTo(EntityInChunk other)
2. {
3.     ulong lhs = (ulong)Chunk;
4.     ulong rhs = (ulong)other.Chunk;
5.     int chunkCompare = lhs < rhs ? -1 : 1;
6.     int indexCompare = IndexInChunk - other.IndexInChunk;
7.     return (lhs != rhs) ? chunkCompare : indexCompare;
8. }

to this:

Code (CSharp):

1.  
2. public int CompareTo(EntityInChunk other)
3. {
4.     // order by archetype first
5.     int archetypeCompare = Chunk->Archetype->StableHash.CompareTo(other.Chunk->Archetype->StableHash);
6.     if (archetypeCompare != 0)
7.         return archetypeCompare;
8.  
9.     // then by chunk index (within archetype chunk list)
10.     int lhs = Chunk->ListIndex;
11.     int rhs = other.Chunk->ListIndex;
12.     int chunkCompare = lhs < rhs ? -1 : 1;
13.  
14.     // then by entity index
15.     int indexCompare = IndexInChunk - other.IndexInChunk;
16.     return (lhs != rhs) ? chunkCompare : indexCompare;
17. }

I'll update this post if I ever find an issue with this approach (crash, bug, etc.).