Search Unity

Does FixedArray have a size limit?

Discussion in 'Entity Component System' started by andywatts, Apr 16, 2018.

  1. andywatts

    andywatts

    Joined:
    Sep 19, 2015
    Posts:
    112
    Hi,

    When adding a FixedArray to an entity I get an error if over a certain size.

    Code (CSharp):
    1. entityManager.AddComponent(entity, ComponentType.FixedArray(typeof(int), size) );
    Strangely, the above works with size <4070, but throws the below errors with size >4071.

    Assertion failed: Assertion failure. Values are equal.
    Expected: 0 != 0
    UnityEngine.Assertions.Assert:AreNotEqual(Int32, Int32)
    Unity.Assertions.Assert:AreNotEqual(Int32, Int32) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/Stubs/Unity.Assertions/Assert.cs:90)
    Unity.Entities.ArchetypeManager:SetChunkCount(Chunk*, Int32) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:534)
    Unity.Entities.ArchetypeManager:AllocateIntoChunk(Chunk*, Int32, Int32&) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:527)
    Unity.Entities.ArchetypeManager:AllocateIntoChunk(Chunk*) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:518)
    Unity.Entities.EntityDataManager:SetArchetype(ArchetypeManager, Entity, Archetype*, Int32*) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityDataManager.cs:388)
    Unity.Entities.EntityDataManager:AddComponent(Entity, ComponentType, ArchetypeManager, SharedComponentDataManager, EntityGroupManager, ComponentTypeInArchetype*) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityDataManager.cs:494)
    Unity.Entities.EntityManager:AddComponent(Entity, ComponentType) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityManager.cs:280)


    IndexOutOfRangeException: Index was outside the bounds of the array.
    (wrapper stelemref) System.Object.virt_stelemref_object(intptr,object)
    Unity.Entities.ArchetypeManager.SetManagedObject (Unity.Entities.Chunk* chunk, System.Int32 type, System.Int32 index, System.Object val) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:602)
    Unity.Entities.ChunkDataUtility.CopyManagedObjects (Unity.Entities.ArchetypeManager typeMan, Unity.Entities.Chunk* srcChunk, System.Int32 srcStartIndex, Unity.Entities.Chunk* dstChunk, System.Int32 dstStartIndex, System.Int32 count) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ChunkDataUtility.cs:224)
    Unity.Entities.EntityDataManager.SetArchetype (Unity.Entities.ArchetypeManager typeMan, Unity.Entities.Entity entity, Unity.Entities.Archetype* archetype, System.Int32* sharedComponentDataIndices) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityDataManager.cs:395)
    Unity.Entities.EntityDataManager.AddComponent (Unity.Entities.Entity entity, Unity.Entities.ComponentType type, Unity.Entities.ArchetypeManager archetypeManager, Unity.Entities.SharedComponentDataManager sharedComponentDataManager, Unity.Entities.EntityGroupManager groupManager, Unity.Entities.ComponentTypeInArchetype* componentTypeInArchetypeArray) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityDataManager.cs:494)
    Unity.Entities.EntityManager.AddComponent (Unity.Entities.Entity entity, Unity.Entities.ComponentType type) (at /Users/andywatt/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityManager.cs:280)
    ChunkSpawner.Start () (at Assets/ChunkSpawner.cs:58)
    ​
     
    Last edited: Apr 16, 2018
    andywatts, Apr 16, 2018
    #1

  2. andywatts

    andywatts

    Joined:
    Sep 19, 2015
    Posts:
    112
    Here's a sample to recreate the error...

    Code (CSharp):
    1. using Unity.Entities;
    2. using UnityEngine;
    3. using Unity.Collections;
    4.  
    5. public class FixedArrayTest : MonoBehaviour {
    6.  
    7.     private void Start() {
    8.         var entityManager = World.Active.GetOrCreateManager<EntityManager>();
    9.         var entity = entityManager.CreateEntity();
    10.         entityManager.AddComponent(entity, ComponentType.FixedArray(typeof(int), 4096) );
    11.     }
    12. }
     
    andywatts, Apr 17, 2018
    #2

  3. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    There is no direct limit on the number of items in a FixedArray, but right now there is a limit on the size of the data belonging to an Entity.
    All data for entities are stored in chunks of memory which currently are 16k - but we are still tweaking that. Those chunks also contain some metadata which means the memory available for data is slightly less. If an entity is larger than one chunk you will get the errors you are getting above.
    So in short, how many items you can have in your fixed array is right now limited - but the exact limit depends on what other components you have on the entity and what type your FixedArray is storing.

    The fact that it just breaks without a clear error message is a bug, either the limit should be relaxed or there should at the very least be a clear error message about it.
     
    timjohansson, Apr 17, 2018
    #3
    charlesamat, optimise and andywatts like this.

  4. andywatts

    andywatts

    Joined:
    Sep 19, 2015
    Posts:
    112
    Do you anticipate it going much higher than 16k data per entity?

    If we're staying ~16k, I'll guess larger data stays in monobehaviors.
    Unless there's another pattern I'm missing.
     
    andywatts, Apr 17, 2018
    #4

  5. Spy-Shifty

    Spy-Shifty

    Joined:
    May 5, 2011
    Posts:
    546
    Maybe instead of assigning that array to the entity itself...
    Create entities that stores the data for you..

    Code (CSharp):
    1. struct Owner : ISharedComponentData {
    2.       public Entity value;
    3. }
    4.  
    5. public IntegerData : ComponentData {
    6.        public int value;
    7. }
    8.  
    9.  
    10. var dataEntity = entityManager.CreateEntity(typeof(Owner), typeof(IntegerData));
    11. entityManager.SetSharedComponentData(dataEntity, new Owner { value = ownerEntity });
    12. entityManager.SetComponentData(new IntegerData{ value = 1 });
    13.  
    14.  
    15. class MySystem : ComponentSystem {
    16.      struct OwnerData {
    17.            public int Length;
    18.            public EntityArray entities;
    19.            // some more filter...
    20.      }
    21.      [Inject] OwnerData ownerData;
    22.      ComponentGroup dataGroup;
    23.  
    24.      public override void OnCreateEntityManager() {
    25.              var dataGroup = EntityManager.GetComponentGroup(typeof(Owner), typeof(ComponentData));
    26.      }
    27.  
    28.       public overrride void OnUpdate() {
    29.               for(int i = 0; i < ownerData.Length; i++) {
    30.                        dataGroup.SetFiler(new Owner { value = ownerData.entities[i] });
    31.                        var data = dataGroup.GetComponentDataArray<ComponentData>();
    32.                        for(int j = 0; j < data.Length; j++) {
    33.                                //data[j].value...
    34.                        }
    35.               }
    36.      }
    37. }
    38.  
    39.  
     
    Spy-Shifty, Apr 17, 2018
    #5
    andywatts likes this.

  6. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The best pattern for large data depends on what kind of data it is. Some patterns we have used for large data in various cases is to split it into multiple entities, store it in some kind of pool and keep just a handle to it in the entity or store the data in a shared component - which is pretty similar to pool + handle. I don't think having that big entities will be good for performance even if we do life the size restriction for entities.
     
    timjohansson, Apr 17, 2018
    #6

  7. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    669
    @timjohansson - will it eventually be possible to specify / limit chunk sizes for certain archetypes? If we have some lightweight archetypes and a known maximum, this would be useful for controlling memory usage on low-end mobile.
     
    recursive, Apr 17, 2018
    #7

  8. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    We'll probably have to tweak the memory allocation patterns for chunks at some point in the future, and exposing a user configurable default size plus per archetype overrides seems like a good idea. Aligning it more with the os / hw page size would also be good and might limit how fine grained control you can get.
    So I guess the answer is, seems like a good idea to me - but no one is working on it right now so I don't know for sure when it will happen nor exactly how much control it will expose.
     
    timjohansson, Apr 18, 2018
    #8
    MegamaDev and recursive like this.

  9. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    669
    I see, even if it's just a factor of hw page size it would still be useful.
     
    recursive, Apr 18, 2018
    #9
    MegamaDev and FROS7 like this.