Search Unity

Lists of things, collections, arrays, groups

Discussion in 'Entity Component System' started by illinar, Apr 1, 2018.

  1. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    This was asked in many forms all over the forum but I want to see a comprehensive general answer. How to do lists of different things?

    When I need to find all things 'A' that belong to one of the many 'B'. All connections of a node, all units in a squad, all voxels in an object, all apples on a tree, all vertices of a model. Is there really always the one answer: iterate through all of them? What if I have millions of items? What if I need to navigate them as a 3D array? Should I create an array in a component? Is there a type of component that allows that? MonoBehaviour?

    I just found EntityManager.GetFixedArray<T>(Entity) What is this and how to use it?


    Looking forward to having a custom index systems and control over entity sorting to be able to take advantage of fixed and sorted entity order and direct access to required subsets. And having arrays of component data PER entity (if not already possible).
     
    Last edited: Apr 1, 2018
    illinar, Apr 1, 2018
    #1

  2. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    I just cant... Why did no one tell me that I can have arrays of any data on Entities? :D

    Edit: that is a bad code, but it works somehow, unlike the attempts to make it better. Would only work in an empty projects where no other entities are created.

    Code (CSharp):
    1. public class ArrayTestJobSystem : JobComponentSystem
    2. {
    3.     public struct Data
    4.     {
    5.         public EntityArray voxelObjectEntities;
    6.     }
    7.     [Inject] private EntityManager entityManager;
    8.  
    9.     override protected void OnCreateManager(int capacity)
    10.     {
    11.         var e = EntityManager.CreateEntity(ComponentType.FixedArray(typeof(byte), 10000));
    12.     }
    13.  
    14.     struct TestBytesJob : IJob
    15.     {
    16.         public NativeArray<byte> voxels;
    17.  
    18.         [ComputeJobOptimization]
    19.         public void Execute()
    20.         {
    21.             Debug.Log(voxels[1000]);
    22.             voxels[1000] = (byte)123;
    23.         }
    24.     }
    25.  
    26.     protected override JobHandle OnUpdate(JobHandle inputDeps)
    27.     {
    28.         NativeArray<byte> bytes = entityManager.GetFixedArray<byte>(data.voxelObjectEntities[0]);
    29.         return new TestBytesJob() { voxels = bytes }.Schedule(inputDeps);
    30.     }
    31. }
    This is what I've been specifically asking for on more than one occasion.

    There is a problem though! It crashes for me when the array is more than ~16000 bytes!

    Edit: Now if I make the array bigger than ~16000 bytes it gives me an error message:
    Assertion failed: Assertion failure. Values are equal.
    Expected: 0 != 0
    UnityEngine.Assertions.Assert:AreNotEqual(Int32, Int32)
    Unity.Assertions.Assert:AreNotEqual(Int32, Int32) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/Stubs/Unity.Assertions/Assert.cs:90)
    Unity.Entities.ArchetypeManager:SetChunkCount(Chunk*, Int32) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:534)
    Unity.Entities.ArchetypeManager:AllocateIntoChunk(Chunk*, Int32, Int32&) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ArchetypeManager.cs:527)
    Unity.Entities.EntityDataManager:CreateEntities(ArchetypeManager, Archetype*, Entity*, Int32) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityDataManager.cs:604)
    Unity.Entities.EntityManager:CreateEntityInternal(EntityArchetype, Entity*, Int32) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityManager.cs:215)
    Unity.Entities.EntityManager:CreateEntity(EntityArchetype) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityManager.cs:203)
    Unity.Entities.EntityManager:CreateEntity(ComponentType[]) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/EntityManager.cs:209)
    ArrayTestJobSystem:OnCreateManager(Int32) (at Assets/Scripts/Systems/ArrayTestSystem.cs:39)
    Unity.Entities.ScriptBehaviourManager:CreateInstance(World, Int32) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/ScriptBehaviourManager.cs:26)
    Unity.Entities.World:CreateManagerInternal(Type, Int32, Object[]) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/Injection/World.cs:138)
    Unity.Entities.World:GetOrCreateManagerInternal(Type) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/Injection/World.cs:171)
    Unity.Entities.World:GetOrCreateManager(Type) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities/Injection/World.cs:226)
    Unity.Entities.DefaultWorldInitialization:GetBehaviourManagerAndLogException(World, Type) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities.Hybrid/Injection/DefaultWorldInitialization.cs:19)
    Unity.Entities.DefaultWorldInitialization:Initialize(String, Boolean) (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities.Hybrid/Injection/DefaultWorldInitialization.cs:54)
    Unity.Entities.AutomaticWorldBootstrap:Initialize() (at C:/ProgramData/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.11/Unity.Entities.Hybrid/Injection/AutomaticWorldBootstrap.cs:11)
     
    Last edited: Apr 1, 2018
    illinar, Apr 1, 2018
    #2

  3. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    I think there is a bug. ComponentSystem works fine like this:

    Code (CSharp):
    1. public class ArrayTestSystem : ComponentSystem
    2. {
    3.     public struct Data
    4.     {
    5.         public EntityArray entities;
    6.         public ComponentDataArray<VoxelObject> v;
    7.     }
    8.     [Inject] private Data data;
    9.     [Inject] private EntityManager entityManager;
    10.  
    11.     override protected void OnCreateManager(int capacity)
    12.     {
    13.         var e = EntityManager.CreateEntity(typeof(VoxelObject), ComponentType.FixedArray(typeof(byte), 16000));
    14.     }
    15.     protected override void OnUpdate()
    16.     {
    17.         NativeArray<byte> bytes = entityManager.GetFixedArray<byte>(data.entities[0]);
    18.         Debug.Log(bytes[0]);
    19.         bytes[0] = (byte)1;
    20.     }
    21. }
    But in a job system some of those lines cause weird problems:

    Code (CSharp):
    1. public class ArrayTestJobSystem : JobComponentSystem
    2. {
    3.     public struct Data
    4.     {
    5.         public EntityArray entities;
    6.         // public ComponentDataArray<VoxelObject> v; //  CAUSES THE PROBLEM IN THE JOB.
    7.     }
    8.     [Inject] private Data data;
    9.     [Inject] private EntityManager entityManager;
    10.  
    11.     override protected void OnCreateManager(int capacity)
    12.     {
    13.         // var e = EntityManager.CreateEntity(typeof(VoxelObject), ComponentType.FixedArray(typeof(byte), 10000));
    14.         // CAUSES THE PROBLEM IN THE JOB when VoxelObject is added before the array. Works fine in ComponentSystem.
    15.         var e = EntityManager.CreateEntity(ComponentType.FixedArray(typeof(byte), 10000), typeof(VoxelObject));
    16.     }
    17.  
    18.     struct TestBytesJob : IJob
    19.     {
    20.         public NativeArray<byte> bytes;
    21.  
    22.         [ComputeJobOptimization]
    23.         public void Execute()
    24.         {
    25.             // THIS LINE TROWS OUT OF RANGE EXCEPTION
    26.             // if any of the commented lines above are used
    27.             // despite the fact that the array always has length of 10000.
    28.             Debug.Log(bytes[1]);
    29.             bytes[1] = (byte)123;
    30.         }
    31.     }
    32.  
    33.     protected override JobHandle OnUpdate(JobHandle inputDeps)
    34.     {
    35.         NativeArray<byte> bytes = entityManager.GetFixedArray<byte>(data.entities[0]);
    36.         Debug.Log(bytes.Length); // Always shows correct array size 10000
    37.         return new TestBytesJob() { bytes = bytes }.Schedule(inputDeps);
    38.     }
    39. }
     
    Last edited: Apr 1, 2018
    illinar, Apr 1, 2018
    #3

  4. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,657
    Another approach you can use is to use ComponentGroup.SetFilter() with an ISharedComponentData. To pick one example:

    You could have a 
    SquadMembership
    component on your unit entities, and then do something like:

    Code (CSharp):
    1. ComponentGroup units = GetComponentGroup(typeof(Unit), ...other components you want...);
    2. units.SetFilter(new SquadMembership{ SquadID = desiredSquadId });
    You can then iterate over all the data in the ComponentGroup, and it will only give you entities that are in the desired squad.
     
    superpig, Apr 1, 2018
    #4

  5. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    @superpig Oh, interesting, thank you. I was wondering if there is such API. How does Group API work with injection? I heard one of those approaches might be removed. So if I want to have a filter, do I have to use groups instead of injection?
     
    illinar, Apr 1, 2018
    #5

  6. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    Code (CSharp):
    1. NativeArray<byte> bytes = entityManager.GetFixedArray<byte>(data.entities[0]);
    2.  
    You are accessing the data through EntityManager which means the ComponentSystem knows nothing about the data dependencies...

    The easiest way is to grab the data via injection.

    1. Code (CSharp):
      1. public struct Data
      2.     {
      3.         public EntityArray entities;
      4.         public ComponentDataArray<VoxelObject> v;
      5.         public FixedArrayArray<byte> bytes;
      6.     }
      Alternatively ComponentSystem.GetComponentGroup() works too.
     
    Joachim_Ante, Apr 1, 2018
    #6
    Zoey_O likes this.

  7. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    @Joachim_Ante this is great, thank you, works perfectly now.

    Code (CSharp):
    1. public class ArrayTestJobSystem : JobComponentSystem
    2. {
    3.     public struct Data
    4.     {
    5.         public FixedArrayArray<byte> voxelArrays;
    6.         public ComponentDataArray<VoxelObject> v;
    7.     }
    8.     [Inject] private Data data;
    9.  
    10.     [ComputeJobOptimization]
    11.     struct TestBytesJob : IJob
    12.     {
    13.         public FixedArrayArray<byte> arrays;
    14.  
    15.         public void Execute()
    16.         {
    17.             NativeArray<byte> bytes = arrays[0];
    18.             bytes[0] = (byte)1;
    19.         }
    20.     }
    21.  
    22.     protected override JobHandle OnUpdate(JobHandle inputDeps)
    23.     {
    24.         Debug.Log(data.voxelArrays[0][0]);
    25.         return new TestBytesJob() { arrays = data.voxelArrays }.Schedule(inputDeps);
    26.     }
    27. }
     
    Last edited: Apr 1, 2018
    illinar, Apr 1, 2018
    #7

  8. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    No one told me I can have arrays on SharedComponentData... (I pretty much have three threads where I asked about ways to store arrays.) It has no size limits, can change size, might be more useful in some situations.
     
    Last edited: Apr 6, 2018
    illinar, Apr 6, 2018
    #8

  9. andywatts

    andywatts

    Joined:
    Sep 19, 2015
    Posts:
    112
    Would you mind sharing a code sample of your SharedComponentData implementation?
     
    andywatts, Apr 8, 2018
    #9

  10. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    SharedComponentData with array:

    Code (CSharp):
    1. using UnityEngine;
    2. using Unity.Entities;
    3. using Unity.Collections;
    4.  
    5. public struct Grid : ISharedComponentData { public NativeArray<Cell> cells; }
    6. public struct Cell : IComponentData { public int value; }
    7.  
    8. public class CreateGridSystem : ComponentSystem
    9. {
    10.     struct Data
    11.     {
    12.         [ReadOnly] public SharedComponentDataArray<Grid> grid;
    13.     }
    14.     [Inject] Data data;
    15.  
    16.     override protected void OnCreateManager(int capacity)
    17.     {
    18.         var e = EntityManager.CreateEntity(typeof(Grid));
    19.         var cells = new NativeArray<Cell>(32000, Allocator.Persistent);
    20.         cells[0] = new Cell { value = 1 };
    21.         EntityManager.SetSharedComponentData<Grid>(e, new Grid { cells = cells });
    22.     }
    23.  
    24.     protected override void OnUpdate()
    25.     {
    26.         Debug.Log(data.grid[0].cells.Length);
    27.         Debug.Log(data.grid[0].cells[0].value);
    28.         var grid = data.grid[0];
    29.         grid.cells[0] = new Cell { value = 2 };
    30.     }
    31. }
     
    illinar, Apr 8, 2018
    #10

  11. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    Turns out SharedComponentData doesn't work as a static class or doesn't share the same data across all its instances.
    If you create two SCD Grid from the example above and will give a new array to each it won't override. Instead, each entity will have it's own.

    Code (CSharp):
    1. using UnityEngine;
    2. using Unity.Entities;
    3. using Unity.Collections;
    4.  
    5. public struct Grid : ISharedComponentData { public NativeArray<Cell> cells; }
    6. public struct Cell : IComponentData { public int value; }
    7.  
    8. public class CreateGridSystem : ComponentSystem
    9. {
    10.     struct Data
    11.     {
    12.         [ReadOnly] public SharedComponentDataArray<Grid> grid;
    13.     }
    14.     [Inject] Data data;
    15.  
    16.     override protected void OnCreateManager(int capacity)
    17.     {
    18.         var e = EntityManager.CreateEntity(typeof(Grid));
    19.         var cells = new NativeArray<Cell>(1, Allocator.Persistent);
    20.         cells[0] = new Cell { value = 1 };
    21.         EntityManager.SetSharedComponentData<Grid>(e, new Grid { cells = cells });
    22.  
    23.         var e2 = EntityManager.CreateEntity(typeof(Grid));
    24.         var cells2 = new NativeArray<Cell>(1, Allocator.Persistent);
    25.         cells2[0] = new Cell { value = 2 };
    26.         EntityManager.SetSharedComponentData<Grid>(e2, new Grid { cells = cells2 });
    27.     }
    28.  
    29.     protected override void OnUpdate()
    30.     {
    31.         Debug.Log(data.grid[0].cells[0].value);
    32.         Debug.Log(data.grid[1].cells[0].value);
    33.  
    34.         var grid = data.grid[0];
    35.         grid.cells[0] = new Cell { value = 3 };
    36.  
    37.         var grid2 = data.grid[1];
    38.         grid2.cells[0] = new Cell { value = 4 };
    39.     }
    40. }
    Can somebody please explain what is "Shared" in SharedComponentData?

    Oh.. Can I set the same SCD to multiple entities? That makes sense.
     
    Last edited: Apr 8, 2018
    illinar, Apr 8, 2018
    #11
    xXPancakeXx likes this.

  12. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    So, yes, lastly, you can add the same SharedComponentData to any entities you want. I assume SCD is not so tightly linearly packed with the rest of the entity but in some cases, it is, of course, a better approach anyway.
    Code (CSharp):
    1. using UnityEngine;
    2. using Unity.Entities;
    3. using Unity.Collections;
    4.  
    5. public struct Grid : ISharedComponentData { public NativeArray<Cell> cells; }
    6. public struct Cell : IComponentData { public int value; }
    7.  
    8. public class CreateGridSystem : ComponentSystem
    9. {
    10.     struct Data
    11.     {
    12.         [ReadOnly] public SharedComponentDataArray<Grid> grid;
    13.     }
    14.     [Inject] Data data;
    15.  
    16.     override protected void OnCreateManager(int capacity)
    17.     {
    18.         var e = EntityManager.CreateEntity(typeof(Grid));
    19.         var e2 = EntityManager.CreateEntity(typeof(Grid));
    20.         var cells = new NativeArray<Cell>(1, Allocator.Persistent);
    21.         cells[0] = new Cell { value = 1 };
    22.         var grid = new Grid() { cells = cells };
    23.         EntityManager.SetSharedComponentData<Grid>(e, grid);
    24.         EntityManager.SetSharedComponentData<Grid>(e2, grid);
    25.     }
    26.  
    27.     protected override void OnUpdate()
    28.     {
    29.         Debug.Log(data.grid[0].cells[0].value);
    30.         Debug.Log(data.grid[1].cells[0].value);
    31.  
    32.         var grid = data.grid[0];
    33.         grid.cells[0] = new Cell { value = 3 };
    34.  
    35.         var grid2 = data.grid[1];
    36.         grid2.cells[0] = new Cell { value = 4 };
    37.     }
    38. }
     
    illinar, Apr 8, 2018
    #12
    andywatts likes this.

  13. xenonsin

    xenonsin

    Joined:
    Dec 12, 2013
    Posts:
    20
    Thanks illinar for sharing NativeArrays work for ISharedComponentData. Been trying to figure out how to store arrays of things.. where filtering won't make too much sense.
     
    xenonsin, Apr 18, 2018
    #13
    sngdan likes this.

  14. xenonsin

    xenonsin

    Joined:
    Dec 12, 2013
    Posts:
    20
    I wonder, is it best practice to have a SharedComponentData with a NativeArray for components that won't actually be sharing components?

    Let's say, I'm generating a map and I want to be able to know which rooms link to each other.

    upload_2018-4-18_19-16-48.png 
    public struct Room: ISharedComponentData { public NativeArray<float2> Links; }


    A would contain no links, B would contain an array of 2, C would contain 1 link. Is this fine?
     
    xenonsin, Apr 19, 2018
    #14

  15. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    559
    SharedComponentData means if they are different they will be put in different chunks, losing linear memory access and consuming an entire chunk (currently 16kb) per entity.

    you may rather use FixedArray or explicitly enumerating each link (i.e.)
    Code (CSharp):
    1. public struct Room : IComponentData { public float2 north, south, west, east;}
    or
    Code (CSharp):
    1. public struct NorthLink: IComponentData { float2 value; }
    2. public struct SouthLink: IComponentData { float2 value; }
    3. public struct WestLink: IComponentData { float2 value; }
    4. public struct EastLink: IComponentData { float2 value; }
    5.  
     
    M_R, Apr 19, 2018
    #15

  16. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    863
    How to add Fixed array via PostUpdateCommands or via AddComponent??
     
    illinar, May 17, 2018
    #16
    andywatts likes this.