Search Unity

Custom 2D/3D indexes for ComponentData access

Discussion in 'Data Oriented Technology Stack' started by illinar, Apr 11, 2018.

  1. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    607
    I have an API request. Haven't figured out how it could work, but I'm sure it could, and it is a functionality needed all the time in any project I imagined or tried working on.

    When I receive injected ComponentDataArray<> I want to be able to access it's members by a 2D or 3D index that it's members have (or even 1D index in a custom array).

    For example, I have a 2D grid of tile entities. A system gets injected with a specific type of tiles and needs to query them by their 2D position. So I need to be able to get tiles from ComponentDataArray<Tile> by its position, instead of iterating over all the tiles and comparing positions.

    More specific example with code for my current approach that I would love to see replaced by built-in API:

    There is a 2D grid of block entities. Explosive blocks can explode and destroy Destructible blocks around them. ExplodingBlock system receives all Explosive blocks and all Destructible blocks separately. It uses a BlockMap class to map all the Destructible blocks to a 2D array representing the whole block grid. Then it uses that class to get ComponentData index of Destructible blocks by its 2D position, to mark selected blocks as Destroyed.

    Code (CSharp):
    1. using UnityEngine;
    2. using Unity.Entities;
    3. using Unity.Collections;
    4.  
    5. public struct Block : IComponentData { public int PosX, PosY; }
    6. public struct Explosive : IComponentData { public int Radius; }
    7. public struct Destroyed : IComponentData { }
    8. public struct Destructible : IComponentData { }
    9.  
    10. // Converts 2D block positions to linear indexes in a given collection of blocks.
    11. public class BlockMap
    12. {
    13.     private NativeArray<int> indices = new NativeArray<int>(Grid.MaxSize * Grid.MaxSize, Allocator.Persistent);
    14.  
    15.     public void Update(ComponentDataArray<Block> positions)
    16.     {
    17.         for (int i = 0; i < indices.Length; i++)
    18.         {
    19.             indices[i] = -1;
    20.         }
    21.         for (int i = 0; i < positions.Length; i++)
    22.         {
    23.             indices[positions[i].PosY * Grid.MaxSize + positions[i].PosX] = i;
    24.         }
    25.     }
    26.  
    27.     public int this[int x, int y]
    28.     {
    29.         get
    30.         {
    31.             if (x > 0 && x < Grid.MaxSize && y > 0 && y < Grid.MaxSize)
    32.             {
    33.                 var i = Grid.MaxSize * y + x;
    34.                 return indices[i];
    35.             }
    36.             else return -1;
    37.         }
    38.     }
    39. }
    40.  
    41. public class ExplodingBlocksSystem : ComponentSystem
    42. {
    43.     private BlockMap map = new BlockMap();
    44.  
    45.     private struct ExplosiveBlocks
    46.     {
    47.         public int Length;
    48.         [ReadOnly] public ComponentDataArray<Explosive> Explosive;
    49.         [ReadOnly] public ComponentDataArray<Block> Blocks;
    50.     }
    51.     [Inject] private ExplosiveBlocks explosiveBlocks;
    52.  
    53.     private struct DestructibleBlocks
    54.     {
    55.         [ReadOnly] public ComponentDataArray<Destructible> Destructible;
    56.         [ReadOnly] public ComponentDataArray<Block> Blocks;
    57.         [ReadOnly] public EntityArray Entities;
    58.     }
    59.     [Inject] private DestructibleBlocks destructibleBlocks;
    60.  
    61.     protected override void OnUpdate()
    62.     {
    63.         map.Update(destructibleBlocks.Blocks);
    64.         for (int i = 0; i < explosiveBlocks.Length; i++)
    65.         {
    66.             var radius = explosiveBlocks.Explosive[i].Radius;
    67.             var posY = explosiveBlocks.Blocks[i].PosY;
    68.             var posX = explosiveBlocks.Blocks[i].PosX;
    69.             for (int y = -radius; y <= radius; y++)
    70.             {
    71.                 for (int x = -radius; x <= radius; x++)
    72.                 {
    73.                     var index = map[posX + x, posY + y];
    74.                     if (index >= 0)
    75.                     {
    76.                         EntityManager.AddComponentData<Destroyed>(destructibleBlocks.Entities[index], new Destroyed());
    77.                     }
    78.                 }
    79.             }
    80.         }
    81.     }
    82. }
    I would appreciate comments from Unity team on whether or not it is possible to have ECS handle this behind the scenes and provide API to define custom indexes and to get component data by custom indexes.

    Of course, Group.Filter() doesn't work due to the inefficiency of iterating over all entities each time accessing one when I need to do it thousands of times per frame.

    I would appreciate any suggestions on improving my approach meanwhile.

    Maybe I can use:
    var map = new BlockMap(destructibleBlocks.Blocks);
    every frame if that won't cause garbage allocation or other problems. Instead of:
    map.Update(destructibleBlocks.Blocks);
    and a class field. Because I would like to get rid of the field, and to not risk forgetting to Update the map, otherwise unpredictable behavior is guaranteed. And of course, I could use interface for getting indexes instead of Block class.
     
    Last edited: Apr 11, 2018
  2. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    456
    Ya. Currently it does not have the ability to directly return the data you want with the custom index you defined like C# Dictionary. It will need a something like [PrimaryEntityIndex] to query the only one unique data and [EntityIndex] to query multiple data with the same index. I expect something like this works out of the box.

    Code (CSharp):
    1. struct Data
    2.     {
    3.         private ComponentDataArray<Position> Position;
    4.         [PrimaryEntityIndex] Position = new Position(2, 2)
    5.     }
    6.  
    7.     [Inject] private Data Data;
     
  3. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    607
    I know Entitas does it as Dictionary. I'm also thinking arrays might be much more performant in many cases and it should be possible to make some decent API for them.

    [EntityArrayIndex(ArraySizeX, ArraySizeY)] 
    (idk..)

    Would be great to then be able to access data in ComponentDataArray<Health> like this:

    blocks.Health[0,3] = 100;
     
    Last edited: Apr 11, 2018
  4. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    68
    ComponentDataArray is heavily optimized for going through everything linearly. If you need to do random access of it copying the data to another structure is usually faster than using the ComponentDataArray. Because of this I'm pretty sure accessing the ComponentDataArray with a key like that would have really bad performance.

    We do similar things for the boid demo in our ECS repository, copy the boids to a sparse grid backed by a NativeMultiHashMap for faster spatial lookups. It might be possible to generalize that to make what you want to do easier, but it is not something we have on the top of our priority list right now so you should not expect it short term.
     
    laurentlavigne likes this.
  5. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    607
    Thank you for the reply. I never suspected that ComponentDataArray is bad for random access. I will benchmark my approach vs using Array, List, Dictionary or NativeHashMap for my actual real use case, and I think it will be good enough for now.

    Those are no doubt widely used scenarios (2D, 3D array querying), I hope there will be some easy ways to do it somewhere in the future.
     
  6. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    603
    I suspect the best thing to do right now is custom native collections or adaptors over existing native collections, like NativeSlice.
     
  7. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    2,038
    We're still in alpha or beta so that's cool that your priorities are elsewhere, but for release it will be important to have these utilities, not because they're hard to make ourselves, but to help ECS noobs like me understand this mega change in the way we think.
     
  8. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    607
    Oh.. I misunderstood @timjohansson. I thought using native arrays for storing indexes is a problem. But we are talking about ComponentDataArray.

    So I would need to copy the data into a 2D array instead of just indexes? Is that a good approach? Can it have some reasonably good syntax? What are the alternatives?

    I guess I can have a Generic data map class that would copy multiple component data arrays to 2D arrays and will have semi-okay API to access them. Filtering API is a bit heavy and feels counterintuitive (slow).
     
    Last edited: Apr 12, 2018
  9. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    607
    So now I have this to copy data into 2D array:
    Code (CSharp):
    1. public class DataMap2D<TIndexComponent, TData1> where TIndexComponent : struct, IComponentData, IIndexComponent2D
    2.                                                 where TData1 : struct, IComponentData
    3. {
    4.     private NativeArray<Entity> entities;
    5.     private NativeArray<TData1> data1;
    6.     private NativeArray<bool> exists;
    7.     private int sizeX, sizeY;
    8.  
    9.     public DataMap2D(int sizeX, int sizeY)
    10.     {
    11.         this.sizeX = sizeX;
    12.         this.sizeY = sizeY;
    13.         int arraySize = sizeX * sizeY;
    14.         data1 = new NativeArray<TData1>(arraySize, Allocator.Persistent);
    15.         entities = new NativeArray<Entity>(arraySize, Allocator.Persistent);
    16.     }
    17.  
    18.     public void Update(ComponentDataArray<TIndexComponent> indexComponentData, EntityArray entities)
    19.     {
    20.         Clear();
    21.         for (int i = 0; i < indexComponentData.Length; i++)
    22.         {
    23.             var index = FlatIndex(indexComponentData[i].X, indexComponentData[i].Y);
    24.             this.entities[index] = entities[i];
    25.         }
    26.     }
    27.  
    28.     public void Update(ComponentDataArray<TIndexComponent> indexComponentData, EntityArray entities, ComponentDataArray<TData1> componentData1)
    29.     {
    30.         Clear();
    31.         for (int i = 0; i < indexComponentData.Length; i++)
    32.         {
    33.             var index = FlatIndex(indexComponentData[i].X, indexComponentData[i].Y);
    34.             this.entities[index] = entities[i];
    35.             data1[index] = componentData1[i];
    36.         }
    37.     }
    38.  
    39.     public bool Exists(int x, int y)
    40.     {
    41.         return exists[FlatIndex(x, y)];
    42.     }
    43.  
    44.     public Entity GetEntity(int x, int y)
    45.     {
    46.         return entities[FlatIndex(x, y)];
    47.     }
    48.  
    49.     public TData1 GetData1(int x, int y)
    50.     {
    51.         return data1[FlatIndex(x, y)];
    52.     }
    53.  
    54.     private int FlatIndex(int x, int y)
    55.     {
    56.         if (x < 0 || y < 0 || x > sizeX || y > sizeY)
    57.             throw new System.ArgumentOutOfRangeException();
    58.         return sizeX * y + x;
    59.     }
    60.  
    61.     private void Clear()
    62.     {
    63.         int arraySize = sizeX * sizeY;
    64.         for (int i = 0; i < arraySize; i++)
    65.         {
    66.             exists[i] = false;
    67.         }
    68.     }
    69. }
    Usage:
    Code (CSharp):
    1. public class ExplodingBlocksSystem : ComponentSystem
    2. {
    3.     private DataMap2D<BlockPosition> map = new DataMap2D<BlockPosition>(Grid.MaxSize, Grid.MaxSize);
    4.  
    5.     private struct ExplosiveBlocks
    6.     {
    7.         public int Length;
    8.         [ReadOnly] public ComponentDataArray<Explosive> Explosive;
    9.         [ReadOnly] public ComponentDataArray<BlockPosition> Positions;
    10.         [ReadOnly] public ComponentDataArray<Block> Blocks;
    11.     }
    12.     [Inject] private ExplosiveBlocks explosiveBlocks;
    13.  
    14.     private struct DestructibleBlocks
    15.     {
    16.         [ReadOnly] public ComponentDataArray<Destructible> Destructible;
    17.         [ReadOnly] public ComponentDataArray<BlockPosition> Positions;
    18.         [ReadOnly] public ComponentDataArray<Block> Blocks;
    19.         [ReadOnly] public EntityArray Entities;
    20.     }
    21.     [Inject] private DestructibleBlocks destructibleBlocks;
    22.  
    23.     protected override void OnUpdate()
    24.     {
    25.         map.Update(destructibleBlocks.Positions, destructibleBlocks.Entities);
    26.         for (int i = 0; i < explosiveBlocks.Length; i++)
    27.         {
    28.             var radius = explosiveBlocks.Explosive[i].Radius;
    29.             var posY = explosiveBlocks.Positions[i].Y;
    30.             var posX = explosiveBlocks.Positions[i].X;
    31.             for (int y = -radius; y <= radius; y++)
    32.             {
    33.                 for (int x = -radius; x <= radius; x++)
    34.                 {
    35.                     if (map.Exists(x, y))
    36.                     {
    37.                         EntityManager.AddComponentData<Destroyed>(map.GetEntity(posX, posY), new Destroyed());
    38.                     }
    39.                 }
    40.             }
    41.         }
    42.     }
    43. }
    There are Versions without TData and with two or more TData.
     
    Last edited: Apr 13, 2018