Search Unity

ECS - The "Correct" way to handle complex shared data between systems

Discussion in 'Data Oriented Technology Stack' started by TickTakashi, May 20, 2018.

  1. TickTakashi

    TickTakashi

    Joined:
    Mar 14, 2013
    Posts:
    30
    Hi, new to ECS (and the forums).

    I'm trying to build something that isn't typical of the ECS examples (large groups of identical objects) given in talks, both to exercise the claim that anything can be built in ECS and become familiar with the concepts. I'm essentially building a board game, with many unique pieces.

    I'm trying to understand the best way to handle shared data structures. In this case, the Board.

    Many of the systems in the game need to ask questions like:
    "is space (a, b) occupied?"
    "which piece is in position (a, b)?"

    "is (a, b) -> (c, d) a valid move?"
    "is (c, d) on the board?"


    One more concrete example, is that the movement system needs to know what terrain a piece is currently standing on, and what terrain it will be moving into, to calculate whether or not it's a valid move. The point is, sometimes a system that handles the set of Pieces needs to know about arbitrary Tiles, and vice versa.

    Coming from an OOP mindset, one simple solution would be to have a Board object that exposes Maps of Position -> Tile and Position -> Piece, and then systems would check that dictionary for whatever purpose.

    But it doesn't feel like the ECS way of handling the problem - since these objects (Tiles/Pieces) have a wealth of different components that are required by different systems, they would need to be Entity references, and systems would have to call some Get Component method on that entity reference that can be expected to fail in some cases, unless I want to have a dictionary for each tile/piece component.

    One "ECS" option I have explored, is that systems that require this information could instead declare and inject a second group - for example the Movement System could add a group of (Tile, Position, Terrain) as well as it's original (Piece, Position, Movement) group, but this just yields arrays of tile data, which still needs to be converted into Maps in order to avoid iterating over the entire array to find the tile with the correct position, and this has to be done frequently, because tile data changes. Which seems wasteful compared to the solution in OOP.

    Does anyone have any intuition as to how this type of shared data should exist in the ECS world? Perhaps the "wasteful" solution is in fact not as wasteful as it seems?
     
    IC_ likes this.
  2. deplinenoise

    deplinenoise

    Unity Technologies

    Joined:
    Dec 20, 2017
    Posts:
    23
    You can share a board by attaching the same native array to all systems that need it.
    • Define your board members as components
    • In a system, query for the board members and build a cache native array object
    • In other systems that need the information, inject the first system and query its native array. Make sure to mark the array for readonly access if you're scheduling jobs so you can get any concurrency.
     
    GarthSmith, ssellvf, Cynicat and 2 others like this.
  3. TickTakashi

    TickTakashi

    Joined:
    Mar 14, 2013
    Posts:
    30
    Thanks for the suggestion!

    I have a couple of questions:

    This is similar to what I am currently doing, except with a hashmap instead of an array. Is there a reason to use a native array here specifically? Usually i'm trying to get tile data based on it's 2D grid position, so a map made sense to me.

    The larger issue that i'm having is that: if the different systems require different data components from each board tile, the cached board members end up needing to be Entity references so that the systems can call GetComponent for what they need, which i'm trying to avoid. Or are you suggesting that this system create native structures for all possible necessary components that a board tile can have? e.g. map<int2, Terrain>, map<int2, Faction>, map<int2, Health>, map<int2, Foo> etc?
     
  4. TickTakashi

    TickTakashi

    Joined:
    Mar 14, 2013
    Posts:
    30
    Ok, so after fiddling around with injecting systems, I've come to a solution which feels pretty good. I'm essentially building maps of the various properties as mentioned in the previous post but doing so in the systems that are already handling/manipulating those properties, and injecting those systems into other systems as needed.

    Thanks a lot for pointing me in that direction!
     
    deplinenoise and 5argon like this.
  5. xenonsin

    xenonsin

    Joined:
    Dec 12, 2013
    Posts:
    20
    How does one inject a whole system to another?
     
  6. deplinenoise

    deplinenoise

    Unity Technologies

    Joined:
    Dec 20, 2017
    Posts:
    23
    You can
    [Inject] FooSystem m_FooSystem;
     
    Jes28, TickTakashi and starikcetin like this.
  7. IC_

    IC_

    Joined:
    Jan 27, 2016
    Posts:
    31
    How do you creating maps in IComponentDatas? I tried use NativeMultiHashMap and looks like it's not blittable
     
  8. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,073
    What if you flip it so the board is the system, as it is the main source of the data?
     
  9. James3545

    James3545

    Joined:
    Nov 21, 2016
    Posts:
    41
    "sometimes a system that handles the set of Pieces needs to know about arbitrary Tiles, and vice versa"

    One of the ways im trying to solve the same issue is by using NativeArraySharedValues. Its weird but it works lol. Rather than having a board piece have an x,y cord they have indices. (0,0)=> 1 (0,1) => 2 etc. Then I can use the
    GetSharedValueIndicesBySharedIndex method and it will get me the component index.

    int boardPieceIndex = sharedValuesNativeArray.GetSharedValueIndicesBySharedIndex(1) //pos (0,1)
    boardPiecesGroup.piece[boardPieceIndex]

    This way all you have to do is ensure you have indexed all of the board then just enter in the index of the board you want.

    I hope this makes sense lol Ive tried to solve this issue a number of dif ways and so far this might be the best, even if it might not be the "Right" way to do it. It might be better to gut some of the NativeArraySharedValues code as only parts are needed for this to work. I dont think this is the intended purpose for NativeArraySharedValues.


    Code (CSharp):
    1.  private const int primeOne = 27;
    2.     private const int primeTwo = 486187739;
    3.  
    4.     // Use this for initialization
    5.     void Start()
    6.     {
    7.         int indexToFind = 20; // get board index
    8.  
    9.         int mult = 32; // w x h of board
    10.         var intArray = new int[mult * mult];
    11.      
    12.         int count = 0;
    13.  
    14.         //indexing board
    15.         for (int i = 0; i < mult; i++)
    16.         {
    17.             for (int j = mult - 1; j >= 0; j--)
    18.             {
    19.                 intArray[count] = count;
    20.                 count++;
    21.             }
    22.         }
    23.  
    24.         //randomize order. ECS cant promise board entities are in the same order
    25.         intArray = intArray.OrderBy((i => Random.Range(0, 20))).ToArray();
    26.      
    27.         var source = new NativeArray<int>(intArray, Allocator.TempJob);
    28.  
    29.         var sharedValues = new NativeArraySharedValues<int>(source, Allocator.Temp);
    30.         var sharedValuesJobHandle = sharedValues.Schedule(default(JobHandle));
    31.  
    32.         //the board pieces i want will be put here
    33.         NativeArray<int> outputValues = new NativeArray<int>(10, Allocator.Temp);
    34.      
    35.         var jerb = new jerbTest()
    36.         {
    37.             sharedArray = sharedValues,
    38.             output = outputValues,
    39.             numberToGrab = indexToFind
    40.         };
    41.  
    42.         var jerbJH = jerb.Schedule(sharedValuesJobHandle);
    43.      
    44.      
    45.         jerbJH.Complete();
    46.  
    47.  
    48.         for (int i = 0; i < outputValues.Length; i++)
    49.         {
    50.             Debug.Log(intArray[outputValues[i]] + " should be " + indexToFind + "?");
    51.         }
    52.      
    53.         outputValues.Dispose();
    54.      
    55.         //---------------------------------
    56.         Debug.Log("GetSharedIndexArray");
    57.         string output = "";
    58.         var sharedIndexArray = sharedValues.GetSharedIndexArray();
    59.  
    60.         for (int i = 0; i < sharedIndexArray.Length; i++)
    61.         {
    62.             output += sharedIndexArray[i].ToString() + " ,";
    63.         }
    64.  
    65. //        Debug.Log(output);
    66.  
    67.         //---------------------------------
    68.         Debug.Log("GetSharedValueIndexCountArray");
    69.         var normalIndexes = sharedValues.GetSharedValueIndexCountArray();
    70.         output = string.Empty;
    71.  
    72.         for (int i = 0; i < normalIndexes.Length; i++)
    73.         {
    74.             output += normalIndexes[i].ToString() + " ,";
    75.         }
    76.  
    77. //        Debug.Log(output);
    78.  
    79.  
    80.         //---------------------------------
    81.         Debug.Log("GetSharedValueIndicesBySharedIndex");
    82.      
    83.         //there should be a better way to do this than using Shared values?!?!?!?
    84.         var sharedValueIndices = sharedValues.GetSharedValueIndicesBySharedIndex(indexToFind);
    85.              
    86.         output = string.Empty;
    87.  
    88.         for (int i = 0; i < sharedValueIndices.Length; i++)
    89.         {
    90.             if (i == 0)
    91.             {
    92.                Debug.Log(intArray[sharedValueIndices[i]] + " should be " + indexToFind + "?");
    93.             }
    94.          
    95.             output += sharedValueIndices[i].ToString() + ", ";
    96.         }
    97.      
    98. //        Debug.Log(output);
    99. //        Debug.Log(sharedValues.SharedValueCount);
    100.  
    101.         sharedValues.Dispose();
    102.         source.Dispose();
    103.     }
    104.  
    105.  
    106.     struct jerbTest : IJob
    107.     {
    108.         [ReadOnly]public NativeArraySharedValues<int> sharedArray;
    109.         public NativeArray<int> output;
    110.         public int numberToGrab;
    111.         public void Execute()
    112.         {          
    113.             for (int i = 0; i < output.Length; i++)
    114.             {
    115.                 //all will be the same index. Just copy the first one
    116.                 output[i] = sharedArray.GetSharedValueIndicesBySharedIndex(numberToGrab)[0];
    117.             }
    118.         }
    119.     }
    120.  
    121. //https://stackoverflow.com/questions/263400/what-is-the-best-algorithm-for-an-overridden-system-object-gethashcode/263416#263416
    122.     private int customInt2Hash(int2 int3)
    123.     {
    124.         int hash = primeOne;
    125.  
    126.         hash = hash * primeTwo + int3.x.GetHashCode();
    127.         hash = hash * primeTwo + int3.y.GetHashCode();
    128.  
    129.         return hash;
    130.     }
     
  10. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,073
    What about using the reactive pattern in ECS where a request to move from one position to another just checks that there is a piece at A and then calls for a check that B is empty/not blocked.

    Would this make for a much more atomic system where the status of the board is the data?

    Or would the ECS overhead of creating a query chain make it much slower?
     
  11. James3545

    James3545

    Joined:
    Nov 21, 2016
    Posts:
    41
    for me I have a board that has 1024 pieces. I would prefer not to iterate over all of them each time i want to move from A to B or say I need to check multiple positions. For my situation I've make the status of the board the data like you mention. Its just getting that data requires lots of work.
     
  12. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,073
    Honestly, 1024 is tiny for ECS, the demos show that 10s of thousands of entities can be processed in real time.

    I think the trouble with board based games in ECS would be when the size of the 'board' is larger than the CPU cache then you have the issue of chunking the board down to a size that fits or maybe making the system the board.
     
  13. James3545

    James3545

    Joined:
    Nov 21, 2016
    Posts:
    41
    sure 1024 entities isn't a lot unless you have to go over them multiple times. If you only go over them once its no big deal. The boid example does what? 100k entities. But it only itr once basically. In my project if I itr over 1024 just to find a few board positions in multiple systems that adds up.

    I did try mapping the positions and entities in one system so others can use them for their own purposes but I ended up running into an issue with the dependency system in ECS.
     
  14. PsychoStuey

    PsychoStuey

    Joined:
    Mar 4, 2015
    Posts:
    8
    I know this is fairly old by now, but I'm just confused with something. What happens when you have multiple boards? How do you make sure the correct board is injected into the second system?
     
  15. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    1,876
    Do not use Inject, as it's being "out of design" nowadays.
    Get your system in which you've declared your native array via World.Active.GetOrCreateExistingSystem<T>() and fetch array from it directly, e.g. via property.
     
    MostHated likes this.