Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Batch EntityCommandBuffer

Discussion in 'Data Oriented Technology Stack' started by tertle, Dec 6, 2018.

  1. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    So I read @5argon blog post over here this morning https://gametorrahod.com/unity-ecs-batched-operation-on-entitymanager-3c35e8e5ecf4

    And as I use quite a few short lived event entities in my project I was curious if I could write something that could generically replaced EntityCommandBuffer for these. So I did!

    Firstly code, this is my first, not well tested and quickly written proof of concept.

    -edit- dramatically updating code, will post it when i'm done.
    -edit2- latested post here, now 3x faster than original version (10x faster than a barrier): https://forum.unity.com/threads/batch-entitycommandbuffer.593569/#post-3965998

    How it works. It passes a NativeQueue<T> whenever requested.

    Get the barrier with
    this.batchBarrier = this.World.GetOrCreateManager<BatchBarrierSystem>();
    The same barrier is used for all systems.

    To create an entity use

    NativeQueue<T>.Concurrent createQueue = this.batchBarrier.GetCreateQueue<T>().ToConcurrent();


    where T is an IComponentData that will be added and set on the entity.

    Then just pass it to a job, for example

    Code (CSharp):
    1.         [BurstCompile]
    2.         private struct CreateEntitiesJob : IJobParallelFor
    3.         {
    4.             public NativeQueue<TestComponent>.Concurrent AddComponentQueue;
    5.  
    6.             /// <inheritdoc />
    7.             public void Execute(int index)
    8.             {
    9.                 this.AddComponentQueue.Enqueue(new TestComponent { Index = index });
    10.             }
    11.         }
    To destroy an entity, use the

    NativeQueue<Entity>.Concurrent = this.batchBarrier.GetDestroyQueue().ToConcurrent(),


    Any entity added to this queue will be destroyed.

    Code (CSharp):
    1. [BurstCompile]
    2.         private struct DestroyEntitiesJob : IJobProcessComponentDataWithEntity<TestComponent>
    3.         {
    4.             public NativeQueue<Entity>.Concurrent DestroyEntityQueue;
    5.  
    6.             /// <inheritdoc />
    7.             public void Execute(Entity entity, int index, ref TestComponent data)
    8.             {
    9.                 this.DestroyEntityQueue.Enqueue(entity);
    10.             }
    11.         }
    The one extra bit you need to do atm is pass the dependency to the system because I don't have any access to AfterUpdate();

    Code (CSharp):
    1. this.batchBarrier.AddDependency(handle);
    2.  
    3. return handle;
    Why bother?

    At the moment it is about 3x faster than using EntityCommandBuffers and completely garbage free.
    Also it works in Burst jobs

    I believe I could probably optimize it quite a bit if I limit the use.

    100,000 entities being added and removed each frame.
    This is done in a build (with profiler attached obviously)
    I'm not sure why endframebarrier is throwing garbage.

    upload_2018-12-6_11-56-26.png

     
    Last edited: Dec 6, 2018
  2. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    upload_2018-12-6_12-19-13.png

    upload_2018-12-6_12-19-55.png

    upload_2018-12-6_12-20-11.png

    Significant performance increase if I force these entities to only live 1 frame (also saves you having to clean them up myself). Has become more like a dedicated event system, which I kind of like.
     
    hippocoder likes this.
  3. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Moving the SetComponent to a job significantly improves performance (renamed to EventSystem though that kind of clashes with the unity ui system so i'll probably change that)

    upload_2018-12-6_14-59-8.png

    Thats 100k entities created in 3.4ms on a pretty old 3570k, vs 34ms for a barrier.

    I can only think of 1 more optimization that needs doing now, and that's making different types set in parallel (won't affect this benchmark results as they are only a single type test)
     
    Last edited: Dec 6, 2018
    wobes likes this.
  4. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Entities will live 1 frame exactly. Destroying entity is now done automatically.

    -edit 2018/12/28-

    did some unit testing, found some issues and fixed these. also added parallel setting support, should be quite a bit faster.

    Source code is now here: https://github.com/tertle/EventBatchSystem

    -edit 2019/01/09- Read below, repo moved.
     
    Last edited: Jan 9, 2019
    pahe, hippocoder and davenirline like this.
  5. Jay-Pavlina

    Jay-Pavlina

    Joined:
    Feb 19, 2012
    Posts:
    195
    @tertle Can you briefly explain how this EventSystem works? I am converting a game from Rx-driven events to ECS and have the most trouble with events.
     
  6. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Sure. So I have a few short lived entities with a single component attached that I use as kind of an event.

    For example to request a path in my project, you create a new entity with a FindPath component. This component will be handled by the pathfinding system (though other events might be handled by multiple systems) and then destroyed.

    The point of this, I think I called it BatchSystem for the moment as it shared the same name as the unity UI class though it really needs a better name, is to batch create all entity events at once (of the same type, though as I typed this I just realized I could maybe merge the creation of all of event entities at once) instead of creating them individually 1 at a time.

    So instead of using EntityCommandBuffer and creating the entity yourself, you BatchSystem.GetEventBatch<T>() where T where the event component. This will return a NativeQueue<T> which every time the method is called so you don't need to worry about other system dependencies and will be automatically merged. You simply add the event struct to this queue instead and when BatchSystem updates, it'll create a bunch of entities and set the component data.

    All these newly created entities will be stored in the BatchSystem and destroyed 1 frame later in a single batch destroy entity call. You don't need to worry about maintaining the life of the event entities it's handled for you. (Whichcan be very useful if your event entities are used by more than 1 system, you don't have to worry about ordering.)

    NativeQueue has a ToConcurrent() method so can be used in parallel but can also be burst compiled unlike entity command buffer.

    -edit-

    side note, if anyone was using this before today I updated the code with a small bug fix a few hours ago.
     
  7. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Amazing work, thank you
     
  8. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Can we have permission to use your BatchSystem and NativeUnit code?
    I'm asking because I've noticed the all rights reserved copyright.
     
  9. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Oh yeah of course. They are just auto generated by StyleCop and I'm too lazy to strip it when copy/pasting.

    ~feel free to use any code I post on unity forums, now or in the future.

    -edit-

    side note, NativeUnit is really just a NativeArray with a fixed length of 1. More of a convenience.
     
    Last edited: Dec 19, 2018
    iam2bam likes this.
  10. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Cool, thanks!

    I thought of it as a "NativeReference" :). Even if it's just like a 1-array, it makes code clearer and I'm all about that!

    I just hope someone figures out a way to return a reference, to access direct memory to get rid of the
    o = arr[0]; o.var = 123; arr[0] = o
    idiom for structs.
     
    Arkade likes this.
  11. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    finally got around to doing some unit testing and found some issues (and fixed them).
    Also added parallel setting support, should be quite a bit faster. -note- while this is passing all tests, i probably need to do some large scale validation to ensure thread safety.

    Source code is now here: https://github.com/tertle/com.bovinelabs.entities
     
    Last edited: Dec 30, 2018
    Antypodish and iam2bam like this.
  12. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,265
    A little recommendation about repo's "shape" to support UPM git pulling, if you move the git init location one level deeper inside the Assets folder then put package.json, then anyone can pull that from Unity. It would appears in Project window like other Unity packages. (Wrapped one level in a folder named after the name specified in package.json) dependencies will go in package.json instead of manifest.json. .gitignore will also has to be Assets in this case.

    I have 1 Unity project named PackageDev where in the Assets I have multiple folders representing each small package with its own repo. I set a local UPM on each folder so that the change applies to all games, and git would be able to take them online.
     
    Arkade and tertle like this.
  13. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Actually I forgot about this feature. I looked at it when it first came out but ran into an issue and totally forgot.

    I've moved my code to here: https://github.com/tertle/com.bovinelabs.entities if anyone wants.
     
    Arkade and wobes like this.
  14. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    677
    @tertle could you add a small ReadMe that describes how to get started if it's possible?
     
  15. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Yeah sure, I'm about to head out for NYE so I'll do it soon as I get back in the new year.
     
    wobes likes this.
  16. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    677
    Happy holidays to you.
     
    Antypodish likes this.
  17. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    wobes likes this.
  18. Abbrew

    Abbrew

    Joined:
    Jan 1, 2018
    Posts:
    164
    Awesome work! Is there a way to add multiple IComponentData to an entity? Also, is it possible to add an ISharedComponentData?
     
  19. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,676
    Looks like cool work has been done here.
    But got question, since I maybe missed, or misunderstood.
    You have mentioned short leaving entities. But is this approach good for creating persistent entities? Sorry, haven't chance yet, to test the code.
     
  20. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Neither of these are that hard to add but I don't really see the point of them for this particular sytem.

    Yes, if you're creating a lot of (persistent) entities in a frame you should batch create them, much faster (this system won't do that for you though.)
     
    Last edited: Jan 8, 2019
    Antypodish likes this.
  21. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Found kind of an elegant solution my dependency issues that someone might be interested in.

    So previously to handle dependencies, I was grabbing the internal JobHandle m_PreviousFrameDependency with reflection but unfortunately as JobHandle is a struct, and without reflection and GetValue is designed structs are always boxed it was generating garbage each frame.

    I've gone in and redesigned how I handle dependencies now. Instead, I pass a fake BarrierSystem to the JobComponentSystem (handled automatically with reflection once per life of JobComponentSystem) and then manually update this system.

    It works remarkably well. The barrier doesn't have any command buffers, but just completes all dependencies. This system is lazy created to avoid being added to the default loop, but if you happen to use this and update your loop mid way through for some reason, this will break.

    TLDR: nothing has changed in how you use it, just no more garbage.
     
  22. kork

    kork

    Joined:
    Jul 14, 2009
    Posts:
    265
    So I read all of this code and like it very much. I have one question regarding this section:
    Code (CSharp):
    1.   private class EventBatch<T> : EventBatchBase
    2.             where T : struct, IComponentData
    3.         {
    4.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
    5.  
    6. // .. SNIP ...
    7.  
    8.             public NativeQueue<T> GetNew()
    9.             {
    10.                 // Having allocation leak warnings when using TempJob
    11.                 var queue = new NativeQueue<T>(Allocator.Persistent);
    12.                 this.queues.Add(queue);
    13.  
    14.                 return queue;
    15.             }
    16. }
    Why is it necessary to create multiple queues per event type?
    NativeQueue 
    has a
    ToConcurrent
    method which would make the Queue safe to use from multiple threads and we actually don't really care about the order of things as all entities get created in one fell swoop. So just from looking at it I would say you could do with a single queue (which would simplify some code later where it maps the single queues to segments in the chunk). Then again, I have little experience with this, so I would really like to know why you chose to design it that way. Thanks for sharing this!
     
  23. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Because this

    Code (CSharp):
    1.         var handle1 = new Job1
    2.             {
    3.                 Entities = this.queue.ToConcurrent()
    4.             }
    5.             .Schedule(this, inputDeps);
    6.  
    7.         var handle2 = new Job2
    8.             {
    9.                 Entities = this.queue.ToConcurrent()
    10.             }
    11.             .Schedule(this, inputDeps);
    12.  
    13.         var handle = JobHandle.CombineDependencies(handle1, handle2);
    Will throw this specific error

    Note job1 dependency isn't passed to job2 so these jobs run in parallel, not sequential (which is fine.)

    NativeContainers.Concurrent are only safety checked within the same job, not across multiple jobs so by creating a separate container for each system I avoid these dependency issues.
     
    Last edited: Jan 25, 2019
    kork likes this.
  24. kork

    kork

    Joined:
    Jul 14, 2009
    Posts:
    265
    So is this really a problem or is it just a false positive of the safety system? My thinking is that if a
    NativeContainer.Concurrent
    is safe to use by multiple threads within one job, in theory it should also be safe to use across jobs, after all in both cases you have multiple threads concurrently accessing the container.

    So if I would add some
    NativeDisableContainerSafetyRestriction 
    attribute to
    Entities 
    and therefore circumvent the suspected false positive, would it work? I'm not saying that this is a particulary good idea, I just would like to know if I understood the thing correctly :D
     
  25. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    483
    1) different jobs scheduling ordering is non-deterministic without dependencies, even on the same thread. (this is also true for different "parts" of the same parallel job, but a parallel job should be designed to do the same thing to all of its parts, so it shouldn't matter
    2) concurrent containers may work by using an internal [ThreadIndex] and that may be set uniquely only in a single .Schedule call (e.g. different scheduled jobs may have conflicting threadIndex in different threads, we can't know unless @Joachim_Ante tells us otherwise)
     
  26. tbriley

    tbriley

    Joined:
    Sep 10, 2013
    Posts:
    111
    is it possible to use this with UPM? am getting a package dependency error when adding "com.bovinelabs.entities": "0.1.2" to my manifest (2019.1.0b5). am a bit new with how custom packages work.
     
  27. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Been busy with something so haven't properly updated my libraries. I have updated this to the latest version of entities but I haven't updated the dependencies of the package yet. I'll l do that tomorrow thanks for letting me know.
     
  28. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,265
    This is how to UPM to GitHub
     "com.bovinelabs.entities": "git://github.com/tertle/com.bovinelabs.entities.git",
    to update delete the new lock section appearing in your manifest after the first pull.
     
    Anichale, kork and tbriley like this.
  29. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    851
    I tried to actually add this with package manager today but keep getting:

    Code (CSharp):
    1. An error occurred while resolving packages:
    2.   Package com.bovinelabs.entities@git://github.com/tertle/com.bovinelabs.entities.git has invalid dependencies:
    3.     com.bovinelabs.naughtyattributes: Version 'https://github.com/tertle/com.bovinelabs.naughtyattributes.git' is invalid. Expected a 'SemVer' compatible value.
    I tried removing the locks, removing it, restarting Unity, adding again, etc, but it seems to keep ending up with that. I tried to see if just adding your naughtyattributes repo as well might help, but it didn't seem to make a difference.
     
  30. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    oh i think i just broke this today by accident as i was removing old repos from harddrive and pushed a few random things just to make sure i wasn't deleting anything.

    i have reorganized my packages internally a lot so this is a separate standalone repo for me (which is private atm)

    if you just need the code, this is my latest version. i'll see if i can make this public so you can pull from git, has a few unit tests.

    Code (CSharp):
    1. // <copyright file="EntityEventSystem.cs" company="BovineLabs">
    2. //     Copyright (c) BovineLabs. All rights reserved.
    3. // </copyright>
    4.  
    5. namespace BovineLabs.Event
    6. {
    7.     using System;
    8.     using System.Collections.Generic;
    9.     using BovineLabs.Common.Jobs;
    10.     using JetBrains.Annotations;
    11.     using Unity.Burst;
    12.     using Unity.Collections;
    13.     using Unity.Collections.LowLevel.Unsafe;
    14.     using Unity.Entities;
    15.     using Unity.Jobs;
    16.     using UnityEngine.Profiling;
    17.  
    18.     /// <summary>
    19.     /// The BatchBarrierSystem.
    20.     /// </summary>
    21.     [UsedImplicitly]
    22.     public abstract class EntityEventSystem : ComponentSystem
    23.     {
    24.         private readonly Dictionary<Type, IEventBatch> types = new Dictionary<Type, IEventBatch>();
    25.  
    26.         private JobHandle producerHandle;
    27.  
    28.         /// <summary>
    29.         /// The interface for the batch systems.
    30.         /// </summary>
    31.         private interface IEventBatch : IDisposable
    32.         {
    33.             /// <summary>
    34.             /// Updates the batch, destroy, create, set.
    35.             /// </summary>
    36.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
    37.             /// <returns>A <see cref="JobHandle"/>.</returns>
    38.             JobHandle Update(EntityManager entityManager);
    39.  
    40.             /// <summary>
    41.             /// Resets the batch for the next frame.
    42.             /// </summary>
    43.             void Reset();
    44.         }
    45.  
    46.         /// <summary>
    47.         /// Creates a queue where any added component added will be batch created as an entity event
    48.         /// and automatically destroyed 1 frame later.
    49.         /// </summary>
    50.         /// <typeparam name="T">The type of <see cref="IComponentData"/>.</typeparam>
    51.         /// <returns>A <see cref="NativeQueue{T}"/> which any component that is added will be turned into a single frame event.</returns>
    52.         public NativeQueue<T> CreateEventQueue<T>()
    53.             where T : struct, IComponentData
    54.         {
    55.             if (!this.types.TryGetValue(typeof(T), out var create))
    56.             {
    57.                 create = this.types[typeof(T)] = new EventBatch<T>(this.EntityManager);
    58.             }
    59.  
    60.             return ((EventBatch<T>)create).GetNew();
    61.         }
    62.  
    63.         /// <summary>
    64.         /// Add a dependency handle.
    65.         /// </summary>
    66.         /// <param name="handle">The dependency handle.</param>
    67.         public void AddJobHandleForProducer(JobHandle handle)
    68.         {
    69.             this.producerHandle = JobHandle.CombineDependencies(this.producerHandle, handle);
    70.         }
    71.  
    72.         /// <inheritdoc />
    73.         protected override void OnDestroy()
    74.         {
    75.             foreach (var t in this.types)
    76.             {
    77.                 t.Value.Dispose();
    78.             }
    79.  
    80.             this.types.Clear();
    81.         }
    82.  
    83.         /// <inheritdoc />
    84.         protected override void OnUpdate()
    85.         {
    86.             this.producerHandle.Complete();
    87.             this.producerHandle = default;
    88.  
    89.             var handles = new NativeArray<JobHandle>(this.types.Count, Allocator.TempJob);
    90.  
    91.             int index = 0;
    92.  
    93.             foreach (var t in this.types)
    94.             {
    95.                 handles[index++] = t.Value.Update(this.EntityManager);
    96.             }
    97.  
    98.             JobHandle.CompleteAll(handles);
    99.             handles.Dispose();
    100.  
    101.             foreach (var t in this.types)
    102.             {
    103.                 t.Value.Reset();
    104.             }
    105.         }
    106.  
    107.         private class EventBatch<T> : IEventBatch
    108.             where T : struct, IComponentData
    109.         {
    110.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
    111.             private readonly EntityQuery query;
    112.  
    113.             private readonly EntityArchetype archetype;
    114.  
    115.             public EventBatch(EntityManager entityManager)
    116.             {
    117.                 this.query = entityManager.CreateEntityQuery(ComponentType.ReadWrite<T>());
    118.                 this.archetype = entityManager.CreateArchetype(typeof(T));
    119.             }
    120.  
    121.             public NativeQueue<T> GetNew()
    122.             {
    123.                 // Having allocation leak warnings when using TempJob
    124.                 var queue = new NativeQueue<T>(Allocator.TempJob);
    125.                 this.queues.Add(queue);
    126.                 return queue;
    127.             }
    128.  
    129.             public void Reset()
    130.             {
    131.                 foreach (var queue in this.queues)
    132.                 {
    133.                     queue.Dispose();
    134.                 }
    135.  
    136.                 this.queues.Clear();
    137.             }
    138.  
    139.             /// <inheritdoc />
    140.             public void Dispose()
    141.             {
    142.                 this.Reset();
    143.             }
    144.  
    145.             /// <summary>
    146.             /// Handles the destroying of entities.
    147.             /// </summary>
    148.             /// <param name="entityManager">The entity manager.</param>
    149.             /// <returns>A default handle.</returns>
    150.             public JobHandle Update(EntityManager entityManager)
    151.             {
    152.                 this.DestroyEntities(entityManager);
    153.  
    154.                 if (!this.CreateEntities(entityManager))
    155.                 {
    156.                     return default;
    157.                 }
    158.  
    159.                 return this.SetComponentData(entityManager);
    160.             }
    161.  
    162.             private void DestroyEntities(EntityManager entityManager)
    163.             {
    164.                 Profiler.BeginSample("DestroyEntity");
    165.  
    166.                 entityManager.DestroyEntity(this.query);
    167.  
    168.                 Profiler.EndSample();
    169.             }
    170.  
    171.             private bool CreateEntities(EntityManager entityManager)
    172.             {
    173.                 var count = this.GetCount();
    174.  
    175.                 if (count == 0)
    176.                 {
    177.                     return false;
    178.                 }
    179.  
    180.                 Profiler.BeginSample("CreateEntity");
    181.  
    182.                 // Felt like Temp should be the allocator but gets disposed for some reason.
    183.                 using (var entities = new NativeArray<Entity>(count, Allocator.TempJob, NativeArrayOptions.UninitializedMemory))
    184.                 {
    185.                     entityManager.CreateEntity(this.archetype, entities);
    186.                 }
    187.  
    188.                 Profiler.EndSample();
    189.                 return true;
    190.             }
    191.  
    192.             private JobHandle SetComponentData(EntityManager entityManager)
    193.             {
    194.                 var isZeroSized = TypeManager.GetTypeInfo<T>().IsZeroSized;
    195.  
    196.                 if (isZeroSized)
    197.                 {
    198.                     return default;
    199.                 }
    200.  
    201.                 var componentType = entityManager.GetArchetypeChunkComponentType<T>(false);
    202.  
    203.                 var chunks = this.query.CreateArchetypeChunkArray(Allocator.TempJob);
    204.  
    205.                 int startIndex = 0;
    206.  
    207.                 var handles = new NativeArray<JobHandle>(this.queues.Count, Allocator.TempJob);
    208.  
    209.                 // Create a job for each queue. This is designed so that these jobs can run simultaneously.
    210.                 for (var index = 0; index < this.queues.Count; index++)
    211.                 {
    212.                     var queue = this.queues[index];
    213.                     var job = new SetComponentDataJob
    214.                     {
    215.                         Chunks = chunks,
    216.                         Queue = queue,
    217.                         StartIndex = startIndex,
    218.                         ComponentType = componentType,
    219.                     };
    220.  
    221.                     startIndex += queue.Count;
    222.  
    223.                     handles[index] = job.Schedule();
    224.                 }
    225.  
    226.                 var handle = JobHandle.CombineDependencies(handles);
    227.                 handles.Dispose();
    228.  
    229.                 // Deallocate the chunk array
    230.                 handle = new DeallocateJob<ArchetypeChunk>(chunks).Schedule(handle);
    231.  
    232.                 return handle;
    233.             }
    234.  
    235.             private int GetCount()
    236.             {
    237.                 var sum = 0;
    238.                 foreach (var i in this.queues)
    239.                 {
    240.                     sum += i.Count;
    241.                 }
    242.  
    243.                 return sum;
    244.             }
    245.  
    246.             [BurstCompile]
    247.             private struct SetComponentDataJob : IJob
    248.             {
    249.                 public int StartIndex;
    250.  
    251.                 public NativeQueue<T> Queue;
    252.  
    253.                 [ReadOnly]
    254.                 public NativeArray<ArchetypeChunk> Chunks;
    255.  
    256.                 [NativeDisableContainerSafetyRestriction]
    257.                 public ArchetypeChunkComponentType<T> ComponentType;
    258.  
    259.                 /// <inheritdoc />
    260.                 public void Execute()
    261.                 {
    262.                     this.GetIndexes(out var chunkIndex, out var entityIndex);
    263.  
    264.                     for (; chunkIndex < this.Chunks.Length; chunkIndex++)
    265.                     {
    266.                         var chunk = this.Chunks[chunkIndex];
    267.  
    268.                         var components = chunk.GetNativeArray(this.ComponentType);
    269.  
    270.                         while (this.Queue.TryDequeue(out var item) && entityIndex < components.Length)
    271.                         {
    272.                             components[entityIndex++] = item;
    273.                         }
    274.  
    275.                         if (this.Queue.Count == 0)
    276.                         {
    277.                             return;
    278.                         }
    279.  
    280.                         entityIndex = entityIndex < components.Length ? entityIndex : 0;
    281.                     }
    282.                 }
    283.  
    284.                 private void GetIndexes(out int chunkIndex, out int entityIndex)
    285.                 {
    286.                     var sum = 0;
    287.  
    288.                     for (chunkIndex = 0; chunkIndex < this.Chunks.Length; chunkIndex++)
    289.                     {
    290.                         var chunk = this.Chunks[chunkIndex];
    291.  
    292.                         var length = chunk.Count;
    293.  
    294.                         if (sum + length < this.StartIndex)
    295.                         {
    296.                             sum += length;
    297.                             continue;
    298.                         }
    299.  
    300.                         entityIndex = this.StartIndex - sum;
    301.                         return;
    302.                     }
    303.  
    304.                     throw new ArgumentOutOfRangeException(nameof(this.StartIndex));
    305.                 }
    306.             }
    307.         }
    308.     }
    309. }
    Code (CSharp):
    1. // <copyright file="EndFrameEntityEventSystems.cs" company="BovineLabs">
    2. // Copyright (c) BovineLabs. All rights reserved.
    3. // </copyright>
    4.  
    5. namespace BovineLabs.Event
    6. {
    7.     using Unity.Entities;
    8.  
    9.     /// <summary>
    10.     /// <see cref="EntityEventSystem"/> that runs in the <see cref="LateSimulationSystemGroup"/>.
    11.     /// </summary>
    12.     [UpdateInGroup(typeof(LateSimulationSystemGroup))]
    13.     public sealed class EndSimulationEntityEventSystem : EntityEventSystem
    14.     {
    15.     }
    16.  
    17.     /// <summary>
    18.     /// <see cref="EntityEventSystem"/> that runs in the <see cref="PresentationSystemGroup"/>.
    19.     /// </summary>
    20.     [UpdateInGroup(typeof(PresentationSystemGroup))]
    21.     public sealed class PresentationEntityEventSystem : EntityEventSystem
    22.     {
    23.     }
    24. }
     
  31. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    851
    I am trying to figure out exactly how to use this, is it strictly for creating entities with a component as well as destroying them, or am I able to also add/remove from already existing entities as well?

    I was following your advice from another thread and isnteadof removing a component tag in a job with command buffer in the main job, I made a queue and added the entities to that and then that dequeues and runs after the main job so that the main one can be burstable.

    When I saw this and that it produced no garbage, I thought that would be great as my overall GC is killing me, so I am trying to optimize wherever possible and this seemed like a good place to start if it could remove components.
     
    Last edited: Sep 4, 2019
  32. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    This specific system is an event system. It lets you create events that last 1 frame (handles cleaning up for you) and does this optimally by grouping the same events that could have been created from multiple different systems into a single batch call (which is much faster than using an EntityCommandBuffer in each system.)

    It's also somewhat designed to stop the need for short lived component tags that are often used. Instead of adding a Damage tag (or Killed tag etc) to an entity, you could create a Damage event with a reference to the entity and query that instead.

    This way you can keep your entity archetype quite static and it does not have any costly moving operations. Combining this with simple polling (and chunk has changed etc) you can in most circumstances never modify an entities archetype. It can also help by having to worry less about system order.

    If you just want to efficiently remove a bunch of components from entities, you should consider doing by query if possible.

    EntityManager.RemoveComponent<T>(EntityQuery)
     
  33. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    851
    I understand now. Sounds like a great system! I was using EntityQuery for the addition and removal of the entities, but that is just as of recently, I had setup the queue to remove the tag before that I think, so... perhaps I didn't even think to try doing it by query since I saw your way and it seemed a lot better than my original way with ECB in the main job. I will give that a try and see how it goes.

    Side note, I followed your example yet again from another-another thread, lol. I was doing ijobforeachwithentity but I have an insane amount of allocations and job/tempalloc overflow (for some reason it happens no matter what size I try to make the size of the container : / ), so I did as you mentioned and instead of IJFEWE I moved to IJobCunk so that I had less overall allocations because it was no longer 'per entity'. Do you have any recommendations on how I might be able to take it even further?

    Granted, my waypoint system and navigation is fully working, and works well, but when I drop in 10k vehicles, it's not nearly as good as I would like it to be, so really I am just trying to learn some tips and tricks for optimization because my entire system was thrown together with the level of planning being "oh nice, that worked!", simply because I didn't know any other way to accomplish what I was doing. Hence my finding and trying out a lot of your suggestions, as so far each one has greatly improved my system overall, I just want to try an optimize it as far as I possibly can to learn.

    That being said, my biggest offenders are these guys. I just exposted allocator size to the inspector because I got tired of recompiling.

    Code (CSharp):
    1.                
    2.                 var frontTier = new NativeList<Node>(allocatorSize, Allocator.Temp);
    3.                 var path = new NativeList<WaypointECS>(allocatorSize, Allocator.Temp);
    4.                 var connTemp = new NativeList<WaypointConnections>(allocatorSize, Allocator.Temp);
    5.                 var costSoFar = new NativeHashMap<WaypointECS, float>(allocatorSize, Allocator.Temp);
    6.                 var cameFrom = new NativeHashMap<WaypointECS, WaypointECS>(allocatorSize, Allocator.Temp);
    It's a pretty standard A* system, and as I mentioned I was able to move those outside of the main iteration of the chunk so it is not getting hit each and every time which definitely did help a lot when I have anywhere from 10-50k vehicles pathing.

    Do you have any recommendation on another way to go about this perhaps? I tried to see if maybe I could allocate all of these things outside the job itself and then pass it in all empty and then just clear it on each iteration, but I kept running into issues with concurrent writing and needing to mark them as ReadOnly, etc. Does any of that change if I am doing Chunk Iter now? I don't think it might considering it still parallelizes the chunks. I was wondering if there was a way maybe to more or less fake using a NHM by using something else that could be passed in blank and cleared. Could the EventQueue somehow play a role in this? I guess it just depends on how fast everything ends up completing. At first I was splitting the work over a few frames because it was a hefty beast of a job. It was Oritinally something like 1800ms, but I got it down to this below, then a bit more after some more tweaking and moving around some iterations inside the job. Now it is probably 300ms total or so when I don't split the frames a bit.

    Anyways, my apologies, I didn't expect this post to get as long as it did. Once I started typing I just kind of kept on going. I probably should have just made a new thread, lol.


     
  34. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Hey, sorry about slow reply i've been ill.

    IJobChunk is exactly how I would have suggested doing it. There's no real way to reduce allocations more than caching them 1 per thread as you're doing except to do some seriously hackory (like I did with my original pathfinding solution before you could allocate in jobs) which I would not suggest.

    I'm assuming by what you previously said, they are only allocated once per chunk (outside of the loop) and are cleared. Unless you have a very fragmented entities you really shouldn't have that many chunks therefore very few allocations so I'm not sure why it'd be /that/ much of an issue.

    If the clear is causing issues because the hashmaps grow very large and you are using 2019.3 you could use the solution i posted over here: https://forum.unity.com/threads/nativehashmap-clear-4x-performance-improvement-request.732782/ as it's about 4x faster (let me know and I'll find my code.)

    You'd have to switch to IJob and allocate 1 container per job and manually schedule them. I would not recommend (how I create my original pathfinding though before in job allocations existed.)

    Anyway, pathfinding is expensive as the world grid grows. Doing 10k requests/frame is pretty unheard of. How large is your grid?

    In the past I've done a few optimizations for pathfinding. When I was using a huge 4096x4096 grid I used to limit paths to 100 steps. If the target was further away than this it'd return the best expected 100 steps and follow this and would simply do another request for a path when it got near to the end. Works great as long as you don't have any long dead ends (>100 steps).

    Also you could simply look at limiting the number of requests/frame to smooth out the frame rate
     
    MostHated likes this.
  35. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    851
    I appreciate the reply. I wanted to try and use 2019.3 but I have an incident opened with Unity as my project is using LWRP (or well, Universal RP on 2019.3), when I updated, this happened : https://i.imgur.com/m0m9i6Y.gifv any time I would try to spawn my entities.

    Also, I am not using a grid setup. Each waypoint has a list of the next hop waypoints it is directly connected to https://i.imgur.com/OpSD58s.png. This was to be able to easily create paths in a single direction so I could have 2-way streets. Each waypoint in the pic has a color between it and the next hop, if one waypoint has two colors coming off of it, that is because it is connected to those two waypoints for turns and what not. I am colorblind, so not sure why I even bothered, I can't really tell with most of them if it's accurate, lol. The red waypoints are simply "connecting" waypoints to feed the path, the blue ones are actual possible destinations randomly spread out.

    As we speak I am working on a way to try and remove the tags I was using to designate when it's time for each entity to obtain a new waypoint. When it reached it's goal I added a VehicleNeedsPathTag and the path system just had a RequireComponentTag of that and then it would remove it when it was finished. I was going to see about adding a new IComponentData to each one with just an int on it of whether it needs a path or not. Set it to 1 when it does, 0 otherwise.

    I don't know much about ChangeFilters yet, but you mentioned tagging moving them around so I wanted to try something else so I was about to start researching how those work and see if it might help. It could very well be that there is fragmentation and I just am not sure exactly what I should be looking for to fix it. This is my first system and as such I been just learning things as I go along trying to redo each bottleneck I come across.
     
    Last edited: Sep 5, 2019
  36. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    851
    Nice, removing the use of tags and in turn removing the ECB, it seems the ever growing allocations gone. For whatever reason it would end up accumulating a MB or two every few seconds, but I just let it run for several minutes and it held steady. After a few more tweaks it looks like I got the pathfind situated as well. Looking at the Entity debugger I don't see tons of changes all over the place like I did before. I appreciate the advice.

    The only thing I can't seem to figure out how to get rid of is is the tons of TempAlloc.Overflow / JobAlloc.Grow. It seems no matter how I set the size of the allocations I always end up having them.
     
    Last edited: Sep 6, 2019
  37. joseph-t83

    joseph-t83

    Joined:
    Mar 28, 2014
    Posts:
    22
    Hey @tertle, I've been messing with this and it's working awesome except i'm having an issue where sending an event queue to a job invalidates any other nativecontainers sent to the job. Have you had any issues with that?

    Here's some example code that causes the issue. Am I doing something wrong?

    Code (CSharp):
    1.         private EndSimulationEntityEventSystem m_EventSystem;
    2.  
    3.         protected override void OnCreate()
    4.         {
    5.             m_EventSystem = World.GetOrCreateSystem<EndSimulationEntityEventSystem>();
    6.         }
    7.         public struct ReactJob : IJobForEach<HitPosition>
    8.         {
    9.             [ReadOnly]
    10.             public ComponentDataFromEntity<LocalToWorld> localToWorldFromEntity;
    11.             [ReadOnly]
    12.             [DeallocateOnJobCompletion]
    13.             public NativeArray<Entity> unitEntities;
    14.             public NativeQueue<ReactToHit>.ParallelWriter eventQueue;
    15.             public float spread;
    16.  
    17.             public void Execute([ReadOnly] ref HitPosition hit)
    18.             {
    19.                 for (int i = 0; i < unitEntities.Length; i++)
    20.                 {
    21.                     if (localToWorldFromEntity.Exists(unitEntities[i]))
    22.                     {
    23.                         if (distance(hit.Position, localToWorldFromEntity[unitEntities[i]].Position) < spread)
    24.                         {
    25.                             eventQueue.Enqueue(new ReactToHit()
    26.                             {
    27.                                 Entity = unitEntities[i]
    28.                             });
    29.                         }
    30.                     }
    31.                 }
    32.             }
    33.         }
    34.  
    35.         protected override JobHandle OnUpdate(JobHandle inputDept)
    36.         {
    37.  
    38.             var job = new ReactJob()
    39.             {
    40.                 localToWorldFromEntity = GetComponentDataFromEntity<LocalToWorld>(true),
    41.                 unitEntities = units,
    42.                 eventQueue = m_EventSystem.CreateEventQueue<ReactToHit>().AsParallelWriter(),
    43.                 spread = 2.5f,
    44.             };
    45.             var handle = job.Schedule(this, inputDept);
    46.             m_EventSystem.AddJobHandleForProducer(handle);
    47.             return handle;
    48.         }
     
  38. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    That's a bit strange.

    Also what is units?
     
  39. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,492
    CreateArchetype synch point isn’t it? (From mobile, can’t check) In CreateEvenQueue it creates EventBatch which in turn creates EntityQuery and Archetype. If yes in his case just move creating at top of OnUpdate and use it later. But anyway seems it’s not full his code...
     
    Last edited: Sep 29, 2019
  40. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Probably correct but I'm surprised a new archetype invalidates arrays (haven't tested). If so I'll have to reorganize it a little because I don't think calls to this should be order dependent.
     
  41. joseph-t83

    joseph-t83

    Joined:
    Mar 28, 2014
    Posts:
    22
    eizenhorn was correct!!! That does indeed solve the issue. So I guess CreateArchetype is a sync point.

    Sorry, I should have made it more clear. The code example I posted was just simplified example of how I'm using the event stuff. It's GetComponentDataFromEntity, GetArchetypeChunkComponentType, and ToComponentDataArray that get invalidated.
     
  42. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,492
    Finally on PC, If archetype exists it returns you this archetype without invalidation, if not, it calling BeforeStructuralChange which in turn completes all jobs and invalidate arrays. They described why it's invalidate arrays:
    Code (CSharp):
    1. // Creating an archetype invalidates all iterators / jobs etc
    2. // because it affects the live iteration linked lists...
    Code (CSharp):
    1. internal EntityArchetype CreateArchetype(ComponentType* types, int count)
    2.         {
    3.             ComponentTypeInArchetype* typesInArchetype = stackalloc ComponentTypeInArchetype[count + 1];
    4.             var cachedComponentCount = FillSortedArchetypeArray(typesInArchetype, types, count);
    5.  
    6.             // Lookup existing archetype (cheap)
    7.             EntityArchetype entityArchetype;
    8.             entityArchetype.Archetype =
    9.                 EntityComponentStore->GetExistingArchetype(typesInArchetype, cachedComponentCount);
    10.             if (entityArchetype.Archetype != null)
    11.                 return entityArchetype;
    12.  
    13.             // Creating an archetype invalidates all iterators / jobs etc
    14.             // because it affects the live iteration linked lists...
    15.             BeforeStructuralChange();
    16.  
    17.             var archetypeChanges = EntityComponentStore->BeginArchetypeChangeTracking();
    18.  
    19.             entityArchetype.Archetype = EntityComponentStore->GetOrCreateArchetype(typesInArchetype,
    20.                 cachedComponentCount);
    21.  
    22.             var changedArchetypes = EntityComponentStore->EndArchetypeChangeTracking(archetypeChanges);
    23.             EntityQueryManager.AddAdditionalArchetypes(changedArchetypes);
    24.  
    25.             return entityArchetype;
    26.         }
     
  43. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,707
    Thanks for investigating this. I've been a bit busy and hadn't got around to looking at it yet. Only a simple change is required to fix this.

    My latest version, might be slightly different to above.
    I haven't thoroughly tested this. But tight on time at the moment, I'll have a good look at it in a few days but I think it should work.

    Code (CSharp):
    1. // <copyright file="EntityEventSystem.cs" company="BovineLabs">
    2. //     Copyright (c) BovineLabs. All rights reserved.
    3. // </copyright>
    4.  
    5. namespace BovineLabs.Event
    6. {
    7.     using System;
    8.     using System.Collections.Generic;
    9.     using BovineLabs.Common.Jobs;
    10.     using JetBrains.Annotations;
    11.     using Unity.Burst;
    12.     using Unity.Collections;
    13.     using Unity.Collections.LowLevel.Unsafe;
    14.     using Unity.Entities;
    15.     using Unity.Jobs;
    16.     using UnityEngine.Profiling;
    17.  
    18.     /// <summary>
    19.     /// The EntityEventSystem.
    20.     /// </summary>
    21.     [UsedImplicitly]
    22.     public abstract class EntityEventSystem : ComponentSystem
    23.     {
    24.         private readonly Dictionary<Type, IEventBatch> types = new Dictionary<Type, IEventBatch>();
    25.  
    26.         private JobHandle producerHandle;
    27.  
    28.         /// <summary>
    29.         /// The interface for the batch systems.
    30.         /// </summary>
    31.         private interface IEventBatch : IDisposable
    32.         {
    33.             /// <summary>
    34.             /// Destroys and create the entities.
    35.             /// </summary>
    36.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
    37.             void UpdateEntities(EntityManager entityManager);
    38.  
    39.             /// <summary>
    40.             /// Sets component data if required.
    41.             /// </summary>
    42.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
    43.             /// <returns>A <see cref="JobHandle"/>.</returns>
    44.             JobHandle SetComponentData(EntityManager entityManager);
    45.  
    46.             /// <summary>
    47.             /// Resets the batch for the next frame.
    48.             /// </summary>
    49.             void Reset();
    50.         }
    51.  
    52.         /// <summary>
    53.         /// Creates a queue where any added component added will be batch created as an entity event
    54.         /// and automatically destroyed 1 frame later.
    55.         /// </summary>
    56.         /// <typeparam name="T">The type of <see cref="IComponentData"/>.</typeparam>
    57.         /// <returns>A <see cref="NativeQueue{T}"/> which any component that is added will be turned into a single frame event.</returns>
    58.         public NativeQueue<T> CreateEventQueue<T>()
    59.             where T : struct, IComponentData
    60.         {
    61.             if (!this.types.TryGetValue(typeof(T), out var create))
    62.             {
    63.                 create = this.types[typeof(T)] = new EventBatch<T>(this.EntityManager);
    64.             }
    65.  
    66.             return ((EventBatch<T>)create).GetNew();
    67.         }
    68.  
    69.         /// <summary>
    70.         /// Add a dependency handle.
    71.         /// </summary>
    72.         /// <param name="handle">The dependency handle.</param>
    73.         public void AddJobHandleForProducer(JobHandle handle)
    74.         {
    75.             this.producerHandle = JobHandle.CombineDependencies(this.producerHandle, handle);
    76.         }
    77.  
    78.         /// <inheritdoc />
    79.         protected override void OnDestroy()
    80.         {
    81.             foreach (var t in this.types)
    82.             {
    83.                 t.Value.Dispose();
    84.             }
    85.  
    86.             this.types.Clear();
    87.         }
    88.  
    89.         /// <inheritdoc />
    90.         protected override void OnUpdate()
    91.         {
    92.             this.producerHandle.Complete();
    93.             this.producerHandle = default;
    94.  
    95.             var handles = new NativeArray<JobHandle>(this.types.Count, Allocator.TempJob);
    96.  
    97.             int index = 0;
    98.  
    99.             foreach (var t in this.types)
    100.             {
    101.                 t.Value.UpdateEntities(this.EntityManager);
    102.             }
    103.  
    104.             foreach (var t in this.types)
    105.             {
    106.                 handles[index++] = t.Value.SetComponentData(this.EntityManager);
    107.             }
    108.  
    109.             JobHandle.CompleteAll(handles);
    110.             handles.Dispose();
    111.  
    112.             foreach (var t in this.types)
    113.             {
    114.                 t.Value.Reset();
    115.             }
    116.         }
    117.  
    118.         private class EventBatch<T> : IEventBatch
    119.             where T : struct, IComponentData
    120.         {
    121.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
    122.             private readonly EntityQuery query;
    123.  
    124.             private EntityArchetype archetype;
    125.  
    126.             public EventBatch(EntityManager entityManager)
    127.             {
    128.                 this.query = entityManager.CreateEntityQuery(ComponentType.ReadWrite<T>());
    129.             }
    130.  
    131.             public NativeQueue<T> GetNew()
    132.             {
    133.                 // Having allocation leak warnings when using TempJob
    134.                 var queue = new NativeQueue<T>(Allocator.TempJob);
    135.                 this.queues.Add(queue);
    136.                 return queue;
    137.             }
    138.  
    139.             public void Reset()
    140.             {
    141.                 foreach (var queue in this.queues)
    142.                 {
    143.                     queue.Dispose();
    144.                 }
    145.  
    146.                 this.queues.Clear();
    147.             }
    148.  
    149.             /// <inheritdoc />
    150.             public void Dispose()
    151.             {
    152.                 this.Reset();
    153.             }
    154.  
    155.             /// <inheritdoc />
    156.             public void UpdateEntities(EntityManager entityManager)
    157.             {
    158.                 if (!this.archetype.Valid)
    159.                 {
    160.                     this.archetype = entityManager.CreateArchetype(typeof(T));
    161.                 }
    162.  
    163.                 this.DestroyEntities(entityManager);
    164.  
    165.                 this.CreateEntities(entityManager);
    166.             }
    167.  
    168.             /// <inheritdoc />
    169.             public JobHandle SetComponentData(EntityManager entityManager)
    170.             {
    171.                 var isZeroSized = TypeManager.GetTypeInfo<T>().IsZeroSized;
    172.  
    173.                 if (isZeroSized)
    174.                 {
    175.                     return default;
    176.                 }
    177.  
    178.                 var componentType = entityManager.GetArchetypeChunkComponentType<T>(false);
    179.  
    180.                 var chunks = this.query.CreateArchetypeChunkArray(Allocator.TempJob);
    181.  
    182.                 int startIndex = 0;
    183.  
    184.                 var handles = new NativeArray<JobHandle>(this.queues.Count, Allocator.TempJob);
    185.  
    186.                 // Create a job for each queue. This is designed so that these jobs can run simultaneously.
    187.                 for (var index = 0; index < this.queues.Count; index++)
    188.                 {
    189.                     var queue = this.queues[index];
    190.                     var job = new SetComponentDataJob
    191.                     {
    192.                         Chunks = chunks,
    193.                         Queue = queue,
    194.                         StartIndex = startIndex,
    195.                         ComponentType = componentType,
    196.                     };
    197.  
    198.                     startIndex += queue.Count;
    199.  
    200.                     handles[index] = job.Schedule();
    201.                 }
    202.  
    203.                 var handle = JobHandle.CombineDependencies(handles);
    204.                 handles.Dispose();
    205.  
    206.                 // Deallocate the chunk array
    207.                 handle = new DeallocateJob<ArchetypeChunk>(chunks).Schedule(handle);
    208.  
    209.                 return handle;
    210.             }
    211.  
    212.             private void DestroyEntities(EntityManager entityManager)
    213.             {
    214.                 Profiler.BeginSample("DestroyEntity");
    215.  
    216.                 entityManager.DestroyEntity(this.query);
    217.  
    218.                 Profiler.EndSample();
    219.             }
    220.  
    221.             private void CreateEntities(EntityManager entityManager)
    222.             {
    223.                 var count = this.GetCount();
    224.  
    225.                 if (count == 0)
    226.                 {
    227.                     return;
    228.                 }
    229.  
    230.                 Profiler.BeginSample("CreateEntity");
    231.  
    232.                 // Felt like Temp should be the allocator but gets disposed for some reason.
    233.                 using (var entities =
    234.                     new NativeArray<Entity>(count, Allocator.TempJob, NativeArrayOptions.UninitializedMemory))
    235.                 {
    236.                     entityManager.CreateEntity(this.archetype, entities);
    237.                 }
    238.  
    239.                 Profiler.EndSample();
    240.             }
    241.  
    242.             private int GetCount()
    243.             {
    244.                 var sum = 0;
    245.                 foreach (var i in this.queues)
    246.                 {
    247.                     sum += i.Count;
    248.                 }
    249.  
    250.                 return sum;
    251.             }
    252.  
    253.             [BurstCompile]
    254.             private struct SetComponentDataJob : IJob
    255.             {
    256.                 public int StartIndex;
    257.  
    258.                 public NativeQueue<T> Queue;
    259.  
    260.                 [ReadOnly]
    261.                 public NativeArray<ArchetypeChunk> Chunks;
    262.  
    263.                 [NativeDisableContainerSafetyRestriction]
    264.                 public ArchetypeChunkComponentType<T> ComponentType;
    265.  
    266.                 /// <inheritdoc />
    267.                 public void Execute()
    268.                 {
    269.                     this.GetIndexes(out var chunkIndex, out var entityIndex);
    270.  
    271.                     for (; chunkIndex < this.Chunks.Length; chunkIndex++)
    272.                     {
    273.                         var chunk = this.Chunks[chunkIndex];
    274.  
    275.                         var components = chunk.GetNativeArray(this.ComponentType);
    276.  
    277.                         while (this.Queue.TryDequeue(out var item) && entityIndex < components.Length)
    278.                         {
    279.                             components[entityIndex++] = item;
    280.                         }
    281.  
    282.                         if (this.Queue.Count == 0)
    283.                         {
    284.                             return;
    285.                         }
    286.  
    287.                         entityIndex = entityIndex < components.Length ? entityIndex : 0;
    288.                     }
    289.                 }
    290.  
    291.                 private void GetIndexes(out int chunkIndex, out int entityIndex)
    292.                 {
    293.                     var sum = 0;
    294.  
    295.                     for (chunkIndex = 0; chunkIndex < this.Chunks.Length; chunkIndex++)
    296.                     {
    297.                         var chunk = this.Chunks[chunkIndex];
    298.  
    299.                         var length = chunk.Count;
    300.  
    301.                         if (sum + length < this.StartIndex)
    302.                         {
    303.                             sum += length;
    304.                             continue;
    305.                         }
    306.  
    307.                         entityIndex = this.StartIndex - sum;
    308.                         return;
    309.                     }
    310.  
    311.                     // TODO probably should remove this
    312.                     throw new ArgumentOutOfRangeException(nameof(this.StartIndex));
    313.                 }
    314.             }
    315.         }
    316.     }
    317. }
     
    Last edited: Sep 30, 2019
  44. Ziboo

    Ziboo

    Joined:
    Aug 30, 2011
    Posts:
    295
    Just wanted to say thank you. I used it a bit, and loving it. Completely changed the way I do things now.

    Maybe a nice addition would be to have a generic AddComponent, DestroyComponent Queue like:

    addComponentQueue.Enqueue(entity, new MyComponent()
    {
    MyField = Field
    });