Batch EntityCommandBuffer

So I read @5argon blog post over here this morning https://gametorrahod.com/unity-ecs-batched-operation-on-entitymanager-3c35e8e5ecf4

And as I use quite a few short lived event entities in my project I was curious if I could write something that could generically replaced EntityCommandBuffer for these. So I did!

Firstly code, this is my first, not well tested and quickly written proof of concept.

-edit- dramatically updating code, will post it when i'm done.
-edit2- latested post here, now 3x faster than original version (10x faster than a barrier): https://forum.unity.com/threads/batch-entitycommandbuffer.593569/#post-3965998

How it works. It passes a NativeQueue<T> whenever requested.

Get the barrier with

this.batchBarrier = this.World.GetOrCreateManager<BatchBarrierSystem>();

The same barrier is used for all systems.

To create an entity use

NativeQueue<T>.Concurrent createQueue = this.batchBarrier.GetCreateQueue<T>().ToConcurrent();

where T is an IComponentData that will be added and set on the entity.

Then just pass it to a job, for example

Code (CSharp):

1.         [BurstCompile]
2.         private struct CreateEntitiesJob : IJobParallelFor
3.         {
4.             public NativeQueue<TestComponent>.Concurrent AddComponentQueue;
5.  
6.             /// <inheritdoc />
7.             public void Execute(int index)
8.             {
9.                 this.AddComponentQueue.Enqueue(new TestComponent { Index = index });
10.             }
11.         }

To destroy an entity, use the

NativeQueue<Entity>.Concurrent = this.batchBarrier.GetDestroyQueue().ToConcurrent(),

Any entity added to this queue will be destroyed.

Code (CSharp):

1. [BurstCompile]
2.         private struct DestroyEntitiesJob : IJobProcessComponentDataWithEntity<TestComponent>
3.         {
4.             public NativeQueue<Entity>.Concurrent DestroyEntityQueue;
5.  
6.             /// <inheritdoc />
7.             public void Execute(Entity entity, int index, ref TestComponent data)
8.             {
9.                 this.DestroyEntityQueue.Enqueue(entity);
10.             }
11.         }

The one extra bit you need to do atm is pass the dependency to the system because I don't have any access to AfterUpdate();

Code (CSharp):

1. this.batchBarrier.AddDependency(handle);
2.  
3. return handle;

Why bother?

At the moment it is about 3x faster than using EntityCommandBuffers and completely garbage free.
Also it works in Burst jobs

I believe I could probably optimize it quite a bit if I limit the use.

100,000 entities being added and removed each frame.
This is done in a build (with profiler attached obviously)
I'm not sure why endframebarrier is throwing garbage.

 

Significant performance increase if I force these entities to only live 1 frame (also saves you having to clean them up myself). Has become more like a dedicated event system, which I kind of like.

 

Moving the SetComponent to a job significantly improves performance (renamed to EventSystem though that kind of clashes with the unity ui system so i'll probably change that)



Thats 100k entities created in 3.4ms on a pretty old 3570k, vs 34ms for a barrier.

I can only think of 1 more optimization that needs doing now, and that's making different types set in parallel (won't affect this benchmark results as they are only a single type test)

 

Entities will live 1 frame exactly. Destroying entity is now done automatically.

-edit 2018/12/28-

did some unit testing, found some issues and fixed these. also added parallel setting support, should be quite a bit faster.

Source code is now here: https://github.com/tertle/EventBatchSystem

-edit 2019/01/09- Read below, repo moved.

 

@tertle Can you briefly explain how this EventSystem works? I am converting a game from Rx-driven events to ECS and have the most trouble with events.

 

Sure. So I have a few short lived entities with a single component attached that I use as kind of an event.

For example to request a path in my project, you create a new entity with a FindPath component. This component will be handled by the pathfinding system (though other events might be handled by multiple systems) and then destroyed.

The point of this, I think I called it BatchSystem for the moment as it shared the same name as the unity UI class though it really needs a better name, is to batch create all entity events at once (of the same type, though as I typed this I just realized I could maybe merge the creation of all of event entities at once) instead of creating them individually 1 at a time.

So instead of using EntityCommandBuffer and creating the entity yourself, you BatchSystem.GetEventBatch<T>() where T where the event component. This will return a NativeQueue<T> which every time the method is called so you don't need to worry about other system dependencies and will be automatically merged. You simply add the event struct to this queue instead and when BatchSystem updates, it'll create a bunch of entities and set the component data.

All these newly created entities will be stored in the BatchSystem and destroyed 1 frame later in a single batch destroy entity call. You don't need to worry about maintaining the life of the event entities it's handled for you. (Whichcan be very useful if your event entities are used by more than 1 system, you don't have to worry about ordering.)

NativeQueue has a ToConcurrent() method so can be used in parallel but can also be burst compiled unlike entity command buffer.

-edit-

side note, if anyone was using this before today I updated the code with a small bug fix a few hours ago.

 

Amazing work, thank you

 

Can we have permission to use your BatchSystem and NativeUnit code?
I'm asking because I've noticed the all rights reserved copyright.

 

Oh yeah of course. They are just auto generated by StyleCop and I'm too lazy to strip it when copy/pasting.

~feel free to use any code I post on unity forums, now or in the future.

-edit-

side note, NativeUnit is really just a NativeArray with a fixed length of 1. More of a convenience.

 

Cool, thanks!

I thought of it as a "NativeReference" . Even if it's just like a 1-array, it makes code clearer and I'm all about that!

I just hope someone figures out a way to return a reference, to access direct memory to get rid of the

o = arr[0]; o.var = 123; arr[0] = o

idiom for structs.

 

finally got around to doing some unit testing and found some issues (and fixed them).
Also added parallel setting support, should be quite a bit faster. -note- while this is passing all tests, i probably need to do some large scale validation to ensure thread safety.

Source code is now here: https://github.com/tertle/com.bovinelabs.entities

 

A little recommendation about repo's "shape" to support UPM git pulling, if you move the git init location one level deeper inside the Assets folder then put package.json, then anyone can pull that from Unity. It would appears in Project window like other Unity packages. (Wrapped one level in a folder named after the name specified in package.json) dependencies will go in package.json instead of manifest.json. .gitignore will also has to be Assets in this case.

I have 1 Unity project named PackageDev where in the Assets I have multiple folders representing each small package with its own repo. I set a local UPM on each folder so that the change applies to all games, and git would be able to take them online.

 

Actually I forgot about this feature. I looked at it when it first came out but ran into an issue and totally forgot.

I've moved my code to here: https://github.com/tertle/com.bovinelabs.entities if anyone wants.

 

@tertle could you add a small ReadMe that describes how to get started if it's possible?

 

Yeah sure, I'm about to head out for NYE so I'll do it soon as I get back in the new year.

 

Happy holidays to you.

 

Added a short readme to https://github.com/tertle/com.bovinelabs.entities

If there is anything in particular that needs more explaining let me know, i'm terrible at explaining things.

 

Awesome work! Is there a way to add multiple IComponentData to an entity? Also, is it possible to add an ISharedComponentData?

 

Looks like cool work has been done here.
But got question, since I maybe missed, or misunderstood.
You have mentioned short leaving entities. But is this approach good for creating persistent entities? Sorry, haven't chance yet, to test the code.

 

Neither of these are that hard to add but I don't really see the point of them for this particular sytem.

Yes, if you're creating a lot of (persistent) entities in a frame you should batch create them, much faster (this system won't do that for you though.)

 

Found kind of an elegant solution my dependency issues that someone might be interested in.

So previously to handle dependencies, I was grabbing the internal JobHandle m_PreviousFrameDependency with reflection but unfortunately as JobHandle is a struct, and without reflection and GetValue is designed structs are always boxed it was generating garbage each frame.

I've gone in and redesigned how I handle dependencies now. Instead, I pass a fake BarrierSystem to the JobComponentSystem (handled automatically with reflection once per life of JobComponentSystem) and then manually update this system.

It works remarkably well. The barrier doesn't have any command buffers, but just completes all dependencies. This system is lazy created to avoid being added to the default loop, but if you happen to use this and update your loop mid way through for some reason, this will break.

TLDR: nothing has changed in how you use it, just no more garbage.

 

So I read all of this code and like it very much. I have one question regarding this section:

Code (CSharp):

1.   private class EventBatch<T> : EventBatchBase
2.             where T : struct, IComponentData
3.         {
4.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
5.  
6. // .. SNIP ...
7.  
8.             public NativeQueue<T> GetNew()
9.             {
10.                 // Having allocation leak warnings when using TempJob
11.                 var queue = new NativeQueue<T>(Allocator.Persistent);
12.                 this.queues.Add(queue);
13.  
14.                 return queue;
15.             }
16. }

Why is it necessary to create multiple queues per event type?

NativeQueue 

has a

ToConcurrent

method which would make the Queue safe to use from multiple threads and we actually don't really care about the order of things as all entities get created in one fell swoop. So just from looking at it I would say you could do with a single queue (which would simplify some code later where it maps the single queues to segments in the chunk). Then again, I have little experience with this, so I would really like to know why you chose to design it that way. Thanks for sharing this!

 

Because this

Code (CSharp):

1.         var handle1 = new Job1
2.             {
3.                 Entities = this.queue.ToConcurrent()
4.             }
5.             .Schedule(this, inputDeps);
6.  
7.         var handle2 = new Job2
8.             {
9.                 Entities = this.queue.ToConcurrent()
10.             }
11.             .Schedule(this, inputDeps);
12.  
13.         var handle = JobHandle.CombineDependencies(handle1, handle2);

Will throw this specific error

Note job1 dependency isn't passed to job2 so these jobs run in parallel, not sequential (which is fine.)

NativeContainers.Concurrent are only safety checked within the same job, not across multiple jobs so by creating a separate container for each system I avoid these dependency issues.

 

So is this really a problem or is it just a false positive of the safety system? My thinking is that if a

NativeContainer.Concurrent

is safe to use by multiple threads within one job, in theory it should also be safe to use across jobs, after all in both cases you have multiple threads concurrently accessing the container.

So if I would add some

NativeDisableContainerSafetyRestriction 

attribute to

Entities 

and therefore circumvent the suspected false positive, would it work? I'm not saying that this is a particulary good idea, I just would like to know if I understood the thing correctly

 

1) different jobs scheduling ordering is non-deterministic without dependencies, even on the same thread. (this is also true for different "parts" of the same parallel job, but a parallel job should be designed to do the same thing to all of its parts, so it shouldn't matter
2) concurrent containers may work by using an internal [ThreadIndex] and that may be set uniquely only in a single .Schedule call (e.g. different scheduled jobs may have conflicting threadIndex in different threads, we can't know unless @Joachim_Ante tells us otherwise)

 

is it possible to use this with UPM? am getting a package dependency error when adding "com.bovinelabs.entities": "0.1.2" to my manifest (2019.1.0b5). am a bit new with how custom packages work.

 

Been busy with something so haven't properly updated my libraries. I have updated this to the latest version of entities but I haven't updated the dependencies of the package yet. I'll l do that tomorrow thanks for letting me know.

 

This is how to UPM to GitHub

 "com.bovinelabs.entities": "git://github.com/tertle/com.bovinelabs.entities.git",

to update delete the new lock section appearing in your manifest after the first pull.

 

I tried to actually add this with package manager today but keep getting:

Code (CSharp):

1. An error occurred while resolving packages:
2.   Package com.bovinelabs.entities@git://github.com/tertle/com.bovinelabs.entities.git has invalid dependencies:
3.     com.bovinelabs.naughtyattributes: Version 'https://github.com/tertle/com.bovinelabs.naughtyattributes.git' is invalid. Expected a 'SemVer' compatible value.

I tried removing the locks, removing it, restarting Unity, adding again, etc, but it seems to keep ending up with that. I tried to see if just adding your naughtyattributes repo as well might help, but it didn't seem to make a difference.

 

oh i think i just broke this today by accident as i was removing old repos from harddrive and pushed a few random things just to make sure i wasn't deleting anything.

i have reorganized my packages internally a lot so this is a separate standalone repo for me (which is private atm)

if you just need the code, this is my latest version. i'll see if i can make this public so you can pull from git, has a few unit tests.

Code (CSharp):

1. // <copyright file="EntityEventSystem.cs" company="BovineLabs">
2. //     Copyright (c) BovineLabs. All rights reserved.
3. // </copyright>
4.  
5. namespace BovineLabs.Event
6. {
7.     using System;
8.     using System.Collections.Generic;
9.     using BovineLabs.Common.Jobs;
10.     using JetBrains.Annotations;
11.     using Unity.Burst;
12.     using Unity.Collections;
13.     using Unity.Collections.LowLevel.Unsafe;
14.     using Unity.Entities;
15.     using Unity.Jobs;
16.     using UnityEngine.Profiling;
17.  
18.     /// <summary>
19.     /// The BatchBarrierSystem.
20.     /// </summary>
21.     [UsedImplicitly]
22.     public abstract class EntityEventSystem : ComponentSystem
23.     {
24.         private readonly Dictionary<Type, IEventBatch> types = new Dictionary<Type, IEventBatch>();
25.  
26.         private JobHandle producerHandle;
27.  
28.         /// <summary>
29.         /// The interface for the batch systems.
30.         /// </summary>
31.         private interface IEventBatch : IDisposable
32.         {
33.             /// <summary>
34.             /// Updates the batch, destroy, create, set.
35.             /// </summary>
36.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
37.             /// <returns>A <see cref="JobHandle"/>.</returns>
38.             JobHandle Update(EntityManager entityManager);
39.  
40.             /// <summary>
41.             /// Resets the batch for the next frame.
42.             /// </summary>
43.             void Reset();
44.         }
45.  
46.         /// <summary>
47.         /// Creates a queue where any added component added will be batch created as an entity event
48.         /// and automatically destroyed 1 frame later.
49.         /// </summary>
50.         /// <typeparam name="T">The type of <see cref="IComponentData"/>.</typeparam>
51.         /// <returns>A <see cref="NativeQueue{T}"/> which any component that is added will be turned into a single frame event.</returns>
52.         public NativeQueue<T> CreateEventQueue<T>()
53.             where T : struct, IComponentData
54.         {
55.             if (!this.types.TryGetValue(typeof(T), out var create))
56.             {
57.                 create = this.types[typeof(T)] = new EventBatch<T>(this.EntityManager);
58.             }
59.  
60.             return ((EventBatch<T>)create).GetNew();
61.         }
62.  
63.         /// <summary>
64.         /// Add a dependency handle.
65.         /// </summary>
66.         /// <param name="handle">The dependency handle.</param>
67.         public void AddJobHandleForProducer(JobHandle handle)
68.         {
69.             this.producerHandle = JobHandle.CombineDependencies(this.producerHandle, handle);
70.         }
71.  
72.         /// <inheritdoc />
73.         protected override void OnDestroy()
74.         {
75.             foreach (var t in this.types)
76.             {
77.                 t.Value.Dispose();
78.             }
79.  
80.             this.types.Clear();
81.         }
82.  
83.         /// <inheritdoc />
84.         protected override void OnUpdate()
85.         {
86.             this.producerHandle.Complete();
87.             this.producerHandle = default;
88.  
89.             var handles = new NativeArray<JobHandle>(this.types.Count, Allocator.TempJob);
90.  
91.             int index = 0;
92.  
93.             foreach (var t in this.types)
94.             {
95.                 handles[index++] = t.Value.Update(this.EntityManager);
96.             }
97.  
98.             JobHandle.CompleteAll(handles);
99.             handles.Dispose();
100.  
101.             foreach (var t in this.types)
102.             {
103.                 t.Value.Reset();
104.             }
105.         }
106.  
107.         private class EventBatch<T> : IEventBatch
108.             where T : struct, IComponentData
109.         {
110.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
111.             private readonly EntityQuery query;
112.  
113.             private readonly EntityArchetype archetype;
114.  
115.             public EventBatch(EntityManager entityManager)
116.             {
117.                 this.query = entityManager.CreateEntityQuery(ComponentType.ReadWrite<T>());
118.                 this.archetype = entityManager.CreateArchetype(typeof(T));
119.             }
120.  
121.             public NativeQueue<T> GetNew()
122.             {
123.                 // Having allocation leak warnings when using TempJob
124.                 var queue = new NativeQueue<T>(Allocator.TempJob);
125.                 this.queues.Add(queue);
126.                 return queue;
127.             }
128.  
129.             public void Reset()
130.             {
131.                 foreach (var queue in this.queues)
132.                 {
133.                     queue.Dispose();
134.                 }
135.  
136.                 this.queues.Clear();
137.             }
138.  
139.             /// <inheritdoc />
140.             public void Dispose()
141.             {
142.                 this.Reset();
143.             }
144.  
145.             /// <summary>
146.             /// Handles the destroying of entities.
147.             /// </summary>
148.             /// <param name="entityManager">The entity manager.</param>
149.             /// <returns>A default handle.</returns>
150.             public JobHandle Update(EntityManager entityManager)
151.             {
152.                 this.DestroyEntities(entityManager);
153.  
154.                 if (!this.CreateEntities(entityManager))
155.                 {
156.                     return default;
157.                 }
158.  
159.                 return this.SetComponentData(entityManager);
160.             }
161.  
162.             private void DestroyEntities(EntityManager entityManager)
163.             {
164.                 Profiler.BeginSample("DestroyEntity");
165.  
166.                 entityManager.DestroyEntity(this.query);
167.  
168.                 Profiler.EndSample();
169.             }
170.  
171.             private bool CreateEntities(EntityManager entityManager)
172.             {
173.                 var count = this.GetCount();
174.  
175.                 if (count == 0)
176.                 {
177.                     return false;
178.                 }
179.  
180.                 Profiler.BeginSample("CreateEntity");
181.  
182.                 // Felt like Temp should be the allocator but gets disposed for some reason.
183.                 using (var entities = new NativeArray<Entity>(count, Allocator.TempJob, NativeArrayOptions.UninitializedMemory))
184.                 {
185.                     entityManager.CreateEntity(this.archetype, entities);
186.                 }
187.  
188.                 Profiler.EndSample();
189.                 return true;
190.             }
191.  
192.             private JobHandle SetComponentData(EntityManager entityManager)
193.             {
194.                 var isZeroSized = TypeManager.GetTypeInfo<T>().IsZeroSized;
195.  
196.                 if (isZeroSized)
197.                 {
198.                     return default;
199.                 }
200.  
201.                 var componentType = entityManager.GetArchetypeChunkComponentType<T>(false);
202.  
203.                 var chunks = this.query.CreateArchetypeChunkArray(Allocator.TempJob);
204.  
205.                 int startIndex = 0;
206.  
207.                 var handles = new NativeArray<JobHandle>(this.queues.Count, Allocator.TempJob);
208.  
209.                 // Create a job for each queue. This is designed so that these jobs can run simultaneously.
210.                 for (var index = 0; index < this.queues.Count; index++)
211.                 {
212.                     var queue = this.queues[index];
213.                     var job = new SetComponentDataJob
214.                     {
215.                         Chunks = chunks,
216.                         Queue = queue,
217.                         StartIndex = startIndex,
218.                         ComponentType = componentType,
219.                     };
220.  
221.                     startIndex += queue.Count;
222.  
223.                     handles[index] = job.Schedule();
224.                 }
225.  
226.                 var handle = JobHandle.CombineDependencies(handles);
227.                 handles.Dispose();
228.  
229.                 // Deallocate the chunk array
230.                 handle = new DeallocateJob<ArchetypeChunk>(chunks).Schedule(handle);
231.  
232.                 return handle;
233.             }
234.  
235.             private int GetCount()
236.             {
237.                 var sum = 0;
238.                 foreach (var i in this.queues)
239.                 {
240.                     sum += i.Count;
241.                 }
242.  
243.                 return sum;
244.             }
245.  
246.             [BurstCompile]
247.             private struct SetComponentDataJob : IJob
248.             {
249.                 public int StartIndex;
250.  
251.                 public NativeQueue<T> Queue;
252.  
253.                 [ReadOnly]
254.                 public NativeArray<ArchetypeChunk> Chunks;
255.  
256.                 [NativeDisableContainerSafetyRestriction]
257.                 public ArchetypeChunkComponentType<T> ComponentType;
258.  
259.                 /// <inheritdoc />
260.                 public void Execute()
261.                 {
262.                     this.GetIndexes(out var chunkIndex, out var entityIndex);
263.  
264.                     for (; chunkIndex < this.Chunks.Length; chunkIndex++)
265.                     {
266.                         var chunk = this.Chunks[chunkIndex];
267.  
268.                         var components = chunk.GetNativeArray(this.ComponentType);
269.  
270.                         while (this.Queue.TryDequeue(out var item) && entityIndex < components.Length)
271.                         {
272.                             components[entityIndex++] = item;
273.                         }
274.  
275.                         if (this.Queue.Count == 0)
276.                         {
277.                             return;
278.                         }
279.  
280.                         entityIndex = entityIndex < components.Length ? entityIndex : 0;
281.                     }
282.                 }
283.  
284.                 private void GetIndexes(out int chunkIndex, out int entityIndex)
285.                 {
286.                     var sum = 0;
287.  
288.                     for (chunkIndex = 0; chunkIndex < this.Chunks.Length; chunkIndex++)
289.                     {
290.                         var chunk = this.Chunks[chunkIndex];
291.  
292.                         var length = chunk.Count;
293.  
294.                         if (sum + length < this.StartIndex)
295.                         {
296.                             sum += length;
297.                             continue;
298.                         }
299.  
300.                         entityIndex = this.StartIndex - sum;
301.                         return;
302.                     }
303.  
304.                     throw new ArgumentOutOfRangeException(nameof(this.StartIndex));
305.                 }
306.             }
307.         }
308.     }
309. }

Code (CSharp):

1. // <copyright file="EndFrameEntityEventSystems.cs" company="BovineLabs">
2. // Copyright (c) BovineLabs. All rights reserved.
3. // </copyright>
4.  
5. namespace BovineLabs.Event
6. {
7.     using Unity.Entities;
8.  
9.     /// <summary>
10.     /// <see cref="EntityEventSystem"/> that runs in the <see cref="LateSimulationSystemGroup"/>.
11.     /// </summary>
12.     [UpdateInGroup(typeof(LateSimulationSystemGroup))]
13.     public sealed class EndSimulationEntityEventSystem : EntityEventSystem
14.     {
15.     }
16.  
17.     /// <summary>
18.     /// <see cref="EntityEventSystem"/> that runs in the <see cref="PresentationSystemGroup"/>.
19.     /// </summary>
20.     [UpdateInGroup(typeof(PresentationSystemGroup))]
21.     public sealed class PresentationEntityEventSystem : EntityEventSystem
22.     {
23.     }
24. }

 

I am trying to figure out exactly how to use this, is it strictly for creating entities with a component as well as destroying them, or am I able to also add/remove from already existing entities as well?

I was following your advice from another thread and isnteadof removing a component tag in a job with command buffer in the main job, I made a queue and added the entities to that and then that dequeues and runs after the main job so that the main one can be burstable.

When I saw this and that it produced no garbage, I thought that would be great as my overall GC is killing me, so I am trying to optimize wherever possible and this seemed like a good place to start if it could remove components.

 

This specific system is an event system. It lets you create events that last 1 frame (handles cleaning up for you) and does this optimally by grouping the same events that could have been created from multiple different systems into a single batch call (which is much faster than using an EntityCommandBuffer in each system.)

It's also somewhat designed to stop the need for short lived component tags that are often used. Instead of adding a Damage tag (or Killed tag etc) to an entity, you could create a Damage event with a reference to the entity and query that instead.

This way you can keep your entity archetype quite static and it does not have any costly moving operations. Combining this with simple polling (and chunk has changed etc) you can in most circumstances never modify an entities archetype. It can also help by having to worry less about system order.

If you just want to efficiently remove a bunch of components from entities, you should consider doing by query if possible.

EntityManager.RemoveComponent<T>(EntityQuery)

 

I understand now. Sounds like a great system! I was using EntityQuery for the addition and removal of the entities, but that is just as of recently, I had setup the queue to remove the tag before that I think, so... perhaps I didn't even think to try doing it by query since I saw your way and it seemed a lot better than my original way with ECB in the main job. I will give that a try and see how it goes.

Side note, I followed your example yet again from another-another thread, lol. I was doing ijobforeachwithentity but I have an insane amount of allocations and job/tempalloc overflow (for some reason it happens no matter what size I try to make the size of the container : / ), so I did as you mentioned and instead of IJFEWE I moved to IJobCunk so that I had less overall allocations because it was no longer 'per entity'. Do you have any recommendations on how I might be able to take it even further?

Granted, my waypoint system and navigation is fully working, and works well, but when I drop in 10k vehicles, it's not nearly as good as I would like it to be, so really I am just trying to learn some tips and tricks for optimization because my entire system was thrown together with the level of planning being "oh nice, that worked!", simply because I didn't know any other way to accomplish what I was doing. Hence my finding and trying out a lot of your suggestions, as so far each one has greatly improved my system overall, I just want to try an optimize it as far as I possibly can to learn.

That being said, my biggest offenders are these guys. I just exposted allocator size to the inspector because I got tired of recompiling.

Code (CSharp):

1.                
2.                 var frontTier = new NativeList<Node>(allocatorSize, Allocator.Temp);
3.                 var path = new NativeList<WaypointECS>(allocatorSize, Allocator.Temp);
4.                 var connTemp = new NativeList<WaypointConnections>(allocatorSize, Allocator.Temp);
5.                 var costSoFar = new NativeHashMap<WaypointECS, float>(allocatorSize, Allocator.Temp);
6.                 var cameFrom = new NativeHashMap<WaypointECS, WaypointECS>(allocatorSize, Allocator.Temp);

It's a pretty standard A* system, and as I mentioned I was able to move those outside of the main iteration of the chunk so it is not getting hit each and every time which definitely did help a lot when I have anywhere from 10-50k vehicles pathing.

Do you have any recommendation on another way to go about this perhaps? I tried to see if maybe I could allocate all of these things outside the job itself and then pass it in all empty and then just clear it on each iteration, but I kept running into issues with concurrent writing and needing to mark them as ReadOnly, etc. Does any of that change if I am doing Chunk Iter now? I don't think it might considering it still parallelizes the chunks. I was wondering if there was a way maybe to more or less fake using a NHM by using something else that could be passed in blank and cleared. Could the EventQueue somehow play a role in this? I guess it just depends on how fast everything ends up completing. At first I was splitting the work over a few frames because it was a hefty beast of a job. It was Oritinally something like 1800ms, but I got it down to this below, then a bit more after some more tweaking and moving around some iterations inside the job. Now it is probably 300ms total or so when I don't split the frames a bit.

Anyways, my apologies, I didn't expect this post to get as long as it did. Once I started typing I just kind of kept on going. I probably should have just made a new thread, lol.

 

Hey, sorry about slow reply i've been ill.

IJobChunk is exactly how I would have suggested doing it. There's no real way to reduce allocations more than caching them 1 per thread as you're doing except to do some seriously hackory (like I did with my original pathfinding solution before you could allocate in jobs) which I would not suggest.

I'm assuming by what you previously said, they are only allocated once per chunk (outside of the loop) and are cleared. Unless you have a very fragmented entities you really shouldn't have that many chunks therefore very few allocations so I'm not sure why it'd be /that/ much of an issue.

If the clear is causing issues because the hashmaps grow very large and you are using 2019.3 you could use the solution i posted over here: https://forum.unity.com/threads/nativehashmap-clear-4x-performance-improvement-request.732782/ as it's about 4x faster (let me know and I'll find my code.)

You'd have to switch to IJob and allocate 1 container per job and manually schedule them. I would not recommend (how I create my original pathfinding though before in job allocations existed.)

Anyway, pathfinding is expensive as the world grid grows. Doing 10k requests/frame is pretty unheard of. How large is your grid?

In the past I've done a few optimizations for pathfinding. When I was using a huge 4096x4096 grid I used to limit paths to 100 steps. If the target was further away than this it'd return the best expected 100 steps and follow this and would simply do another request for a path when it got near to the end. Works great as long as you don't have any long dead ends (>100 steps).

Also you could simply look at limiting the number of requests/frame to smooth out the frame rate

 

I appreciate the reply. I wanted to try and use 2019.3 but I have an incident opened with Unity as my project is using LWRP (or well, Universal RP on 2019.3), when I updated, this happened : https://i.imgur.com/m0m9i6Y.gifv any time I would try to spawn my entities.

Also, I am not using a grid setup. Each waypoint has a list of the next hop waypoints it is directly connected to https://i.imgur.com/OpSD58s.png. This was to be able to easily create paths in a single direction so I could have 2-way streets. Each waypoint in the pic has a color between it and the next hop, if one waypoint has two colors coming off of it, that is because it is connected to those two waypoints for turns and what not. I am colorblind, so not sure why I even bothered, I can't really tell with most of them if it's accurate, lol. The red waypoints are simply "connecting" waypoints to feed the path, the blue ones are actual possible destinations randomly spread out.

As we speak I am working on a way to try and remove the tags I was using to designate when it's time for each entity to obtain a new waypoint. When it reached it's goal I added a VehicleNeedsPathTag and the path system just had a RequireComponentTag of that and then it would remove it when it was finished. I was going to see about adding a new IComponentData to each one with just an int on it of whether it needs a path or not. Set it to 1 when it does, 0 otherwise.

I don't know much about ChangeFilters yet, but you mentioned tagging moving them around so I wanted to try something else so I was about to start researching how those work and see if it might help. It could very well be that there is fragmentation and I just am not sure exactly what I should be looking for to fix it. This is my first system and as such I been just learning things as I go along trying to redo each bottleneck I come across.

 

Nice, removing the use of tags and in turn removing the ECB, it seems the ever growing allocations gone. For whatever reason it would end up accumulating a MB or two every few seconds, but I just let it run for several minutes and it held steady. After a few more tweaks it looks like I got the pathfind situated as well. Looking at the Entity debugger I don't see tons of changes all over the place like I did before. I appreciate the advice.

The only thing I can't seem to figure out how to get rid of is is the tons of TempAlloc.Overflow / JobAlloc.Grow. It seems no matter how I set the size of the allocations I always end up having them.

 

Hey @tertle, I've been messing with this and it's working awesome except i'm having an issue where sending an event queue to a job invalidates any other nativecontainers sent to the job. Have you had any issues with that?

Here's some example code that causes the issue. Am I doing something wrong?

Code (CSharp):

1.         private EndSimulationEntityEventSystem m_EventSystem;
2.  
3.         protected override void OnCreate()
4.         {
5.             m_EventSystem = World.GetOrCreateSystem<EndSimulationEntityEventSystem>();
6.         }
7.         public struct ReactJob : IJobForEach<HitPosition>
8.         {
9.             [ReadOnly]
10.             public ComponentDataFromEntity<LocalToWorld> localToWorldFromEntity;
11.             [ReadOnly]
12.             [DeallocateOnJobCompletion]
13.             public NativeArray<Entity> unitEntities;
14.             public NativeQueue<ReactToHit>.ParallelWriter eventQueue;
15.             public float spread;
16.  
17.             public void Execute([ReadOnly] ref HitPosition hit)
18.             {
19.                 for (int i = 0; i < unitEntities.Length; i++)
20.                 {
21.                     if (localToWorldFromEntity.Exists(unitEntities[i]))
22.                     {
23.                         if (distance(hit.Position, localToWorldFromEntity[unitEntities[i]].Position) < spread)
24.                         {
25.                             eventQueue.Enqueue(new ReactToHit()
26.                             {
27.                                 Entity = unitEntities[i]
28.                             });
29.                         }
30.                     }
31.                 }
32.             }
33.         }
34.  
35.         protected override JobHandle OnUpdate(JobHandle inputDept)
36.         {
37.  
38.             var job = new ReactJob()
39.             {
40.                 localToWorldFromEntity = GetComponentDataFromEntity<LocalToWorld>(true),
41.                 unitEntities = units,
42.                 eventQueue = m_EventSystem.CreateEventQueue<ReactToHit>().AsParallelWriter(),
43.                 spread = 2.5f,
44.             };
45.             var handle = job.Schedule(this, inputDept);
46.             m_EventSystem.AddJobHandleForProducer(handle);
47.             return handle;
48.         }

 

That's a bit strange.

Also what is units?

 

CreateArchetype synch point isn’t it? (From mobile, can’t check) In CreateEvenQueue it creates EventBatch which in turn creates EntityQuery and Archetype. If yes in his case just move creating at top of OnUpdate and use it later. But anyway seems it’s not full his code...

 

Probably correct but I'm surprised a new archetype invalidates arrays (haven't tested). If so I'll have to reorganize it a little because I don't think calls to this should be order dependent.

 

eizenhorn was correct!!! That does indeed solve the issue. So I guess CreateArchetype is a sync point.

Sorry, I should have made it more clear. The code example I posted was just simplified example of how I'm using the event stuff. It's GetComponentDataFromEntity, GetArchetypeChunkComponentType, and ToComponentDataArray that get invalidated.

 

Finally on PC, If archetype exists it returns you this archetype without invalidation, if not, it calling BeforeStructuralChange which in turn completes all jobs and invalidate arrays. They described why it's invalidate arrays:

Code (CSharp):

1. // Creating an archetype invalidates all iterators / jobs etc
2. // because it affects the live iteration linked lists...

Code (CSharp):

1. internal EntityArchetype CreateArchetype(ComponentType* types, int count)
2.         {
3.             ComponentTypeInArchetype* typesInArchetype = stackalloc ComponentTypeInArchetype[count + 1];
4.             var cachedComponentCount = FillSortedArchetypeArray(typesInArchetype, types, count);
5.  
6.             // Lookup existing archetype (cheap)
7.             EntityArchetype entityArchetype;
8.             entityArchetype.Archetype =
9.                 EntityComponentStore->GetExistingArchetype(typesInArchetype, cachedComponentCount);
10.             if (entityArchetype.Archetype != null)
11.                 return entityArchetype;
12.  
13.             // Creating an archetype invalidates all iterators / jobs etc
14.             // because it affects the live iteration linked lists...
15.             BeforeStructuralChange();
16.  
17.             var archetypeChanges = EntityComponentStore->BeginArchetypeChangeTracking();
18.  
19.             entityArchetype.Archetype = EntityComponentStore->GetOrCreateArchetype(typesInArchetype,
20.                 cachedComponentCount);
21.  
22.             var changedArchetypes = EntityComponentStore->EndArchetypeChangeTracking(archetypeChanges);
23.             EntityQueryManager.AddAdditionalArchetypes(changedArchetypes);
24.  
25.             return entityArchetype;
26.         }

 

Thanks for investigating this. I've been a bit busy and hadn't got around to looking at it yet. Only a simple change is required to fix this.

My latest version, might be slightly different to above.
I haven't thoroughly tested this. But tight on time at the moment, I'll have a good look at it in a few days but I think it should work.

Code (CSharp):

1. // <copyright file="EntityEventSystem.cs" company="BovineLabs">
2. //     Copyright (c) BovineLabs. All rights reserved.
3. // </copyright>
4.  
5. namespace BovineLabs.Event
6. {
7.     using System;
8.     using System.Collections.Generic;
9.     using BovineLabs.Common.Jobs;
10.     using JetBrains.Annotations;
11.     using Unity.Burst;
12.     using Unity.Collections;
13.     using Unity.Collections.LowLevel.Unsafe;
14.     using Unity.Entities;
15.     using Unity.Jobs;
16.     using UnityEngine.Profiling;
17.  
18.     /// <summary>
19.     /// The EntityEventSystem.
20.     /// </summary>
21.     [UsedImplicitly]
22.     public abstract class EntityEventSystem : ComponentSystem
23.     {
24.         private readonly Dictionary<Type, IEventBatch> types = new Dictionary<Type, IEventBatch>();
25.  
26.         private JobHandle producerHandle;
27.  
28.         /// <summary>
29.         /// The interface for the batch systems.
30.         /// </summary>
31.         private interface IEventBatch : IDisposable
32.         {
33.             /// <summary>
34.             /// Destroys and create the entities.
35.             /// </summary>
36.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
37.             void UpdateEntities(EntityManager entityManager);
38.  
39.             /// <summary>
40.             /// Sets component data if required.
41.             /// </summary>
42.             /// <param name="entityManager">The <see cref="EntityManager"/>.</param>
43.             /// <returns>A <see cref="JobHandle"/>.</returns>
44.             JobHandle SetComponentData(EntityManager entityManager);
45.  
46.             /// <summary>
47.             /// Resets the batch for the next frame.
48.             /// </summary>
49.             void Reset();
50.         }
51.  
52.         /// <summary>
53.         /// Creates a queue where any added component added will be batch created as an entity event
54.         /// and automatically destroyed 1 frame later.
55.         /// </summary>
56.         /// <typeparam name="T">The type of <see cref="IComponentData"/>.</typeparam>
57.         /// <returns>A <see cref="NativeQueue{T}"/> which any component that is added will be turned into a single frame event.</returns>
58.         public NativeQueue<T> CreateEventQueue<T>()
59.             where T : struct, IComponentData
60.         {
61.             if (!this.types.TryGetValue(typeof(T), out var create))
62.             {
63.                 create = this.types[typeof(T)] = new EventBatch<T>(this.EntityManager);
64.             }
65.  
66.             return ((EventBatch<T>)create).GetNew();
67.         }
68.  
69.         /// <summary>
70.         /// Add a dependency handle.
71.         /// </summary>
72.         /// <param name="handle">The dependency handle.</param>
73.         public void AddJobHandleForProducer(JobHandle handle)
74.         {
75.             this.producerHandle = JobHandle.CombineDependencies(this.producerHandle, handle);
76.         }
77.  
78.         /// <inheritdoc />
79.         protected override void OnDestroy()
80.         {
81.             foreach (var t in this.types)
82.             {
83.                 t.Value.Dispose();
84.             }
85.  
86.             this.types.Clear();
87.         }
88.  
89.         /// <inheritdoc />
90.         protected override void OnUpdate()
91.         {
92.             this.producerHandle.Complete();
93.             this.producerHandle = default;
94.  
95.             var handles = new NativeArray<JobHandle>(this.types.Count, Allocator.TempJob);
96.  
97.             int index = 0;
98.  
99.             foreach (var t in this.types)
100.             {
101.                 t.Value.UpdateEntities(this.EntityManager);
102.             }
103.  
104.             foreach (var t in this.types)
105.             {
106.                 handles[index++] = t.Value.SetComponentData(this.EntityManager);
107.             }
108.  
109.             JobHandle.CompleteAll(handles);
110.             handles.Dispose();
111.  
112.             foreach (var t in this.types)
113.             {
114.                 t.Value.Reset();
115.             }
116.         }
117.  
118.         private class EventBatch<T> : IEventBatch
119.             where T : struct, IComponentData
120.         {
121.             private readonly List<NativeQueue<T>> queues = new List<NativeQueue<T>>();
122.             private readonly EntityQuery query;
123.  
124.             private EntityArchetype archetype;
125.  
126.             public EventBatch(EntityManager entityManager)
127.             {
128.                 this.query = entityManager.CreateEntityQuery(ComponentType.ReadWrite<T>());
129.             }
130.  
131.             public NativeQueue<T> GetNew()
132.             {
133.                 // Having allocation leak warnings when using TempJob
134.                 var queue = new NativeQueue<T>(Allocator.TempJob);
135.                 this.queues.Add(queue);
136.                 return queue;
137.             }
138.  
139.             public void Reset()
140.             {
141.                 foreach (var queue in this.queues)
142.                 {
143.                     queue.Dispose();
144.                 }
145.  
146.                 this.queues.Clear();
147.             }
148.  
149.             /// <inheritdoc />
150.             public void Dispose()
151.             {
152.                 this.Reset();
153.             }
154.  
155.             /// <inheritdoc />
156.             public void UpdateEntities(EntityManager entityManager)
157.             {
158.                 if (!this.archetype.Valid)
159.                 {
160.                     this.archetype = entityManager.CreateArchetype(typeof(T));
161.                 }
162.  
163.                 this.DestroyEntities(entityManager);
164.  
165.                 this.CreateEntities(entityManager);
166.             }
167.  
168.             /// <inheritdoc />
169.             public JobHandle SetComponentData(EntityManager entityManager)
170.             {
171.                 var isZeroSized = TypeManager.GetTypeInfo<T>().IsZeroSized;
172.  
173.                 if (isZeroSized)
174.                 {
175.                     return default;
176.                 }
177.  
178.                 var componentType = entityManager.GetArchetypeChunkComponentType<T>(false);
179.  
180.                 var chunks = this.query.CreateArchetypeChunkArray(Allocator.TempJob);
181.  
182.                 int startIndex = 0;
183.  
184.                 var handles = new NativeArray<JobHandle>(this.queues.Count, Allocator.TempJob);
185.  
186.                 // Create a job for each queue. This is designed so that these jobs can run simultaneously.
187.                 for (var index = 0; index < this.queues.Count; index++)
188.                 {
189.                     var queue = this.queues[index];
190.                     var job = new SetComponentDataJob
191.                     {
192.                         Chunks = chunks,
193.                         Queue = queue,
194.                         StartIndex = startIndex,
195.                         ComponentType = componentType,
196.                     };
197.  
198.                     startIndex += queue.Count;
199.  
200.                     handles[index] = job.Schedule();
201.                 }
202.  
203.                 var handle = JobHandle.CombineDependencies(handles);
204.                 handles.Dispose();
205.  
206.                 // Deallocate the chunk array
207.                 handle = new DeallocateJob<ArchetypeChunk>(chunks).Schedule(handle);
208.  
209.                 return handle;
210.             }
211.  
212.             private void DestroyEntities(EntityManager entityManager)
213.             {
214.                 Profiler.BeginSample("DestroyEntity");
215.  
216.                 entityManager.DestroyEntity(this.query);
217.  
218.                 Profiler.EndSample();
219.             }
220.  
221.             private void CreateEntities(EntityManager entityManager)
222.             {
223.                 var count = this.GetCount();
224.  
225.                 if (count == 0)
226.                 {
227.                     return;
228.                 }
229.  
230.                 Profiler.BeginSample("CreateEntity");
231.  
232.                 // Felt like Temp should be the allocator but gets disposed for some reason.
233.                 using (var entities =
234.                     new NativeArray<Entity>(count, Allocator.TempJob, NativeArrayOptions.UninitializedMemory))
235.                 {
236.                     entityManager.CreateEntity(this.archetype, entities);
237.                 }
238.  
239.                 Profiler.EndSample();
240.             }
241.  
242.             private int GetCount()
243.             {
244.                 var sum = 0;
245.                 foreach (var i in this.queues)
246.                 {
247.                     sum += i.Count;
248.                 }
249.  
250.                 return sum;
251.             }
252.  
253.             [BurstCompile]
254.             private struct SetComponentDataJob : IJob
255.             {
256.                 public int StartIndex;
257.  
258.                 public NativeQueue<T> Queue;
259.  
260.                 [ReadOnly]
261.                 public NativeArray<ArchetypeChunk> Chunks;
262.  
263.                 [NativeDisableContainerSafetyRestriction]
264.                 public ArchetypeChunkComponentType<T> ComponentType;
265.  
266.                 /// <inheritdoc />
267.                 public void Execute()
268.                 {
269.                     this.GetIndexes(out var chunkIndex, out var entityIndex);
270.  
271.                     for (; chunkIndex < this.Chunks.Length; chunkIndex++)
272.                     {
273.                         var chunk = this.Chunks[chunkIndex];
274.  
275.                         var components = chunk.GetNativeArray(this.ComponentType);
276.  
277.                         while (this.Queue.TryDequeue(out var item) && entityIndex < components.Length)
278.                         {
279.                             components[entityIndex++] = item;
280.                         }
281.  
282.                         if (this.Queue.Count == 0)
283.                         {
284.                             return;
285.                         }
286.  
287.                         entityIndex = entityIndex < components.Length ? entityIndex : 0;
288.                     }
289.                 }
290.  
291.                 private void GetIndexes(out int chunkIndex, out int entityIndex)
292.                 {
293.                     var sum = 0;
294.  
295.                     for (chunkIndex = 0; chunkIndex < this.Chunks.Length; chunkIndex++)
296.                     {
297.                         var chunk = this.Chunks[chunkIndex];
298.  
299.                         var length = chunk.Count;
300.  
301.                         if (sum + length < this.StartIndex)
302.                         {
303.                             sum += length;
304.                             continue;
305.                         }
306.  
307.                         entityIndex = this.StartIndex - sum;
308.                         return;
309.                     }
310.  
311.                     // TODO probably should remove this
312.                     throw new ArgumentOutOfRangeException(nameof(this.StartIndex));
313.                 }
314.             }
315.         }
316.     }
317. }

 

Just wanted to say thank you. I used it a bit, and loving it. Completely changed the way I do things now.

Maybe a nice addition would be to have a generic AddComponent, DestroyComponent Queue like:

addComponentQueue.Enqueue(entity, new MyComponent()
{
MyField = Field
});

 

So bit of an update. As I've developed over time and thought about how I would like to design applications my requirements have changed a bit over time.

I've been working on a completely new version of the event system to solve some issues with development, make it more universal and just better all around.

So here are a few features of V2

• Events are no longer entities
• Events to be consumed same frame, no 1 frame delay
  
• Uses NativeStreams
  
• Supporting any number of producers and consumers

Sample Producer

Code (CSharp):

1.     protected override JobHandle OnUpdate(JobHandle handle)
2.     {
3.         const int count = 1000;
4.  
5.         handle = new ProduceJob
6.             {
7.                 Events = this.eventSystem.CreateEventWriter<TestEvent>(count),
8.             }
9.             .Schedule(count, 64, handle);
10.  
11.         this.eventSystem.AddJobHandleForProducer<TestEvent>(handle);
12.  
13.         return handle;
14.     }

Sample Consumer

Code (CSharp):

1. protected override JobHandle OnUpdate(JobHandle handle)
2.     {
3.         handle = this.eventSystem.GetEventReaders<TestEvent>(handle, out var readers);
4.  
5.         for (var index = 0; index < readers.Count; index++)
6.         {
7.             var (reader, count) = readers[index];
8.  
9.             handle = new ConsumerJob
10.                 {
11.                     Stream = reader,
12.                 }
13.                 .Schedule(count, 64, handle);
14.         }
15.  
16.         this.eventSystem.AddJobHandleForConsumer(handle);
17.  
18.         return handle;
19.     }
20.  

• Fully threaded. Previously queues limited threading when reading.
• No sync points, no ComponentSystems, no Complete(), no EntityCommandBuffer
• Dependencies managed for you.

As shown here there are 0 sync points allowing jobs to cross frame borders maximizing cpu utilization.

• Even better performance, as there are no sync points, no need to copy data to entities and it utilizes the extremely fast NativeStream performance is pretty ridiculous.

This is the same frame zoomed in. It has 2 separate producers creating 1,000,000 events each (2mill total) and 3 consumers consuming the jobs in just a few ms.

• Support different Simulation and Presentation update rates

This was very important and why originally started revisiting this. If and when entities moves to a separate tick rate for Simulation and Presentation groups I wanted a way that an event could fire from the simulation group, be consumed just once in simulation and also be consumed just once in presentation and this was not possible with entities.

• Primary drawback at the moment is the use of NativeStream makes IJobForEach a bit unviable for large entity sets and IJobChunk needs to be used instead.

Coming soon~

 

That's sound nice but I used a lot your library, and sometime I didn't know the number of events I will raised during a job, is the "count" parameter mandatory ?

 

Count in that example isn't the number of events. It's the number of the foreachIndex in a the native stream.

This value will depend on how you schedule your job.
For a IJobParallelFor it's the array length.
For a IJobChunk it's the chunk count (query.CalculateChunkCount)
etc

i.e. you can have a count of 1000 and push no events or you can push 1million events.

So the job for that producer example looks like this.

Code (CSharp):

1.     [BurstCompile]
2.     private struct ProduceJob : IJobParallelFor
3.     {
4.         public NativeStream.Writer Events;
5.  
6.         public void Execute(int index)
7.         {
8.             this.Events.BeginForEachIndex(index);
9.             for (var i = 0; i < 1000; i++)
10.             {
11.                 this.Events.Write(new TestEvent { Value = 1000 + i });
12.             }
13.  
14.             this.Events.EndForEachIndex();
15.         }
16.     }

If you wanted to use an IJobChunk for example, could do it like this

Code (CSharp):

1.     [BurstCompile]
2.     private struct ChunkProduceJob : IJobChunk
3.     {
4.         public NativeStream.Writer Events;
5.  
6.         public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
7.         {
8.             this.Events.BeginForEachIndex(chunkIndex);
9.             for (var i = 0; i < 1000; i++)
10.             {
11.                 this.Events.Write(new TestEvent { Value = 1000 + i });
12.             }
13.  
14.             this.Events.EndForEachIndex();
15.         }
16.     }

And you could set it up like this

Code (CSharp):

1.         var foreachCount = this.query.CalculateChunkCount();
2.  
3.         handle = new ChunkProduceJob
4.             {
5.                 Events = this.eventSystem.CreateEventWriter<TestEvent>(foreachCount),
6.             }
7.             .Schedule(this.query, handle);
8.  
9.         this.eventSystem.AddJobHandleForProducer<TestEvent>(handle);

You can do something similar with IJobForEachWithEntity but you need to use an index per entity so it's a lot less efficient and inadvisable for 'large' entity sets (will be fine for 'smaller' sets.)

 

Hum ok. I'm mostly using IJobForEach jobs currently. So maybe staying with your old system might be better in my case.

 

@tertle I'm curious about your new event system.

How are you managing your NativeStreams? Are you storing them in a Dictionary give the fact that you can't nest streams dynamically? Also, are you creating a new stream on every CreateEventWriter call?

 

Posted a write up here: https://forum.unity.com/threads/event-system.779711/
includes full source code.

To answer your questions.

Yes they are stored in a dictionary with a custom compare to avoid garbage.

Code (CSharp):

1. private readonly Dictionary<NativeStream, StreamHandles> streams = new Dictionary<NativeStream, StreamHandles>(new NativeStreamCompare());

And yes a new stream is created every CreateEventWriter call. They don't seem reusable.