Comparing different approaches for Events in DOTS

Github repo: https://github.com/PhilSA/DOTSEventsStressTest

I decided I wanted to try all kinds of different approaches to DOTS events, and profile them to better understand the pros & cons of each. I'm dumping my findings here if it can be useful to anyone else.

The Test
I've decided to use "Damage Events" as a common use case across all of my approaches. We create X health entities, and we create Y damager entities for each health entity. Each damager creates a "damage event" that will apply damage to a target health entity. So this means a single health entity can receive multiple individual damage events in the same frame.

Open the "StressTest" scene in the project. In the subscene, you will find a "StressTest" object where you can configure how many Health entities to spawn, how many damagers per health, and what kind of event strategy to use


For each approach, we test with 500,000 health entities.
And for each approach, we look at the profiler when each health receive a damage event every frame, VS when no damage events are sent (this is so we can compare the constant overhead of each approach when no changes are happening)
____________________________________________________

When to use events
Generally-speaking, DOTS events can be a good idea in these situations:

• When you need parallel jobs to call for an action that cannot be processed in parallel
• When you need multiple parallel modifications on a same target entity to be grouped by entity, so they can then be processed in parallel
• When a certain action has to be processed at a specific point in the frame, but you want to be able to schedule it from a system that doesn't update at that point
• When you want to hide the ComponentDataFromEntity access requirements of an action from whoever calls that action
• When an action called from a job can have multiple "consumers" or "listeners", each implemented with their own different job and with their own component data access. Much like an equivalent of a delegate or a System.Action
• When an action with unique parameters on a target entity must be callable from certain jobs, but its execution logic depends on the archetype of the targeted entity (Approach A, B, C)
• When an action with unique parameters on a target entity must be callable from certain jobs, but you'd prefer the processing of that action to use chunk iteration, so that component data access on the target entity will be more efficient (Approach A. B. C)
• When Bursted code must request actions that involve managed (or main thread) code

A more detailed use case analysis is available here:
https://forum.unity.com/threads/com...or-events-in-dots.1267775/page-2#post-8055923

____________________________________________________

Note: The measured times here are not the total frame time; but just the time related to all operations done by the approach (job, main thread, etc...)

Approach A (13.4ms)

Spoiler

System Code

• A parallel job makes each damager write a DamageEvent to a nativestream
• a singlethread job writes each DamageEvent from the stream to a DynamicBuffer on its target entity
• A parallel job polls DamageEvent buffers for event processing

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 13.4ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: 0.07ms ]

Approach B (12.9ms)

Spoiler

System Code

• A singlethread job makes each damager write to damageEvent buffers on their target entity
• A parallel job polls DamageEvent buffers for event processing

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 12.9ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: 0.07ms ]

Note: this is faster than Approach A in this simplistic test, but Approach A has the advantage of allowing multiple jobs to create damage events in parallel (so A would allow all damage-creating jobs in your project to execute in parallel, while B would constrain them to execute one after the other on a single thread)

Approach C (16.0ms)

Spoiler

System Code

• A parallel job writes to damageEvent buffers on their target entities using ECB.AppendToBuffer
• A parallel job polls DamageEvent buffers for event processing

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 16.0ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: 0.07ms ]

Approach D (3.7ms)

Spoiler

System Code

• A parallel job makes each damager write a DamageEvent to a nativestream
  
• A singlethread job reads events from stream and directly applies the damage to health component on the target entity

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 3.7ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: ~0.01ms ]

Approach E (1.9ms)

Spoiler

System Code

• A parallel job makes each damager write a DamageEvent to a nativestream
  
• A parallel job reads events from stream and directly applies the damage to health component on the target entity, using "Interlocked.Add"

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 1.9ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: ~0.01ms ]

Note: this approach is pretty limited as to what you can do with it. If your event needs to do more than add to a value, I don't think this would be viable

Approach F (67ms)

Spoiler

System Code

• A parallel job makes each damager create an entity representing a Damage Event
• A singlethread job iterates on all damage event entities, and applies them directly to the health component on their target entity

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 67ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: ~0.01ms ]

Rejected: Approach G

Spoiler

Rejected for being pointless. I was assuming I couldn't read from NativeStream multiple times, so I was offloading the stream to a NativeList. This is entirely pointless, because we in fact can read a NativeStream multiple times

Approach H (7.2ms)

Spoiler

System Code

• A parallel job makes each damager write a DamageEvent to a nativestream
• A singlethread job takes events from nativestream and writes them to a NativeMultiHashMap where the Key is the target entity
• A singlethread job iterates events from the NativeMultiHashMap, and applies them

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 7.2ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: ~0.01ms ]

Approach I (5.6ms)

Spoiler

System Code

• A parallel job makes each damager write a DamageEvent to a nativestream
• A singlethread job writes events to hashmap
• A parallel job iterates events from the NativeMultiHashMap (same Entities always end up on same thread), and applies them

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 5.6ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: ~0.01ms ]

NOTE: I've tried an approach where we ensure hashmap capacity and then write events to hashmap in parallel, but this has poorer performance then simply writing in a single-thread job. Even without considering the "ensure capacity" job

Bonus Approach : Direct single-threaded modification (3.1ms)

Spoiler

System Code

• a single-thread job simply iterates on all damagers, gets target health from entity, and modifies it directly

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 3.1ms ]
Profiler when no damage events are happening [ Snapshot ] [ Avg: 0.7ms ]

Note: this approach is here to serve as a reference point, but the use case is too simple for this to be a fair comparison with other event systems. Event systems often exist either in order to allow reactive/modular behaviour or to allow parallel processing of heavy logic, but there is none of that in this test

Bonus Approach : Typical Monobehaviour Update() (83ms)

Spoiler

Damager Code
Health Code

• A TestHealth monobehaviour on a GameObject has a health value
• For each TestHealth, we create a TestDamager monobehaviour on a GameObject, with the TestHealth as target
• In TestDamager Update(), it subtracts its damage from TestHealth

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 83ms ]

Bonus Approach : Manual Monobehaviour Update() (3.5ms)

Spoiler

ManualUpdateManager
Damager Code
Health Code

• A TestHealth monobehaviour on a GameObject has a health value
• For each TestHealth, we create a TestDamager monobehaviour on a GameObject, with the TestHealth as target
• Each TestDamager registers itself in a List in a central "ManualUpdateManager". The ManualUpdateManager goes through the damagers list and update each one of them every frame

Profiler when each health receives 1 damage event every frame [ Snapshot ] [ Avg: 3.5ms ]

____________________________________________________

Conclusions

Performance-wise: we have a very clear loser: Events-as-entities (Approach F). But the winner is not so clear:

• Approach E has the shortest execution time of all, but it is very limited in what it can do. If it has to do more than just adding numbers to a value, we can't make this work. This approach is rather specialized for this particular test, and depending on how we use Interlocked, it could be non-deterministic (not suitable for network prediction)
• Approach D is in second place. However, in terms of total time across all threads, the Approach D takes 15.8ms, while Approach E takes 33.2ms. For this reason it could be argued that Approach D is better for performance, because it allows other stuff to be done at the same time on other threads. If you think you could be running any other job in your game at the same time as some of your event jobs (or multiple different event jobs), then you're likely to get better performance out of Approach D.
• Apporach I takes a bit longer than Approach D, but it supports parallel processing of events. So there might be situations where Approach I would be best for performance. If event execution is really heavy, the parallelism could give this approach the advantage.
• Approach A (or B) take much longer, but they have a few advantages that could also make them faster than Approach D in certain scenarios. Like Approach I, they also have the advantage of parallel processing for heavier events. Moreover, they can potentially benefit from archetype filtering strategies, and also and they have the advantage of using chunk iteration for reading/writing, instead of ComponentDataFromEntity, so their component access is a lot faster. However, while B takes less time than A in this test, A would perform better if you want the jobs that create those events to be parallel

Overall, when not just taking into account performance, and when looking at all this from the perspective of "what strategies would be good all-purpose defaults to keep in mind for day-to-day gameplay programming tasks", I'd sum it up like this:

I think what this table shows is that regardless of their individual performance, none of the solutions are the best for every scenario. They each have situations where they're better than others. There is more to performance than the simple execution time of each approach in a vacuum; how the strategy fits in with the rest of your project is equally important

NOTE: C can be good if you need to have your events in buffers, but you want to avoid the constant cost of polling for changes. A and B necessarily require every possible target to already have a DynamicBuffer for events, but C doesn't. So C can add a buffer, and the buffer can then be removed after events are processed, meaning it doesn't have to be polled for changes. On the other hand, this incurs a much heavier cost when events do happen, and the performance results of this test don't demonstrate that additional cost. It would probably raise the cost to something similar to approach F

 

Thanks for putting this together and sharing!

I noticed that in approach F, you are creating an entity and then adding the damage event component to it. I'm not sure if ECB playback is smart enough to avoid the archetype change here, but I was wondering if switching this to ECB.Instantiate() would perform better.

Code (CSharp):

1. // In OnCreate
2. Entity eventPrefab = EntityManager.CreateEntity();
3. EntityManager.AddComponent<DamageEventComp>(eventPrefab);
4.  
5. // In parallel job
6. Entity damageEventEntity = createEventsECB.Instantiate(entityInQueryIndex, eventPrefab);
7. createEventsECB.SetComponent(entityInQueryIndex, damageEventEntity, new DamageEventComp { Source = entity, Target = damager.Target, Damage = damager.Damage });

Btw, I'm not trying to advocate for approach F over the others. I was surprised to see how much worse it was and curious if it could be improved at all.

 

I think in approach E, you should not multiply the index with sizeof(Health). That would only make sense if the pointer is a byte*. Because the pointer is an int* you are moving 4 bytes for each incrementation.

 

My event system was build on the idea of DE being superior for raw speed https://forum.unity.com/threads/event-system.779711/

Under the hood it's basically just a custom NativeStream (that doesn't need indexing or foreach pre-calculated), gives you automatic safety between systems, allows multiple consumer/producers and custom jobs to avoid boilerplate of iterating NativeStreams.

It's been used in at least a few shipped games so I'm pretty confident in it's reliability (the 1.x branch).

-edit-
i should point out because of the thread safe nature of the NativeEvent container in my library it's not as fast as using a NativeStream directly (might even be 2x slower writing). I decided the advantage of not having to setup indexes so it works much better with Entities.ForEach outweighed the performance benefits.

-edit2-
i actually loaded my event system into your benchmarks just to compare. wasn't exactly happy with how the write performance had dropped so i greatly improved it

Approach D


EventSystem 2.1 Before



EventSystem 2.1 After


I nearly got my custom EventStream back to the performance of NativeStream while not having to setup indices/foreach. Very happy with that

Attached the source. Based off the 2.1 branch of my event system (I haven't pushed the performance improvement yet.)

Writing job

Code (CSharp):

1. [BurstCompile]
2. public struct DamagersWriteToStreamJob : IJobEntityBatch
3. {
4.     [ReadOnly] public EntityTypeHandle EntityType;
5.     [ReadOnly] public ComponentTypeHandle<Damager> DamagerType;
6.  
7.     public NativeEventStream.Writer StreamDamageEvents;
8.  
9.     public void Execute(ArchetypeChunk batchInChunk, int batchIndex)
10.     {
11.         var chunkEntity = batchInChunk.GetNativeArray(EntityType);
12.         var chunkDamager = batchInChunk.GetNativeArray(DamagerType);
13.  
14.         for (int i = 0; i < batchInChunk.Count; i++)
15.         {
16.             var entity = chunkEntity[i];
17.             var damager = chunkDamager[i];
18.             StreamDamageEvents.Write(new StreamDamageEvent { Target = damager.Target, DamageEvent = new DamageEvent { Source = entity, Value = damager.Damage } });
19.         }
20.     }
21. }

Reading job

Code (CSharp):

1. [BurstCompile]
2. public struct SingleApplyStreamEventsToEntitiesJob : IJobEvent<StreamDamageEvent>
3. {
4.     public ComponentDataFromEntity<Health> HealthFromEntity;
5.  
6.     public void Execute(StreamDamageEvent damageEvent)
7.     {
8.         if (HealthFromEntity.HasComponent(damageEvent.Target))
9.         {
10.             Health health = HealthFromEntity[damageEvent.Target];
11.             health.Value -= damageEvent.DamageEvent.Value;
12.             HealthFromEntity[damageEvent.Target] = health;
13.         }
14.     }
15. }

 

Pushed 2 new approaches : G and H

This changes my conclusions. These new approaches give us the same "multiple consumer" advantages as Approach A, but with a performance cost that's pretty close to Approach D. They don't do anything clever really; just dump the NativeStream events into either a List or a MultiHashMap, so we can have multiple consumers processing them

I really wanted to find an efficient way to process events in parallel with Approach H, but I haven't found a good way to do it yet. We need each thread to process unique keys (entities), but getting those unique keys is way too expensive

I'm gonna take a little break but I'll come back for the fix suggestions above

Also @tertle those performance results are pretty convincing indeed. Not a big price to pay for better ease of use

 

H is a decent approach and when I do actually use events it's what I nearly always use so I built the functionality into the event system however, I haven't looked at your most recent benchmark, but the question I have is

> A singlethread job takes events from nativestream and writes them to a NativeMultiHashMap where the Key is the target entity
Why can't you do this in parallel? Native stream is great for parallel reading and NativeMultiHashMap can be written to in parallel.

> A singlethread job iterates events from the NativeMultiHashMap, and applies them
Once your in a NativeMultiHashMap again why can't you iterate in parallel? I assume each key is the target Entity so could be iterated with each key being a separate thread.


You could just insert a quick single threaded job before populating your hashmap to ensure capacity (using NativeStream[.Reader].Count()), then schedule a parallel read job to populate your hashmap, then schedule a parallel job to read your hashmap and write back to your entities.

 

We'd need to call NativeMultiHashMap.GetUniqueKeys() in order to make sure each thread is responsible for a unique range of entities, but in this test, calling GetUniqueKeys() takes about 40ms. So it seems to be too heavy

Maybe I'm missing an obvious solution though

 

This isn't actually true if you iterate per bucket. All keys that share the same hash will end up on the same thread.

Code (CSharp):

1.                     var bucketData = fullData.HashMap.GetUnsafeBucketData();
2.  
3.                     var buckets = (int*)bucketData.buckets;
4.                     var nextPtrs = (int*)bucketData.next;
5.                     var keys = bucketData.keys;
6.                     var values = bucketData.values;
7.  
8.                     for (int i = begin; i < end; i++)
9.                     {
10.                         int entryIndex = buckets[i];
11.  
12.                         while (entryIndex != -1)
13.                         {
14.                             var key = UnsafeUtility.ReadArrayElement<TKey>(keys, entryIndex);
15.                             var value = UnsafeUtility.ReadArrayElement<TValue>(values, entryIndex);
16.  
17.                             fullData.JobData.ExecuteNext(key, value);
18.  
19.                             entryIndex = nextPtrs[entryIndex];
20.                         }
21.                     }

-edit-

The old IJobNativeMultiHashMapVisitKeyValue that were removed do this (the code is copied from that.)

 

Awesome, I'll give this a try later

 

I tried with Instantiate, but surprisingly it was slightly more expensive. But I replaced it with ecb.Create(myArchetype), and that reduced the cost a lot.
This got us from 177ms to 115ms

Then, instead of destroying events with ecb.Destroy(), I destroyed them with EntityManager.DestroyEntity(eventsQuery).
This got us from 115ms to 65ms

I wonder if fully bursted ISystems will eventually give us a huge perf boost to EntityManager operations (and by extension; to ECB operations as well)

You're totally right, I've applied the fix

 

Based off my quick test I don't think writing to the hashmap like this is worth it.

I just did a bunch of testing and you really need to batch add to a hashmap (which isn't supported by default but can be added as an extension) to really start getting noticeable benefits.

 

To put things into perspective, I've created an similar test but in OOP/Monobehaviours:
https://github.com/PhilSA/DOTSEventsStressTest/blob/master/Assets/StressTest/OOPTest/TestDamager.cs

This test takes 83ms per frame in a build. It's interesting to think that the events-as-entities approach actually performs better (65ms), even though it has a reputation for being too costly. Maybe it's a sign we shouldn't be too worried about using events-as-entities every now and then

 

Totally agree. For any smaller count (1-100s/frame) entity events are totally fine and often make a lot of sense.

This had basically been the case since ecb playback was burst compiled, since most entity events would be created this way.

The only real issue is they are generally 1 frame delayed, unless you insert another sync point which isn't great. Often though this isn't an issue and I use these types of events all the time for high level state changes.

 

So besides the fact that I would have expected A and B to be flipped, your performance measurements are about what I expected. Your conclusions however were a bit unexpected.

About Approach E:
1) Health is a float but you are treating it as an int in this test.
2) I have no idea why you are even building the NativeStream. It serves no purpose here.
3) I suspect you might be write-bound on the atomics. Your job isn't doing much other than writing atomically, which means all your CPU cores are going to be trying to flush their store buffers at the same time and reducing the benefit of parallelism. If you did more work in the job, I would expect the job to scale a lot better. A way to test this is to try making that job run single-threaded. If frame times don't increase with roughly the number of cores, then you are having store buffer contention (or perhaps false sharing, but seems unlikely). If they do increase proportionally, then atomics are just stupidly slow on your hardware for some reason.
4) It might seem really specialized because it is. It is a specific solution to a specific problem which offers the most performance. With that said, there are a few tweaks that can be added to gain back a good amount of flexibility.

About Approach F:

Nearly all EM operations run in Burst, though you pay for the Mono-Native interop. But Burst ISystems don't pay that cost and do iterations a lot faster. ECB playback also runs in Burst for most operations, and since that is a bulk operation, you probably won't see much of a performance benefit from a Burst ISystem other than system overhead.

About Approach G:

Approach D allows that too. You can read a NativeStream more than once, and from more than one job at once.

Anyways, if I were to need events processed individually in a separate job from generation, I would probably try to find a way to bin each event by its target's chunk and then have an IJobEntityBatch apply the events to the entities.

If that is deterministic, that is very impressive! Otherwise, I actually have NativeStream beat in my block list implementation. But I am cheating a little and require all events written to it to be a fixed size. NativeStream gives up a fair amount of performance for the flexibility of variable-sized data. If you don't always need that and care about performance, then have a version that does away with that logic.

 

I've treated the float as an int just to not have to create a whole new Health component just for that approach

EDIT: As for the NativeStream, the idea would be to hide that complexity from whoever calls that event. I'm assuming there would be all kinds of things creating damage events in a game, and we'd want to avoid having to remember to do all that & passing all that data to our jobs

Oh... I guess I got confused. Well that's good news! So approach G would be pretty pointless then

 

Awesome report!

Just a quick note: unless events are happening every frame and at excessive numbers, I prefer to look at the „no events“ stats. And in doing that I find the „create event entities“ approach absolutely viable when there‘s only a few dozen or hundreds of events in a frame and specifically when most of the frames see little to no events raised.

 

Hmmm why is the event entitiy approach applying in a single thread? Can it not run in parallel?

 

Nah. In my 1.X branch i had a deterministic mode but I've completely scrapped it in my 2.X branch. I just never found a use for it, especially since I pretty much use it exclusively for debugging tools. I don't really write many events anymore in actual game play logic.

Unfortunately I find variable sizing one of the best features and I have quite a few special use cases for it.

For example my drawing very much depends on variable size



Another more specialized case is I have a WriteLarge method that can write any size data (NativeStream is limited to a max size of 4088) and is Orders of magnitude faster than writing individual elements. This is useful if I can batch a write together, say an entire chunk etc.

 

Awesome work.

Hi, is there a reason why you are not using the Unity Performance testing framework? Looked at the packages and it didn't look like it was there.
It would give a lot better view what is actually happening, like min max and average.
You can also do warm up counts etc. You can example do warm up for ten frames and then record then frames.
Then snapshot of the profiler etc

 

Since those events modify values on another entity, and 2 events can affect the same entity, you could have 2 events from different threads trying to modify the same entity

 

I actually never tried it, but sounds useful

 

Totally recommend it.
Especially since it shows the minimum, maximum and averages.
https://docs.unity3d.com/Packages/com.unity.test-framework.performance@1.0/manual/index.html

Example code
.

Code (CSharp):

1.         [Test, Performance]
2.         public void SpeedOfAllocNativeMultiHashMap()
3.         {
4.             Measure.Method(() =>
5.                 {
6.                     NativeMultiHashMap<int3, Target> targets =
7.                         new NativeMultiHashMap<int3, Target>(TargetingSystemSettings.MaxTargets, Allocator.TempJob);
8.                     targets.Dispose();
9.                 })
10.                 .IterationsPerMeasurement(1).MeasurementCount(100000).WarmupCount(8).Run();
11.         }

 

coming back to this:

I'm reading the deprecated implementation of IJobNativeMultiHashMapMergedSharedKeyIndices, trying to understand all of this better. I haven't tried creating custom job types before, so I'm learning about that.

So, would this approach necessarily involve creating a new job type to iterate on each different kind of multiHashMap, or is it something that should be doable inside a regular IJobParallelFor? Just trying to understand why that job type was deprecated. Is it because it was restricted to a MultiHashMap<int,int>, and there was no way of making it generic?

 

Good question, maybe.
Could also be something that was needed by someone at some time and made in a hurry and now there is a better way.
Or maybe they saw that they don't want it to be used that way.

You can use Native multi hash map inside almost any job.

Code (CSharp):

1. do{      
2. ......
3. } while (targetsAndindices.TryGetNextValue(out target, ref iterator));

 

There were 3 jobs
IJobNativeMultiHashMapMergedSharedKeyIndices
was very specialized but there was also but there was also

IJobNativeMultiHashMapVisitKeyValue<TKey, TValue>
IJobNativeHashMapVisitKeyValue<TKey, TValue>

I've attached my copies of them. Not sure what I've changed since they were removed (at the minimum it's styled as my library).

 

Thanks so much for this, it worked right away

Updated top post with Approach I. But I haven't pushed your job code to my repo

Interestingly enough, it doesn't perform as well as Approach D, and it has a bit of a high baseline cost when no events are created. I've re-profiled both approaches and double-checked that it wasn't because of JobsDebugger or SafetyChecks options and whatnot.

I suppose it would gain the upper hand if the event execution logic was heavier though

The job is scheduled like this:

Code (CSharp):

1. Dependency = new ParallelPollDamageEventHashMapJob
2. {
3.     HealthFromEntity = GetComponentDataFromEntity<Health>(false),
4. }.ScheduleParallel(DamageEventsMap, 1000, Dependency);

And the job looks like this:

Code (CSharp):

1. [BurstCompile]
2. public struct ParallelPollDamageEventHashMapJob : BovineLabs.Core.Jobs.IJobNativeMultiHashMapVisitKeyValue<Entity, DamageEvent>
3. {
4.     [NativeDisableParallelForRestriction]
5.     public ComponentDataFromEntity<Health> HealthFromEntity;
6.  
7.     public void ExecuteNext(Entity targetEntity, DamageEvent damageEvent)
8.     {
9.         if (HealthFromEntity.HasComponent(targetEntity))
10.         {
11.             Health health = HealthFromEntity[targetEntity];
12.             health.Value -= damageEvent.Value;
13.             HealthFromEntity[targetEntity] = health;
14.         }
15.     }
16. }

 

That's because you aren't writing to the hashmap in parallel. You want a single-threaded job to count the elements in the NativeStream and set the capacity of the hashmap. Then you want a parallel job that actually writes to the hashmap.

 

That makes sense,

But comparing the 2 profiler snapshots: ApproachD - ApproachI

It looks like even the "event execution" part isn't a whole lot faster in the parallel hashmap approach. It's 2.06ms for I, versus 2.81ms for D. Considering that I occupies all of my 24 threads while D occupies only one, I'm not sure it's worth the small time saving

But if the event execution were much heavier, I'm sure 'I' would outperform 'D'

 

That is exactly what I do.
https://github.com/WAYN-Games/MGM-Ability/blob/master/Runtime/Systems/AbilityEffectTriggerSystems.cs
https://github.com/WAYN-Games/MGM-A...ntime/Systems/AbilityEffectConsumerSystems.cs

The stream is created by the trigger only if there are any eligible entity to consume the event (if no entity has health there is no point in handeling a damage event) and if no stream is created the consumer does not run (if no damage event were sent there is no need to check all the entities that have health).

The nativehash map is persistant and only grow when necessary to avoid as much allocation as possible.
And instead of iteration over the hash map in a job I iterate over all entities that are able to consume the type of hevent handled by the system and early out if the entity has no values in the event map.

I recently added the ability to have multiple trigger system for one consumer (basically allowing a list of native stream) in an unpublished branch.

 

@PhilSA, @tertle Quick question, do your tests/implementation have more than one chunk when writing to the stream ?
I look at your implementation and I do the same thing has you but as soon as the foreach count of the native stream is above 1 (more than one chunk) I end up with this error :

Code (CSharp):

1. ArgumentException: Index 2 is out of restricted IJobParallelFor range [0...-1] in NativeStream.

I already reported the bug a while back (entities 0.16) and have a work arround (Before using BeginForEachIndex(batchIndex) use PatchMinMaxRange(batchIndex).) but I want to make sure I'm not doing anything stupid...

 

Looks good. I'll give it a test at some point

 

Well I was testing with my own EventStream which is thread safe so I don't have that issue.
As for NativeStream yes this is often a flaw with it, you need to turn off parallel safety a lot of the time.

 

I'm doing a little related test in a separate project and I'm not sure why I'm getting errors here:

I'm creating a stream, parallel writing to it in IJobParallelFor, and then parallel reading it in IJobParallelFor. I make sure the Dependency is passed along at every step:

Code (CSharp):

1.  
2. public partial class TestStreamEventsWriteReadSystem : SystemBase
3. {
4.     protected override void OnUpdate()
5.     {
6.         NativeStream testStream = new NativeStream(16, Allocator.TempJob);
7.  
8.         // Parallel write
9.         Dependency = new TestStreamWriteParallelJob
10.         {
11.             WriteCount = 10000,
12.             WriterStream = testStream.AsWriter(),
13.         }.Schedule(testStream.ForEachCount, 1, Dependency);
14.  
15.         // Parallel read
16.         Dependency = new TestStreamReadParallelJob
17.         {
18.             ReaderStream = testStream.AsReader(),
19.         }.Schedule(testStream.ForEachCount, 1, Dependency);
20.  
21.         Dependency = testStream.Dispose(Dependency);
22.     }
23. }
24.  
25. [BurstCompile]
26. public struct TestStreamWriteParallelJob : IJobParallelFor
27. {
28.     public int WriteCount;
29.     public NativeStream.Writer WriterStream;
30.     public void Execute(int index)
31.     {
32.         WriterStream.BeginForEachIndex(index);
33.         for (int i = 0; i < WriteCount; i++)
34.         {
35.             WriterStream.Write(default(DamageEvent));
36.         }
37.         WriterStream.EndForEachIndex();
38.     }
39. }
40.  
41. [BurstCompile]
42. public struct TestStreamReadParallelJob : IJobParallelFor
43. {
44.     public NativeStream.Reader ReaderStream;
45.     public void Execute(int index)
46.     {
47.         ReaderStream.BeginForEachIndex(index);
48.         while (ReaderStream.RemainingItemCount > 0)
49.         {
50.             ReaderStream.Read<DamageEvent>();
51.         }
52.         ReaderStream.EndForEachIndex();
53.     }
54. }
55.  

And during play, I get

"TestStreamEventsSystem.cs:44" is line 16 in the code sample. I thought I was doing more or less the same thing in one of my events tests here: ParallelWriteToStream_ParallelApplyEventsToEntities , so I don't really understand. Am I missing something obvious?

 

Your issue is ForEachCount, as it's not safe to read it after write schedule

 

ah, yes thank you. That did the trick

 

Great thread and work Phil!

For my project I'm using Interlocked on health values.
And I know this feels like cheating for this kind of test but when using Interlocked, you don't actually have to create an event. Just apply it where it occurs. The cost of stalling the threads is minimal.

 

The main reason why I'm not totally convinced by that approach is the "absolute job completion time" VS "total time used across all threads" ratio. It's the same issue I had with the parallel hashmap approach (Approach I)

In my test, the "Interlocked" approach fully occupies 24 threads for 2.9ms, while the "single-thread execute events from stream" approach occupies (mostly) 1 thread for 4.3ms

My thinking is that there are pretty good chances that in a real game you'll have plenty of other jobs that could execute at the same time as the single-threaded stream approach, while the interlocked approach would not allow that. It doesn't seem like the ~30% time gain (or maybe ~50% if we skip the stream as you explained) is entirely worth it when you can find other jobs to run at the same time

It's a bit of an ideal example, but: imagine you have 10 different event systems, all creating 500k events per frame like in this test, and all writing to different components. The "Interlocked" approach would take about (10*2.9) = 29ms, while the "SingleThreaded Stream" approach would take (10*0.8 + 3.5) = 11.5ms. Note: in that last time calculation here, "0.8" represents the parallel writing time, and "3.5" represents the single-thread read time that is executed for all 10 event types in parallel.

There will be cases where the Interlocked approach will be better, but I think the single-thread approach has higher chances of being the best in a typical project

 

Certainly true Phil. I get the feel this is still quite unexplored and lots of solutions are floating around. With enough time and collaborative work we could manage to get this down to 1-3 approaches, maybe even less.

The Interlocked solution is very specialized. With a job, big enough the stalling is minimal, in your test the job is very focused on just this specific task which certainly brings the performance down and stalling occurs more often.

I also need to stress the fact that I would prefer other solutions. There just has to be a few more mechanics that are tied to "getting damaged" - like an IsInCombat flag and Interlocked isn't that great any more when it comes to flexibility. There's no real way for a DamageEvent producer to know if this flag has already been applied to a target, so it could be set multiple times which wastes a lot of CPU cycles, even just for checks.

This can be avoided with my personal favourite when it comes to style and flexibility, the NativeMultiHashMap. But there are quite some complicated things involved to get it down to something really great.
Iteration time isn't that great. When many DamageEvents gets summed up for a certain target it's getting quite slow.
So cases where many entities are fighting a single boss monster is like a worst case scenario.
I shortly discussed this with Tertle and he mentioned allocating a NativeArray for a key. This is pretty much the fastest way you can hope for in terms of iteration time. But, knowing how much to allocate, etc... and not losing too much time in the process is also something to consider and overall I didn't get results that I was really satisfied with.

 

Also, regarding why I use a NativeStream for Approach E (Interlocked):

I think the most common reasons for using "events" in DOTS are:

• Hide complexity, data access requirements, ordering requirements, etc... from whoever performs the action
• Call actions in parallel, which must then have all or part of their execution done in single-thread
• Be able to have multiple systems "subscribe" to an action (split implementation in multiple places & make it extensible)
• Be able to have logic that "reacts" to each individual change (that can each have parameters), rather than only being able to react to the sum of all changes (change filtering)

DOTS events can often play the role of a "user-friendly public-facing API" for interacting with more complex systems. And the comparisons I'm doing in this thread are also done from that perspective, on top of performance

We could handle damage without events and with Interlocked directly, but then we'd have these limitations:

• Every single job that can create damage must get all the data access requirements of damage logic (StorageInfoFromEntity, HealthFromEntity, DefenseFromEntity, TeamFromEntity, etc... )
  • This makes it more tedious to code anything that can create damage, and makes the codebase more difficult to maintain (if damage data access changes, you need to change it everywhere)
• Every job that can create damage must have the correct system update order for damage processing (again, if ordering requirements change, you need to change it everywhere)
• All things that react to individual damages can only be implemented in one place (probably a static function)

So I think if I weren't using NativeStream for the Interlocked approach, I wouldn't really be comparing different "event system" approaches anymore

 

Thanks for sharing, I'm aware Bonus Approach seems not the way most people writing event/callback system right?
Using mono update will be slow as we know. And because you are trying your best to optimise in dots way, I think it worth to at least use some common techniques(Like pooling) in OO to do a fair comparison.

I wrote my test, and got only 14ms In Editor! Single thread on an AMD 5600 cpu.

Spoiler

Code (CSharp):

1. using System.Collections.Generic;
2. using UnityEngine;
3. using System.Diagnostics;
4.  
5. public class OOTest : MonoBehaviour
6. {
7.     public class Creature
8.     {
9.         public float health;
10.  
11.         public void OnAttack(AttackEvent e)
12.         {
13.             health -= e.damage;
14.         }
15.     }
16.  
17.     public struct AttackEvent
18.     {
19.         public Creature target;
20.         public float damage;
21.     }
22.  
23.     public int totalEntity = 500000;
24.     public bool runTest;
25.     Stopwatch stopwatch;
26.  
27.     Creature[] creatures;
28.     List<AttackEvent> attackEvents;
29.    
30.  
31.     void Start()
32.     {
33.         creatures = new Creature[totalEntity];
34.         for (int i = 0, length = creatures.Length; i < length; i++)
35.             creatures[i] = new Creature();
36.         attackEvents = new List<AttackEvent>(creatures.Length);
37.  
38.         stopwatch = new Stopwatch();
39.     }
40.  
41.     // Update is called once per frame
42.     void Update()
43.     {
44.         if (runTest)
45.         {
46.             runTest = false;
47.  
48.             stopwatch.Start();
49.             // create events
50.             for (int i = 0, length = creatures.Length; i < length; i++)
51.             {
52.                 attackEvents.Add(new AttackEvent()
53.                 {
54.                     target = creatures[i],
55.                     damage = 1
56.                 });
57.             }
58.             // inform events
59.             for (int i = 0, length = attackEvents.Count; i < length; i++)
60.             {
61.                 var attack = attackEvents[i];
62.                 attack.target.OnAttack(attack);
63.             }
64.             attackEvents.Clear();
65.             stopwatch.Stop();
66.             UnityEngine.Debug.Log(stopwatch.ElapsedMilliseconds);
67.             stopwatch.Reset();
68.         }
69.     }
70. }
71.  

 

This is awesome!!!!!!

I'm still mulling over the benchmarks so I don't have many questions/comments just yet.

For the OOP test, since we're testing events, I wonder how well a C# event and Invoke() would shake out. I once saw benchmarks showing C# events blowing Unity events out of the water by 20x or so. Obviously Unity events are more versatile for use in the editor, but wow, they sure don't come cheap!

 

I think there is a good middle ground to find between what I did and what you did. It's true that my Monobehaviour implementation isn't efficient, but I do think it's somewhat representative of how the average Unity gameplay programmer would end up doing things in a real game.

In my tests, i measure total frame time of everything. And in a real game, I think every "creature" would be a GameObject with a Creature monobehaviour. I would assume these 500k GOs would have some impact on frame time (I don't actually know). "Attackers" should also be monobehaviours that have a reference to a Creature to attack.

But it does seem fair to skip the monobehaviour update callback and have a single update manager for this test. On the other hand, when taking into account the weight of GOs, it starts to become less of an eventSystems comparison, and more of a "DOTS vs Old Unity" comparison.... It's a bit tricky to find a DOTS vs OOP events comparison that makes sense, because DOTS has use cases for "events" that OOP doesn't need. Damage-handling is an example of that. In OOP, damage can just be an abstract function call on a Damageable child class, maybe with a OnDamaged delegate that multiple behaviours can subscribe to

I'll come back later to run your test on my machine, and probably update the top post. I'll keep the "dumb" monobehaviour implementation for reference, but I'll also add one that does:

• spawn 500k creatures
• spawn 500k attackers with a ref to a creature
• a central update manager iterates attackers List and calls .AttackTarget() on each

 

There's also a difference of running the events on main thread vs worker thread.

 

Just for fun, I tried the same code of yours in a .Net 6.0 console app and low and behold, 3-4ms after some warm up time.
This is really a testament to how awfully slow Mono is and how damn fast .Net 6.0 is. I'm honestly shocked how well it performs with such simple code.

After that, Burst can't be missed. It's exceptionally fast but also a little harder to write with needing pointer access to maintain this style of code. It runs at around 1.70ms single-threaded.
Round of applause for Burst!

I'd say, of course this is relative to my PC, but 1.70ms with this test is like the floor benchmark we could reach on a single thread.

Spoiler

Code (CSharp):

1. public partial class OOTest : SystemBase
2.     {
3.         public struct Creature
4.         {
5.             public float health;
6.  
7.             public void OnAttack(AttackEvent e)
8.             {
9.                 health -= e.damage;
10.             }
11.         }
12.  
13.         public unsafe struct AttackEvent
14.         {
15.             public Creature* target;
16.             public float damage;
17.         }
18.  
19.         public int totalEntity = 500000;
20.         public bool runTest;
21.         System.Diagnostics.Stopwatch stopwatch;
22.  
23.         NativeArray<Creature> creatures;
24.         NativeList<AttackEvent> attackEvents;
25.  
26.         protected override void OnCreate()
27.         {
28.             creatures = new NativeArray<Creature>(totalEntity, Allocator.Persistent);
29.             for (int i = 0, length = creatures.Length; i < length; i++)
30.             {
31.                 creatures[i] = new Creature()
32.                 {
33.                     health = 1000.0f
34.                 };
35.             }
36.  
37.             attackEvents = new NativeList<AttackEvent>(creatures.Length, Allocator.Persistent);
38.  
39.             stopwatch = new System.Diagnostics.Stopwatch();
40.         }
41.  
42.         protected override void OnUpdate()
43.         {
44.             stopwatch.Start();
45.             // create events
46.  
47.             var job = new OOTestJob()
48.             {
49.                 attackEvents = attackEvents,
50.                 creatures = creatures
51.             };
52.  
53.             job.Run();
54.        
55.             stopwatch.Stop();
56.             Debug.Log("OOTest: " + stopwatch.ElapsedMilliseconds + " first creature health: " + creatures[0].health);
57.             stopwatch.Reset();
58.         }
59.  
60.         [BurstCompile]
61.         public unsafe struct OOTestJob : IJob
62.         {
63.             public NativeArray<Creature> creatures;
64.             public NativeList<AttackEvent> attackEvents;
65.  
66.             public void Execute()
67.             {
68.                 var creaturePtr = ((Creature*)creatures.GetUnsafePtr());
69.  
70.                 for (int i = 0, length = creatures.Length; i < length; i++)
71.                 {
72.                     attackEvents.Add(new AttackEvent()
73.                     {
74.                         target = creaturePtr + i,
75.                         damage = 1
76.                     });
77.                 }
78.  
79.                 // inform events
80.                 for (int i = 0, length = attackEvents.Length; i < length; i++)
81.                 {
82.                     var attack = attackEvents[i];
83.                     attack.target->OnAttack(attack);              
84.                 }
85.  
86.                 attackEvents.Clear();
87.             }
88.         }
89.     }

 

Personally, I think an "event system" is an anti-pattern in DoD. DoD is all about looking at the data, and all the requirements and constraints on that data, and then finding the most direct and efficient solution for the problem. Sometimes those requirements and constraints bake in abstractions and API surfaces, or sometimes they bake in desired characteristics and flexibilities of the API. That's fine. But extremely generalized solutions should merely be a byproduct of the same approach being identified multiple times. They also may serve as fallback solutions, but the moment you "fall back" without considering other options, there is a good chance you are giving up performance.

I'm always going to give you my best initial guess as to what the most direct and efficient solution is, even if that means guessing at some of the requirements and constraints that aren't specified. The more specific you get about your real-world problem, the better performance tips I can give you. If you don't want performance, usually you should ignore me (and preferably tell me that you don't care enough about performance and are ignoring me).

Anyways, I am beginning to think it was stupid for me to even engage in this conversation, since what people seem to want here is nonsensical to me. But I digress.

 

I also don't understand why you would need a event system.

A state based system fits much better with ECS. You transform data to change the state directly rather then raise a event to change the state.

State A > State B > State C

 

I explain the reasons (or my reasons at least) for event systems here: https://forum.unity.com/threads/com...ches-for-events-in-dots.1267775/#post-8053421

It's about improving ease of use and code maintainability, which is equally as important as performance

Transforming state directly means that everything that transforms state must know about all data access requirements and ordering requirements. Event systems removes the need to know about all this

 

I don't use anyone's else code and it is very simple to see what mutates the state. At least for me it is no harder than an event system.

I am dubious about reusing logic code in general. I really don't see ECS code being a good fit for the Unity Asset Store.

 

Most game programmers will end up using systems & tools designed by other programmers, whether in-house or third party. In both cases, we can't just expect every programmer to already know all possible pitfalls and implementation details that would be necessary for using those tools correctly. Designing APIs that make your life easier and prevent you from doing things wrong is important

ECS can be a good fit for the asset store, thanks to patterns such as event systems. Among other things, event systems can make sure that everything the user intends to do is processed correctly and safely, and at the right time. I think being 100% focused on performance would be losing sight of the real goal; which is to make a good game. It's much better to find a good balance between performance and usability

 

I agree with you. While I maintain a fantastic event system library, I've said for quite a while I personally only use this for debugging tools where I want to easily and safely extract data. Things such as drawing, recording AI states for visual playback or telemetry. I don't use it at all for anything game play related.

I disagree with this. That said imo it requires writing code in a very specific way to be expandable (for example either generic throughout allowing injection of code or writegroups for everything etc).