Questions on if Unity ECS is a good fit to start developing on now

My team is in a position where we're trying to figure out whether Unity's ECS is in a good position for us to start developing on. The game we're working on has a lot of simulating state and ECS is a really good fit for it. We currently have our version of ECS that's built on Unity's Job System and Burst Compiler so I've been able to see just how efficient it can run. However we're lacking features that's causing me to reevaluate Unity's ECS before development really starts in earnest.

However going over examples, API lists, and decompiling Unity's ECS I'm having a hard time answering if Unity ECS has the features that we'll end up needing in order to deliver on our game. A big wrench that's thrown is that our game is networked and uses a predicted deterministic lock step networking model (think GGPO but we won't predict the entire state of the game...similar to how Factorio does it based on their blog posts). We had been working to make our own ECS have easy support for this model but if Unity's ECS can support it as well then that would save us significant time.

Checksum State
If our game is deterministic then we'll need a way to detect that we are in sync at least while in development. In some ways ECS makes this easy because it packs state in such a easy to look at way. Would it be easy to checksum whole archetypes? This should also include being able to checksum dynamic buffers as well as a way to custom checksum components that have complex members (like a pointer since that can't be checksummed by memory). I'm aware of other issues such as padding that can make this tricky but there are ways around that.

Also for sanity sake being able to dump a log of granular checksums so we can have tools to show which components, or even entity, had diverged can help with debugging. Are there sufficient hooks to do this kind of thing in Unity ECS?

Archetype Rollback
We're not doing full state prediction so some state will predict every frame while others don't. If in one world then you could imagine there being a Predicting component that specifies certain archetypes as the ones that predict and a predicting system would query to only work on those archetypes. However, once we get the input from other players we need to reset the archetypes that were predicting. I see that there is serialization/deserialization for whole worlds but what about archetypes chunks?

It's possible instead that we have a predicting and non-predicting worlds instead. But I'll have to have a clean way to interact between the worlds, for example, EntityCommandBuffer might actually be from arbitrary worlds?

Messaging
This has less to do with our networking model and more of a question of how this would be accomplished in Unity ECS but how does Unity handle messages that sent from one entity to another (or one system that targets another entity). An example might be a system that explodes and damages entities. You could imagine this system gathers up a bunch of entities within radius of the exploding entity and registers damage to those entities. You could imagine it could do this by adding a component to the damaged entities with the damage amount. But what if two exploding entities overlap an entity, wouldn't we lose one?

Split Up World Into Chunks
Our world will be big enough that we won't want to simulate the entire thing constantly and can sleep chunks if the players are far enough away. As far as I can immediately tell there are two ways I could try to approach this.

1. Split entities into separate worlds based on their location. Then I could only update the worlds that are 'awake'. I'll need a way of wholesale moving entities from one world to another when they move from one chunk to another.
   
2. Have dynamic tags. I could tag entities with a chunk index for example which would put them into specific archetypes and only run the archetypes that are currently 'awake'. If tags have to be typed structs though then I'm not sure how I see how I would do this in Unity ECS. Filter based on a shared component maybe?

Unsafe Component Writes
We had to build our own collision system to ensure determinism. This collision system currently runs in its own 'physics' world and after it runs it'll run a sync system that takes its physics position and syncs it to the game simulation system. We do this unsafely by writing directly to the component memory in our own ECS system. This is safe to do because there's only ever a 1 to 1 relationship between physics entity and simulation entity and won't be subject to the lost update problem. I can't immediately find a way that we would do this with Unity ECS. I understand this is 'risky' behavior but is there a way to do this?

Save/Load Versioning
It looks like this is supported when looking at the APIs but they aren't fully fleshed out so I would want to confirm that deserialization/serialization of worlds can work across build versions. What if a components has a member added, removed, or rearranged? Assuming it's a memory serialization then I imagine this would have issues otherwise.

Update
This one looks supported but wanted to bring it up as it is essential to our model but we have to control the update function tightly. We'll have to pause it when there aren't any inputs and step it forward arbitrary amount of times based on the number prediction steps.

Any guidance here would be awesome I'm excited and hopeful the Unity ECS is a good path forward as not reimplementing even a small part of the features would be an amazing time saver!

 

Thanks for the reply!

I must still misunderstand cause either only one of those explosions would add the system or I'm not sure how they would both accumulate the damage since I don't believe I have access to another entities component. If I kept the value outside ECS then I might hit race condition issues?

The networking model we're following is a little bit different than the one you're talking about but there's definitely some overlap in concepts so maybe it would be a good source. The short of what I need is a way to rollback the state of archetypes chunks to a known good state. The same is probably true in the prediction in a replication style networking model.

Basically if I want to do something along the lines of...

// Pre prediction storing of state
bytes = GetArchetypeBytes(archetype)

// blah do some prediction with systems

// My rollback for this archetype chunk!
SetArchetypeBytes(archetype, bytes)

 

You'll likely want to add a dynamic buffer to the entity. Or rather, add an entry into the buffer. You can specify custom buffers so you'd have something like Damage, DamageType in there, then you can have a system which applies damage and adds status effects based on the DamageType, then removes the buffer entry.

Also you're not adding a system, you're adding a component - semantics

 

Sure I meant component

The dynamic buffer idea would only work if it supports concurrent writes and I'm able to sort the buffer before consuming it (otherwise I might break determinism). I can get it as a NativeArray so I can probably sort it but I don't see how I could do concurrent writes to it.

 

Messaging differs a lot in ECS.
What you would do at this stage is create a combatEvent entity with all the necessary data that is consumed by a CombatEventsSystem
These are fast-living entities which are, if your system order is correct, not even a single frame. So think of messages more like its own entity, completely decoupled from any dependencies but with all the data you need and not like buffers/arrays or anything like that. The reactive nature of ECS and archetype design can deal with that.

On the topic of parallel writes:
You can use NativeDisableParallelForRestriction to multi-thread different parts of the chunk.

Right now, I think determinism is the biggest dealbreaker for you. I haven't seen too much problems myself but I also don't have a running network game out in the wild. When the Unity devs say it's not quite there yet, I trust them.

 

I understand that idea as well but it still kind of falls apart. Even if I create a 'combatEvent' component which contains the entity to target and the damage, then it doesn't seem especially clear to me how I would process those events other than manually through the EventManager. Which I can do! But feels somewhat unfortunate since, while I have to run the system in non-parallel (due to Lost Update problem) I could run different event systems in parallel against each other based on their comp read/writes.

And if I'm gonna go that far I can also create my own command buffer handle that keeps a NativeQueue which I sort before processing by a message handling system. That at least solves the determinism issue but still has the unfortunate problem of being required to go through EntityManager instead of any job/burst friendly interface (as far as I can tell).

 

Entity as event and its brother component as event are popular patterns for their simplicity, but they don't really scale well. Actually, event systems are kinda overrated in DOTS as there are almost always better ways to solve the problem. (I'm not saying you should completely avoid them, just that you should not depend on them to solve every problem.)

In your explosion example, instead of thinking about the explosion gathering all the entities and then applying damage to them (a write dependency), think of your entities gathering the explosions they fall into and have them damage themselves based on the data they read from the explosions (a read dependency which is safe in parallel).

If you do find yourself truly needing events, NativeStream is almost always your answer if you care about performance.

 

I totally agree with @DreamingImLatios and that you should really think through your problem and see if you actually need events or not.
In case you need events there is some work already done here using NativeStream.

 

I'm reading dod-book these days, I think your solution on messaging & Split Up World is super fit to that book! Thanks! Messaging in this way is far better than use the entity as event.

May I ask about how would you handle mutable state such as AI current action? Use SharedComponent? I'm not very sure about why document say SetSharedComponent is costly, it seems like doing exactly the same thing as other structural changes, like adding a tag component.

 

It's working like this, there are two systems, Exploding system, ApplyDamage system.
Let's say there are 2 bombs and a player nearby, the exploding system query all bombs and detect if there is any player in bomb range, if yes, then accumulate damage by using GetComponentDataFromEntity<AccumalateDamage>. Later (maybe 4 frames) the ApplyDamageSystem would query all AccumalateDamage and apply it to the Health component and clear it. In this way you can accumulate many damage by all sources, bomb or bullet, with one component.

What other people suggest is using Entity as event. In Exploding system, after make sure which entity with health is nearby, using EntityCommandBuffer to create a new temp entity with a single component, Damage, with two fields, damage value and receiver entity.
And later in the ApplyDamage System, you query all this kind of temp entity - component, Damage, and apply effect one by one on it's receiver.

I use the latter method, but now I prefer the first one mucccch more. English is poor, hope this is understandable.

 

It kind of depends on your timelines and stuff. I use ECS on a mostly "for fun" project, where I can deal with the occasional bugs/missing features/incomplete documentation. If I was paying a team of people to get a game done in a year, I'd definitely avoid ECS -- we'd be using 2018 LTS and banning anyone from even looking at the "show preview packages" option in the editor .

 

I'll not respond to all the messaging ideas as it would be a little off topic

Thanks for response Joachim_Ante!

This would be our initial approach but I have my doubts it'll scale (our world can be quite large) but it's sufficient to know that it is possible to do even if it's not simple.

This is interesting! We've already built a fixed-point math library and physics system but it does run slower than a floating point system. I guess the pointed question I want to dig at is is it deterministic on all PC CPU/OS that we could expect to support today? We're not looking to do cross platform play but on desktop we would need to rely on determinism.

I guess a follow up question then is would Burst be deterministic on the same platform then?

I think I must not have been clear. I'm trying to get a feel for how serialization/deserialization of a world would work when loading a game after a player has updated their build which means that components might have changed shape. I don't mean allowing players of different versions to play with each other. I'm not sure if determinism is a factor here.

Thanks!

 

It is unclear is floating point determinism on the same architecture close or far? Or just deterministic on the same individual PC.

 

I wouldn't feel comfortable starting development of any sort of *real* game with DOTS at this moment, but I do feel comfortable starting to develop "tools" for it.

Basically the way I approach it right now is that I've just started work on building a library of gameplay tools (characters, cameras, hitboxes, weapons, projectiles, inventories, pickups, stats&buffs, etc...), and I intend to keep doing that for the next 1-2 years while DOTS matures. It's a great opportunity to start learning, and to be ready to make something with it as soon as it's ready.

I'm personally not a fan of any sort of hybrid approach to DOTS, where parts of the game are implemented with the Monobehaviour/GameObjects workflow. It just feels a bit messy to me, and I'd rather go either full-Monobehaviour or full-DOTS. So I'm very much waiting for that to not be required anymore before I consider making a real game with DOTS (I'm excluding the conversion workflow stuff here, it's okay if that stays)

 

Would this be with/without burst or either? If I can comfortably drop our fixed point system then that would make some things MUCH simpler for us.

EDIT: Trying to be very clear with my question as whether or not we can rely on floating point determinism could be a big win for us!

Sounds like on console (all three) we can rely on floating point determinism for each individual console?

On desktop are we able to rely on the following...

1. Same architecture different OS?
2. 32-bit Architecture vs 64-bit Architecture? (maybe not required for us but would be nice to know)
3. Mono is deterministic? The il2cpp is?
4. Does this include the Mathf library or just the math library?
5. I guess really the question I'm digging at is for any expected desktop user should we be able to rely on determinism not just on the same computer but across all desktops?

If these questions seem repetitive to above it's only because this is a critical piece of information for us and I'm trying to make sure I understand

 

Ah I see, it's deterministic when using Burst.

How does that apply to Nintendo Switch? As far as I can tell Switch is not supported according to the Burst documentation. I have no idea if outside of Burst if floating point determinism can be counted on for Switch.