When can we use ECS in production?

I am very excited about ECS, and aspire to use it in our new projects. However, I do not know whether ECS will be ready for production in next few months or next year. Could somebody in Unity give us a rough timeline? So that we can make desition whether we should use ECS right now or wait for till it is mature.

 

Ready for production? a few years at least. They released the experimental version just so we could follow development.

 

We, at work, already use the ECS and Job System in production.

 

If they could only fix the excessive unload/reload in Visual Studio when using Incremental Compiler, we'd use it in our current game. Alas, we can't. I'm practicing ECS in a separate project.

 

I switched to rider for the time being, doesn't have the reloading problem.

Also as for production, no reason you can't already as long as the limitations aren't in your way. Burst also works in windows builds now on latest beta which is nice.

I don't believe they'll list it production ready till at least end of the year though

 

It's so pricey and I'm not the only programmer. It's not a good option for us.

 

Any big issues so far? Is it stable enough?

 

Whilst I understand the seeking free software tools, I'm not sure I agree with the "too pricey".

First you can try it out to make sure you like it, second Jetbrains's plans grant access to a suite of tools, not just c#. Personally I have enjoyed Rider far more than I have ever liked Visual studio.


HOWEVER, as for the intention of the original post it is a shame that it is causing a interference with you development. Have you mentioned this in the incremental thread?

 

Everything is completely stable, there are no blocking bugs or problems.

 

We need three seats to make it workable for us. I can't be the only one that works seamlessly while my colleagues struggle. Thus the "too pricey".

I did. There's also a thread here where others are experiencing the same: https://forum.unity.com/threads/vis...ssembly-definition-files.531358/#post-3551236

 

it works out less than £30 a month. it is not anywhere near pricey as far as IDEs go.

 

I'm curious about what is currently possible with the ECS, given that the physics API is limited and more things I don't know. What is the type of game you're making?

 

You can still use physics and other systems in a game if you're using ecs, that's the whole point of the hybrid approach.

You can make any game with it.

 

Various people are doing various things (mother of truisms, I know ). Almost anything AI related can be rebuit to fit ECS patterns for example. As long as you've got the patience to find solutions (hacks) to things currently missing in ECS, and have a basic understanding of multi-threading .

I've done a fairly fast potential field system for example. Fast enough to handle many hundreds of units across really large spaces, with 0 optimizations.
Also used it for ludicrously cheap decision making ( utility ai ).
Someone used the new NavMesh api to do a really large crowd pathfinding simulation.
Another guy is using it to update and draw thousands of units for his game ( with an animation system hack that abuses indirect instancing and baking animation data to textures ).
Am planning to use it for updating and resolving projectile collisions across the entirety of my simulated space.
Also already useful for AI steering routines.
Can really be used to speed up anything that does similar things across many instances.

The problems with it are currently (imho):
Lack of documentation.
Breaking changes with new updates ( to be expected, but really hinders actual production )
Lack of useful in-built filtering options.
The reactive system implementation ( still better than nothing ! )
Experimental nature of it where things are not consistent across the board ( can not auto deallocate anything except NativeArray, for example ).
The odd Burst compiler warning/ exceptions.
The odd Entity debugger UI exceptions here and there.
The time it takes to think and trial-and-error your way into a pattern that suits your system in the ECS paradigm.
Inconsistent ways to populate Systems data ( like component group injection which you can not really filter versus GetComponentGroup ) which can be confusing at first.
The need to reinvent many types of basic wheels until unity does it for us ( to be expected, but currently a big time drain on production, especially if you picked Unity for this reason alone )

 

Cool - can you elaborate on how you implemented?
In my implementation the map is done on the gpu and all 1k agents use monob and similar steering utility el cheapo AI. monob are not the bottleneck, gpu2cpu transfer is.

It seems that ECS is great at moving tons of stuff simply, the galaxy example is perfect. So for an ecosystem simulation I'm considering it.

 

I had a similar problem in ECS with my first attempt, to be honest. Where I was doing things in a very ECS way. Cells were entities, same as units, etc. As you can imagine, this ended up being fairly slow due to the amount of data being copied around, filtered, etc.

As it is, most of my game code is also in mono land. I'm not comfortable enough with ECS right now and dislike writing game logic in it.

I split the system in 3 ECS parts (systems):
Rarely changing obstacle map. ( currently only done once on startup, but allows for destructible obstacles. Regular bare-bones PF solution for now. Not important enough for my project ).
Unit-unit map ( currently one int per cell: pathfinding to closest unit ( stored as positive half float + aggresive/ friendly bit ), threat ( 1 byte ), 1 free byte )
Pathfinding to point on grid ( done on request, is just a flood fill limited to the unit range I talk about below )

The unit-unit part :
0 optimizations:
Have centralized, static, NativeArray of per-unit "cell data" in monoland.
Pre-allocate that with MaxNumberOfAgents * 8 surrounding cells for each. ( currently testing for 800 units )
Each unit gets an index into that array when it is created. Later used to write values back to it.
Parallel job per unit where:
for each of the 8 cells, for each other unit on the same grid, write value(s) into the NativeArray above.

800 units done in <2ms on a mobile i7. Map size does not matter. Scales super badly with number of units on same grid. ( 1600 is unworkable )

1st bare-bones optimization:
Only update units that move. Done in a separate system with a barrier that adds/ removes a component to entities that have the moved flag. The unit-unit system now only processes those that have changed cell locations.

800 units done in <1.5ms on a mobile i7. Map size does not matter. Cell size does now. Still scales badly. Surprinsingly, adding/ removing so many components is slow. Not a lot of gain here. Since I have to pay the cost of SetComponentData anyway, I'll probably change this.

2nd less obvious optimization:
Filter out units we probably don't care about.
Have a sparse grid size of unit VisionRange.
Have a bounding box for each unit that is VisionRange -1.
Add unit to each sparse grid cell that intersects that bounding box ( 1-4 )
When doing the no-optimization pass above, only update against units in the sparse grid cell in which your current unit is.

Scales slightly better assuming only one unit can "occupy" any cell, and that the vision range is decently tweaked. At least it has enforceable worst-cases.

I use the data in monoland and merge cell info from the various arrays. This will certainly become a job once the design is really nailed down ( no sooner as it would just kill iteration speed ).

Missing:
The "pheromone trail". Still have to fit this into the unit-unit bit. Since I don't really copy that unit cell data around at any point, I'm hoping I can afford some more data. Maybe another int ? ( one can only dream )

If my gpgpu skills would be up-to-speed, I'd probably go that way as well. Would like to hear about your experience. I'm particularly worried about all that data being pushed around.

 

Can you give me an example of what you mean?

Isn't that the sort of code level stuff that 's too specific to each case to be in-built?

I thought allocation was native only, what's the problem here?

Which wheels did you need to reinvent?

 

I'm doing some similar work with npc ai, seeing how far I can push creating information dense maps and querying them via jobs.

What surprised me is how big of a difference burst makes. I figured I would have to use some type of spatial hashing to get the performance I wanted with several hundred npc agents. But I ended up with just using a simple 2d grid with naive iterations over it. Burst plus designing for doing as much work as possible per cache line fetch, is super efficient.

I'm also using the new math library and trying to use best practices there. I'm not going all out on trying to create vectorization friendly code, but I think burst is in fact vectorizing a lot of it pretty well. It must be for the numbers I'm seeing when using burst.

In a nutshell I'm evaluating around 50k cells per frame with an average 1ms cpu utilization. With a lot of work per cell going on. So I plan on using this same dense grid approach for all of my ai.

 

That't it!
Mine does diffusion and that requires whole map processing, that's why it's dependent on the number of cells. I'm not even sure it's that useful since scent or information only carries so far, but for now that's what I have and agents do interesting things with it like hunt or run away.
I think the gpu can probably handle 2kx2k maps without flinching and if I set hte numthread to 1,1,1 it won't even affect the main screen update; but anyway as you guessed, it's the data transfer that's killing it.

I created a thread with a link to the github repro, it might help you get up to speed with compute shaders.
https://forum.unity.com/threads/usi...e-influence-map-with-a-compute-shader.539487/

 

Do the diffusion only on what has changed and also do it lazily. You won't need to use the gpu then.

basically create a 2d grid as a 1d array that's at a more course granularity, one representing the size you want to apply diffusion updates to. When something changes in that grid you just flag it as changed. When you go to make a query against the more granular grid, check the course grid to see if it needs updating, and if so update it.

So your diffusion updates just get interleaved into your queries as needed.

Just saw your code. So the above is all assuming you just replace the gpu and main thread logic you have with jobs.

 

50k cells at 1ms is roughly 12x faster than what I'm getting in the old version of influence map, and I'm sure you're performing way more ops. Can you share some of the Mathematics code?

 

All I really do there is use math.select when possible and of course the new math types everywhere.

I'm using IJobProcessComponentData with burst. Burst is what makes a huge difference, around 10x faster.

I haven't really had to optimize beyond that for the amount of work I'm doing.

 

what's that?

 

If you use things in `using Unity.Mathematics;` and they are in [BurstCompile] area of code they can be potentially inlined with something suitable that might be different for each platforms. (Related : https://forum.unity.com/threads/math-selects-performance-vs-inline-if.532282/#post-3505501)

 

You can use shared cmp filtering on component groups but not on injected data.

On large datasets, something in the vein of mapreduce is a must. Unity is making headway towards this.

This ties back into the above. Can't auto-dealocate NativeList for example. Which makes filtering even more cumbersome.

Using the projectile system I'm about to build as an example (want all projectiles in pure eventually ):
1. Shader monstrosity to replace particle effects.
2. Ballistics, reflection, basic collision & physics.
3. Trail renderer.

Unless I'm missunderstanding this, so can mine. In a PF scenario a unit is only interested in his nearest cell neighbors anyway. I "diffuse" threat as distance from unit scaled by effective attack range. This is summed for all enemy units, for each of the 8 cells. Doing this on the full map would yield the same results.

Arbitrary pathfinding can use the diffused threat to avoid enemy hotspots, etc.

What my system is missing though is arbitrary sampling of map point values, as they are not computed.

P.S: by trail I mean the usual thing done to fix the local optima problem. In other implementations I attempted I also used it to stop units from queing behind each-other when going to the same point. I suppose I'm just going to store the last 3-4 cells visited in the unit data, and use it when computing the unit-unit map.

P.S2: checked out your github. You are doing influence mapping. Similar systems with different goals. Have not attempted one in ECS yet as the influence maps I use operate on graphs ( market graphs ).

P.S3: did not mention that the optimized version is about 20 extra lines of code and runs < 0.5ms for 800 units.