Showcase Unity DOTS case study in production

It’s hybrid approach of A* and Potential Fields, world splitted to big locations (which splitted to 1 unit size grid) On changing obstacles in location, invalidate this location and recalculate portals from this location, to other locations, based on that, I build/change nodes graph and find path from current location to destination location, after that, I’m going through finded graph path and for every location, which included in this graph path, build cost field and then vector field, after all I’m cache locations by some hash in hashmap and all units in same location use this already generated vector fields for moving without requesting new pathfinding. In other words - first request find path, other - reuse this. It’s old approach from Supreme Commander from far 2010 (http://www.gameaipro.com/GameAIPro/...nding_and_Steering_Using_Flow_Field_Tiles.pdf), which in turn, use Crowd Pathfinding approach from far 2006 (http://grail.cs.washington.edu/projects/crowd-flows/continuum-crowds.pdf)

 

This are very interesting readings. Thx for sharing this insight.

 

Debug...Debug never changes...

 

very cool! we are also using flow fields in Spuds Unearthed (pre generated before build for each map, save as an array of bytes - each byte is converted to 2d dir when sampling)

 

Has anybody released games that use Burst? I mean a game that's already in the hands of players? We made massive optimizations using Burst but I'm still reluctant to enable it because players report crashes when we do.

 

It may be, that not every hardware is fully compatible with burst yet. I think there were some issues with iOS not so long ago. Mobiles may act a bit different than desktops as well.

I don't have released product yet. However, since I run tons of tests all the time, in my experience, when burst is enabled, crashes happens most often, when there is some logic error. For example negative index of arrays, or infinity loops. Then I disable burst and test that particular job, until I got that bug again and "kill" it. Sometimes is very rare case. But can happen for one, or other reason.

 

Our game is not using burst, i'm converting some of the systems to utilize it, but some of them give me strange errors but i'm still on 2018.3 and some older version. my work on optimizing the game with multithreading and il2cpp resulted in our game running fast enough on base ps4 so right now i'm really happy with it.

 

Haven't posted our game here yet because we're not yet confident with Burst (used but disabled). Now, I can definitely say that it's working in production. Our early access game just turned Alpha 3 and we enabled Burst compilation. There are no reports of crashes so far. Unlike before where we accidentally enable Burst in release, we get a deluge of reports about the game crashing. Now, I can confidently say that Burst works.

We used DOTS for our rendering as describe in these blog posts:
https://coffeebraingames.wordpress.com/2018/08/19/how-unitys-ecs-expands-your-optimization-space/
https://coffeebraingames.wordpress....o-a-monobehaviour-based-project-a-true-story/
https://coffeebraingames.wordpress....ded-unitys-camera-render-our-game-ran-faster/

Lately, we also used it to optimize our path finding system:
https://coffeebraingames.wordpress.com/2019/04/14/how-fast-could-burst-compiled-a-be/
https://coffeebraingames.wordpress.com/2019/05/19/from-160-to-300-a-requests-per-frame/

We also used it on some of our gameplay systems. Our project is still heavily OOP and MonoBehaviour based. The project started when DOTS wasn't around yet. We probably won't be able to convert the whole project to pure ECS. But the speed gains we got from our partial conversion are already huge.

I want to thank Unity devs! Keep on improving DOTS. Games like ours need it.

 

Hey, forgot to show you our new trailer from last Unity Developer Day Moscow 2019

 

Really cool, but no sounds?

 

Yep, cos stand was just visual, sound was useless cos noise around But we shown our first demo and users played in our game!

 

Thats fine. But you could even add background music from a game. Just saying

Wonder who gave you thumb down on vid?
You got haters already?

 

And can you share the demo ? I would love to test it

 

Maybe. All opinions can exist! Different people, different preferences.

 

Not yet It was our first demo, just one of our regular latest builds more or less stable, but not perfect soon we’ll show public demo

 

very nice!!! How come you dont have the stutter issue, are you rendering your models in a special way?

 

Nope. RenderMeshSystemV2 (from Hybrid Renderer package)

 

300 half-spartans Just testing arrows. Animations same on all objects, but other in progress

 

Everyone bobbing dancing, even dead shadows
Didn't Spartans had spears and shield?

 

Does it have local avoidance where units path (flow) around each other individually? I've been trying to build this SAME algorithm off and on for a long time (years even), based off of Elijah Emmerson's article.

Units switching from one sector (grid square) to another sector was the only thing hanging me up (as well as local avoidance to some extent). Main thing is, how do you transfer goal locations through grid edges (portals I guess)? How do you code portals?

Is this something you could sell as an asset on the asset store? Pretty please Or if not, can you put me in contact with someone who can teach me how to finish my implementation of this pathfinding algorithm? I basically have it half way working already.

I love RTS and pathfinding is very important!


Secondly: What do you recommend to start learning how to use ECS and Job system in a real use-case? That is, in a practical hands-on way. I know you gave some examples, but more explanations maybe. I am a bit scared to try using them honestly. But I want to unleash the power of multi-threading.

Thank you, for all of your work!

 

Would be interested to hear how you've approached this too. It took me a while to get this part working well but it kind of does now.

I've made my portals span across a column/row of nodes in my case. So the flow field is generated starting from multiple goal nodes.

A mistake I made originally was having a single goal node which meant my units would not spread out so much and would clamour together all focussing on one specific location at each portal between flow field tiles.

 

On vacation guys, I’ll answer you all tomorrow. Until that you can look how our movement changed and improved now

Before - movement between locations, in stairs style. Looks ugly

Now - smooth movement between locations and nodes

 

Looks like playing a snake

 

Ok, I found a time on vacation

Actually we don’t have “real” avoidance or steering, I mean based on some cool algorithms, I tried them, but after game test our game designer reject them, cause they looks very... hmm... right, maybe... very pedantic and clear In our game we want some sort of chaos on big scale, cause it looks more realistic, imagine huge crowd, bugs swarm etc, they just like water flow. In our case we use our own basic physics (xz plane collisions) and calculate direction vector (from flow field cache) affecters which relates on collider type (dynamic or static) and depends on they directions (for dynamic colliders) relative to each other, we change direction.
And it looks like how we want to see it.
We have 2 type of caches:

1 - locations cache

2 - goal locations cache

All of them NativeMultiHashMap where key is hash int and value is entity which keeps DynamicBuffer.

First type is flow fields for locations through graph path to goal node. Second is goal location flow field. Difference is how they calculated and stored. First stored by Start + Current + Goal location hash (for reusability) Second stored by Goal point hash in world space.

When unit moves, we get cache and check - we have valid direction for current pos in current location or not. If yes we get direction from current location by current pos in that location (location is 40x40 of float2 flatten to 1d array stored in DynamicBuffer). If no, we create PathRequest, and path generation system finds A* path through locations graph, generates location fields. And when we process next request, we check - maybe any of previous requests already generated our required cache, in that case we just skip this request, cause we already have generated valid path. This helps reduce pathfinding iterations and reuse cache very intensively. It’s in general, we of course have logic for smoothing path, for rebuilding parts of graph (for performance reasons, cause rebuild whole graph every time is stupid ), we have logic for invalidation locations and removing their cache.

It’s too early for assets, I tought about that. May be after release I’ll cleanup logic and will put it on public repo.

For learning I recommend look at sources, dig in to them, look at tests inside packages and try, try and try yourself, for every thing - you see on forum some question about “how implement some thing” just open IDE and try to implement it by yourself, you think about some feature for game, just try prototype it, dirty, ugly, you’ll prettify it later.

 

Ya I spent the last week working on our crowd system. It's a more typical rpg with npc's, up to 500 max is our design peak.

I concur with @eizenhorn that the well known avoidance algorithms emphatically do not really result in good gameplay on their own. Some like Rvo were not even designed for games specifically. Plus every game has differences, different goals and different sets of challenges.

I decided to use Unity.Physics for handling actual avoidance, just queries not using forces. it's overkill for that really but then it's also really damn fast.. So while moving around I do a line circle intersection against other agents to establish the best velocity, using a penalty system. I also have basic depenetration logic if they overlap too much. It's not a perfect algorithm but it's good enough.

We also have a reservation system for goal positions with varying logic for where those are placed. Some are predefined patterns others just force enough separation.

How this fits into pathfinding is the actual pathfinding is a separate system. Then we have a steering system which takes the straight path and then uses methods like NavMeshQuery.MoveLocation to move along the path. Avoidance is interjected into the steering logic.

 

Hey, it's me again
We implemented fog o war as post processing effect with Job system, which collects fog affector) instead of old way with second camera, which was very expensive, cause of culling, and now it looks veeeery cool, now fog feels like real volume and not as flat plane on top of all, it's like dark "eating" everything, you can manipulate fog density, blur amount, resolution! And our navigation system highly improved! Units now moves more organically, uses LOS for direct way and now use straight path everywhere if they can!

 

That's really cool Eizenhorn. Fantastic. Please think about selling some of these systems you're developing on the asset store for all of us RTS fans please

I don't know how to develop my own fog of war, and I'm having to rely on 3rd party assets. So far there are zero fog of war assets for HDRP. You are making me excited with the potential of these new systems.

Great work.

 

It’s pretty easy implementation!
First - you need simple IComponentData with for affector radius field (float Radius for example), or it can be ISharedComponentData for units with same radius, etc.
Second - 2 native arrays, which will be your Static and Dynamic fog map with 0 and 1 values (1000x1000 in my case), and system with job iterate through all affectors in parallel and setting array values.
Third write post processing effect (it's couple of rows, 10-20) and 2 shaders, first which get your native arrays as StructuredBuffers, and fill render texture (256x256) and blur you strict 0 \ 1 arrays.
And second shader blur fog map again and read depth map, from depth by matrix transformations calculate pixel world position, convert world position to fog map space and check value. Depend on checked fog map value lerp your screen image and fog color.

 

Cheers for sharing all of your learnings. About to dive into ECS on a production game. Trying to get my head wrapped around how to allow designers/artists to manipulate the game properties without a programmer's involvement. Typically that would be done via scriptableobjects. I've read a few posts about some ways around using scriptable objects in conjunction with ECS but it doesn't seem ideal.

Do you have any insights there? How does your team setup and save configurable properties for your game?

 

Conversion workflow. Artists and game designers manipulates by good old GameObjects at editor time. Also we have SO for some prefab connections (just for artist convenience)

 

Are you using LWRP

 

Nope, we're on built-in.

 

Is it possible to use LWRP/HDRP/URP with HybridRenderer + ECS ? Im trying to work out how this all fit together? Im not very far along in my game so just want to make sure I pick the right option

EDIT would love to have access to shader graph and vfx graph

 

Ok so since I’m early in development best to switch to LWRP. Will all the work being done on Hybrid/LWRP transfer to URP, as far as I understand from the blog it just a name change

 

VFX Graph doesn't have a pure ECS representation yet. But there's definitely a DOTS game floating around on itch.io that only used Unity spheres and UI and featured non-spherical 3D shapes thanks to vertex manipulation from Shader Graph.

 

Latest actual gameplay! We finally done our core gameplay!
Now time for some small bug fixes, art improvements and new features!

 

Thats super cool

 

@eizenhorn could you do a post mortem after your release? I just would like to know what you have to say once you have all the struggles behind

 

Hm, maybe, why not

 

We rewrote our farms a bit Previous version was huge and wasn't flexible, now you can build field as you want (close to farm building and other fields related to this farm). Quarry build logic now changed a bit too, if previously you can buid in near any mountain (with 2 free cells in line), now you can do it only near specific mountains with stone vein (our world generator now generate this new biome type). Also we did game bars system (health, progress), it's absolutely scalable, cause it's made on job and post processing! Which mean job collects data for all bar drawers in parallel and send data to post processing shader and shader generates screen texture with bars, which mean it's always same resources cost and processing scale (half of resolution) not matter 1 bar or 100k bars, and cause in our game we have more than 1 bar usually, this cost is fine!

 

Hi @eizenhorn

I also use the ECS to create my own game. Your advice helped me a lot. Actually, I have completed 70% of my work. But When I build it, the FPS drops dramatically. did you faced similar problem?

Regards.

(in Editor)


（in build)

 

Nope, in my case all works as expected - Build faster than Editor version (because safety check and other stuff cutted from build). Did you profile build?

 

yes, the result is very strange. The MinionCollisionJob in the Editor only takes 3.5 ~ 4.2ms. However in build, it takes 20ms. Is the [BurstCompile] not compileed into the build?

 

It seems like that because of some reason, [BurstCompile] is not compiled into the Build, Because this job not marked as Burst in Profile. Are there any settings about [BurstCompile] when building that I misssed?

 

which VS, Unity, ECS version did you use?

 

Have you checked the Burst documentation about building standalone players? There is a flag in the player settings that might be set which disables burst compilation in standalone players (aptly called Disable Burst Compilation, found in Player Settings/Burst AOT Settings).

Edit: See also this section of the Burst talk from Unite Copenhagen 2019.

 

Thanks for you reply!

I used the old version 2018.2, there is not the setting menu. I used the old [Burst] Version as well. So, I need to update my [Burst] at first. And check if it works.

Regards.​

 

I have updated my Burst, But some compile errors occur. Hi, @eizenhorn can you tell me which Burst ECS Unity and VS version did you use? I think I need to make sure these softwares are as same as yours.



Regards.

This is the error message:

Burst.Compiler.IL.CompilerException: Error while verifying module: Call parameter type does not match function signature!
%10 = extractvalue [3 x i8] %9, 0
%Unity.Mathematics.bool1 = type { i32 } %11 = call i32 @"Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null:p_Implicit(Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)"(i8 %10)
Call parameter type does not match function signature!
%14 = extractvalue [3 x i8] %13, 1
%Unity.Mathematics.bool1 = type { i32 } %15 = call i32 @"Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null:p_Implicit(Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)"(i8 %14)
Call parameter type does not match function signature!
%18 = extractvalue [3 x i8] %17, 2
%Unity.Mathematics.bool1 = type { i32 } %19 = call i32 @"Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null:p_Implicit(Unity.Mathematics.bool1, Unity.Mathematics, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)"(i8 %18)
at Burst.Compiler.IL.NativeCompiler.Compile()
at Burst.Bcl.BclApp.CompileAndLink(List`1 methodsToCompile)
at Burst.Bcl.Program.Main(String[] args)
​