Search Unity

Unity ECS and Job System in production

Discussion in 'Data Oriented Technology Stack' started by eizenhorn, Sep 26, 2018.

  1. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    It’s hybrid approach of A* and Potential Fields, world splitted to big locations (which splitted to 1 unit size grid) On changing obstacles in location, invalidate this location and recalculate portals from this location, to other locations, based on that, I build/change nodes graph and find path from current location to destination location, after that, I’m going through finded graph path and for every location, which included in this graph path, build cost field and then vector field, after all I’m cache locations by some hash in hashmap and all units in same location use this already generated vector fields for moving without requesting new pathfinding. In other words - first request find path, other - reuse this. It’s old approach from Supreme Commander from far 2010 (http://www.gameaipro.com/GameAIPro/...nding_and_Steering_Using_Flow_Field_Tiles.pdf), which in turn, use Crowd Pathfinding approach from far 2006 (http://grail.cs.washington.edu/projects/crowd-flows/continuum-crowds.pdf)
     
  2. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    This are very interesting readings. Thx for sharing this insight.
     
  3. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Debug...Debug never changes...
    upload_2019-4-22_21-15-54.png
     
    psuong, NeatWolf, fherbst and 3 others like this.
  4. mmankt

    mmankt

    Joined:
    Apr 29, 2015
    Posts:
    33
    very cool! we are also using flow fields in Spuds Unearthed (pre generated before build for each map, save as an array of bytes - each byte is converted to 2d dir when sampling)
     
  5. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    489
    Has anybody released games that use Burst? I mean a game that's already in the hands of players? We made massive optimizations using Burst but I'm still reluctant to enable it because players report crashes when we do.
     
  6. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    It may be, that not every hardware is fully compatible with burst yet. I think there were some issues with iOS not so long ago. Mobiles may act a bit different than desktops as well.

    I don't have released product yet. However, since I run tons of tests all the time, in my experience, when burst is enabled, crashes happens most often, when there is some logic error. For example negative index of arrays, or infinity loops. Then I disable burst and test that particular job, until I got that bug again and "kill" it. Sometimes is very rare case. But can happen for one, or other reason.
     
  7. mmankt

    mmankt

    Joined:
    Apr 29, 2015
    Posts:
    33
    Our game is not using burst, i'm converting some of the systems to utilize it, but some of them give me strange errors but i'm still on 2018.3 and some older version. my work on optimizing the game with multithreading and il2cpp resulted in our game running fast enough on base ps4 so right now i'm really happy with it.
     
  8. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    489
    Haven't posted our game here yet because we're not yet confident with Burst (used but disabled). Now, I can definitely say that it's working in production. Our early access game just turned Alpha 3 and we enabled Burst compilation. There are no reports of crashes so far. Unlike before where we accidentally enable Burst in release, we get a deluge of reports about the game crashing. Now, I can confidently say that Burst works.

    We used DOTS for our rendering as describe in these blog posts:
    https://coffeebraingames.wordpress.com/2018/08/19/how-unitys-ecs-expands-your-optimization-space/
    https://coffeebraingames.wordpress....o-a-monobehaviour-based-project-a-true-story/
    https://coffeebraingames.wordpress....ded-unitys-camera-render-our-game-ran-faster/

    Lately, we also used it to optimize our path finding system:
    https://coffeebraingames.wordpress.com/2019/04/14/how-fast-could-burst-compiled-a-be/
    https://coffeebraingames.wordpress.com/2019/05/19/from-160-to-300-a-requests-per-frame/

    We also used it on some of our gameplay systems. Our project is still heavily OOP and MonoBehaviour based. The project started when DOTS wasn't around yet. We probably won't be able to convert the whole project to pure ECS. But the speed gains we got from our partial conversion are already huge.

    I want to thank Unity devs! Keep on improving DOTS. Games like ours need it.
     
    CypherEA and Sarkahn like this.
  9. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Hey, forgot to show you our new trailer from last Unity Developer Day Moscow 2019 :)
     
    SugoiDev, AdamGoodrich, L2GX and 3 others like this.
  10. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    Really cool, but no sounds?
     
  11. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Yep, cos stand was just visual, sound was useless cos noise around :) But we shown our first demo and users played in our game!
     
  12. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    Thats fine. But you could even add background music from a game. Just saying :)

    Wonder who gave you thumb down on vid? :eek:
    You got haters already? :p
     
  13. Sylmerria

    Sylmerria

    Joined:
    Jul 2, 2012
    Posts:
    237
    And can you share the demo ? I would love to test it
     
  14. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Maybe. All opinions can exist! Different people, different preferences.
     
    Last edited: Sep 13, 2019
    Antypodish likes this.
  15. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Not yet :) It was our first demo, just one of our regular latest builds more or less stable, but not perfect :) soon we’ll show public demo
     
    Sylmerria likes this.
  16. tommox86

    tommox86

    Joined:
    Apr 30, 2015
    Posts:
    39
    very nice!!! How come you dont have the stutter issue, are you rendering your models in a special way?
     
  17. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Nope. RenderMeshSystemV2 (from Hybrid Renderer package)
     
  18. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    300 half-spartans :D Just testing arrows. Animations same on all objects, but other in progress :)
     
  19. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    Everyone bobbing dancing, even dead shadows :D
    Didn't Spartans had spears and shield?:rolleyes:
     
  20. Velo222

    Velo222

    Joined:
    Apr 29, 2012
    Posts:
    1,292
    Does it have local avoidance where units path (flow) around each other individually? I've been trying to build this SAME algorithm off and on for a long time (years even), based off of Elijah Emmerson's article.

    Units switching from one sector (grid square) to another sector was the only thing hanging me up (as well as local avoidance to some extent). Main thing is, how do you transfer goal locations through grid edges (portals I guess)? How do you code portals?

    Is this something you could sell as an asset on the asset store? Pretty please :) Or if not, can you put me in contact with someone who can teach me how to finish my implementation of this pathfinding algorithm? I basically have it half way working already.

    I love RTS and pathfinding is very important!


    Secondly: What do you recommend to start learning how to use ECS and Job system in a real use-case? That is, in a practical hands-on way. I know you gave some examples, but more explanations maybe. I am a bit scared to try using them honestly. But I want to unleash the power of multi-threading.

    Thank you, for all of your work! :)
     
    Last edited: Aug 11, 2019
  21. jdtec

    jdtec

    Joined:
    Oct 25, 2017
    Posts:
    87
    Would be interested to hear how you've approached this too. It took me a while to get this part working well but it kind of does now.

    I've made my portals span across a column/row of nodes in my case. So the flow field is generated starting from multiple goal nodes.

    A mistake I made originally was having a single goal node which meant my units would not spread out so much and would clamour together all focussing on one specific location at each portal between flow field tiles.
     
    Velo222 likes this.
  22. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    On vacation guys, I’ll answer you all tomorrow. Until that you can look how our movement changed and improved now :p

    Before - movement between locations, in stairs style. Looks ugly:confused:


    Now - smooth movement between locations and nodes :cool:
     
    NeatWolf, jdtec, Attatekjir and 4 others like this.
  23. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    Looks like playing a snake :D
     
  24. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Ok, I found a time on vacation :)

    Actually we don’t have “real” avoidance or steering, I mean based on some cool algorithms, I tried them, but after game test our game designer reject them, cause they looks very... hmm... right, maybe... very pedantic and clear :) In our game we want some sort of chaos on big scale, cause it looks more realistic, imagine huge crowd, bugs swarm etc, they just like water flow. In our case we use our own basic physics (xz plane collisions) and calculate direction vector (from flow field cache) affecters which relates on collider type (dynamic or static) and depends on they directions (for dynamic colliders) relative to each other, we change direction.
    And it looks like how we want to see it.
    We have 2 type of caches:

    1 - locations cache

    2 - goal locations cache

    All of them NativeMultiHashMap where key is hash int and value is entity which keeps DynamicBuffer.

    First type is flow fields for locations through graph path to goal node. Second is goal location flow field. Difference is how they calculated and stored. First stored by Start + Current + Goal location hash (for reusability) Second stored by Goal point hash in world space.

    When unit moves, we get cache and check - we have valid direction for current pos in current location or not. If yes we get direction from current location by current pos in that location (location is 40x40 of float2 flatten to 1d array stored in DynamicBuffer). If no, we create PathRequest, and path generation system finds A* path through locations graph, generates location fields. And when we process next request, we check - maybe any of previous requests already generated our required cache, in that case we just skip this request, cause we already have generated valid path. This helps reduce pathfinding iterations and reuse cache very intensively. It’s in general, we of course have logic for smoothing path, for rebuilding parts of graph (for performance reasons, cause rebuild whole graph every time is stupid :) ), we have logic for invalidation locations and removing their cache.

    It’s too early for assets, I tought about that. May be after release I’ll cleanup logic and will put it on public repo.

    For learning I recommend look at sources, dig in to them, look at tests inside packages and try, try and try yourself, for every thing - you see on forum some question about “how implement some thing” just open IDE and try to implement it by yourself, you think about some feature for game, just try prototype it, dirty, ugly, you’ll prettify it later.
     
    Last edited: Aug 15, 2019
    Velo222, jdtec and Antypodish like this.
  25. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,373
    Ya I spent the last week working on our crowd system. It's a more typical rpg with npc's, up to 500 max is our design peak.

    I concur with @eizenhorn that the well known avoidance algorithms emphatically do not really result in good gameplay on their own. Some like Rvo were not even designed for games specifically. Plus every game has differences, different goals and different sets of challenges.

    I decided to use Unity.Physics for handling actual avoidance, just queries not using forces. it's overkill for that really but then it's also really damn fast.. So while moving around I do a line circle intersection against other agents to establish the best velocity, using a penalty system. I also have basic depenetration logic if they overlap too much. It's not a perfect algorithm but it's good enough.

    We also have a reservation system for goal positions with varying logic for where those are placed. Some are predefined patterns others just force enough separation.

    How this fits into pathfinding is the actual pathfinding is a separate system. Then we have a steering system which takes the straight path and then uses methods like NavMeshQuery.MoveLocation to move along the path. Avoidance is interjected into the steering logic.
     
    florianhanke and pcg like this.
  26. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Hey, it's me again :)
    We implemented fog o war as post processing effect with Job system, which collects fog affector) instead of old way with second camera, which was very expensive, cause of culling, and now it looks veeeery cool, now fog feels like real volume and not as flat plane on top of all, it's like dark "eating" everything, you can manipulate fog density, blur amount, resolution! And our navigation system highly improved! Units now moves more organically, uses LOS for direct way and now use straight path everywhere if they can!

     
  27. Velo222

    Velo222

    Joined:
    Apr 29, 2012
    Posts:
    1,292
    That's really cool Eizenhorn. Fantastic. Please think about selling some of these systems you're developing on the asset store for all of us RTS fans please :D

    I don't know how to develop my own fog of war, and I'm having to rely on 3rd party assets. So far there are zero fog of war assets for HDRP. You are making me excited with the potential of these new systems.

    Great work.
     
  28. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    It’s pretty easy implementation!
    First - you need simple IComponentData with for affector radius field (float Radius for example), or it can be ISharedComponentData for units with same radius, etc.
    Second - 2 native arrays, which will be your Static and Dynamic fog map with 0 and 1 values (1000x1000 in my case), and system with job iterate through all affectors in parallel and setting array values.
    Third write post processing effect (it's couple of rows, 10-20) and 2 shaders, first which get your native arrays as StructuredBuffers, and fill render texture (256x256) and blur you strict 0 \ 1 arrays.
    And second shader blur fog map again and read depth map, from depth by matrix transformations calculate pixel world position, convert world position to fog map space and check value. Depend on checked fog map value lerp your screen image and fog color.
     
    Last edited: Sep 6, 2019
  29. markgrossnickle

    markgrossnickle

    Joined:
    Aug 21, 2017
    Posts:
    32
    Cheers for sharing all of your learnings. About to dive into ECS on a production game. Trying to get my head wrapped around how to allow designers/artists to manipulate the game properties without a programmer's involvement. Typically that would be done via scriptableobjects. I've read a few posts about some ways around using scriptable objects in conjunction with ECS but it doesn't seem ideal.

    Do you have any insights there? How does your team setup and save configurable properties for your game?
     
  30. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Conversion workflow. Artists and game designers manipulates by good old GameObjects at editor time. Also we have SO for some prefab connections (just for artist convenience)
     
    markgrossnickle likes this.
  31. francois85

    francois85

    Joined:
    Aug 11, 2015
    Posts:
    645
    Are you using LWRP
     
  32. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Nope, we're on built-in.
     
  33. francois85

    francois85

    Joined:
    Aug 11, 2015
    Posts:
    645
    Is it possible to use LWRP/HDRP/URP with HybridRenderer + ECS ? Im trying to work out how this all fit together? Im not very far along in my game so just want to make sure I pick the right option

    EDIT would love to have access to shader graph and vfx graph
     
  34. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,671
    HDRP/URP/LWRP is our primary target. After 19.3 we will no longer proactively test hybrid renderer with builtin renderer. All optimization work is also going in the HDRP/URP/LWRP direction.
     
  35. francois85

    francois85

    Joined:
    Aug 11, 2015
    Posts:
    645
    Ok so since I’m early in development best to switch to LWRP. Will all the work being done on Hybrid/LWRP transfer to URP, as far as I understand from the blog it just a name change
     
  36. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,671
  37. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    436
    VFX Graph doesn't have a pure ECS representation yet. But there's definitely a DOTS game floating around on itch.io that only used Unity spheres and UI and featured non-spherical 3D shapes thanks to vertex manipulation from Shader Graph.
     
  38. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Latest actual gameplay! We finally done our core gameplay! :rolleyes:
    Now time for some small bug fixes, art improvements and new features!
     
  39. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    Thats super cool :)
     
  40. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    366
    @eizenhorn could you do a post mortem after your release? I just would like to know what you have to say once you have all the struggles behind
     
  41. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    Hm, maybe, why not:)
     
    MNNoxMortem likes this.