Search Unity

  1. Curious about what's going to be in 2020.1? Have a look at the 2020.1 beta blog post.
    Dismiss Notice
  2. Want to see 2020.1b in action? Sign up for our Beta 2020.1 Overview Webinar on April 20th for a live presentation from our evangelists and a Q&A session with guests from R&D.
    Dismiss Notice
  3. Interested in giving us feedback? Join our online research interviews on a broad range of topics and share your insights with us.
    Dismiss Notice
  4. New Unity Live Help updates. Check them out here!

    Dismiss Notice

Unity ECS and Job System in production

Discussion in 'Data Oriented Technology Stack' started by eizenhorn, Sep 26, 2018.

  1. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    I'm always on latest packages and beta. I'm using Rider.
     
  2. wechat_os_Qy0_X1ITy6VrZHYPJFd58JGlA

    wechat_os_Qy0_X1ITy6VrZHYPJFd58JGlA

    Joined:
    Nov 15, 2018
    Posts:
    61
    Thanks for your helps guys.

    I have fixed this problem, I updated my Burst and ECS version.
     
    Last edited: Oct 12, 2019
    MNNoxMortem likes this.
  3. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Just compilation of what we have now:)
    Btw if some one will go to DevGAMM Minsk 2019, come to say hello :) we’ll be showcase and I will be speaker again about our game and how we solved most popular RTS problems and things with DOTS. Also there will be @Fabrice_Lete with talk I’m looking for, don’t miss your chance to see him in person and ask about DOTS guts!
     
    phobos2077, Enzi, pm007 and 12 others like this.
  4. Karsten

    Karsten

    Joined:
    Apr 8, 2012
    Posts:
    105
    This is all nice, interesting and promising, but will it ever be "easy" ?
    For me DOTS feels much much more complex than pure OOP
    leading to much longer development time.
    It somehow makes me think "I'm building block towers in the air without foundation"
    Does any of you guys know an indie/commercial released game that has been build with a
    ~100% DOP approach (Unity or another engine) ?
    Dont get me wrong I don't want to say ECS/DOTS/DOP is generally "bad" I am just interested
    learning about its positive AND negative sides.
     
    Last edited: Nov 9, 2019
  5. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,244
    @Karsten this thread is not dedicated strictyl to DOTS, but applying DOTS with results in production.
    Title is a little misleading in this sense. Thread is more about project using DOTS.

    DOTS main purpose is performance.
    But it gets easier to use from month to month.
    Still nothing close as OOP, and lack of features in comparison.
    It does not means, it stops anyone from building games using DOTS.
     
  6. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    266
    Blizzard have made ECS based engine for Overwatch. There are talks on GDC Vault where they are explaining the benefits of ECS approach for them.
     
  7. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    530
  8. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    We added dynamic to our game, our world become much more alive :)
    Animals system. Birds flying around, animals (chickens, rabbits, deers) living their own lives, avoiding obstacles, eating grass, reacting to player and enemy units (running away)


    Trees shaking a bit, reacting to wind.
     
    Last edited: Nov 15, 2019
    pm007 likes this.
  9. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,043
    Important to note that blizzard used ECS for its architectural benefits, and not for its easy ability to combine with data oriented design and performance improvement. They favoured ease of use, etc. over raw performance.
     
  10. Vacummus

    Vacummus

    Joined:
    Dec 18, 2013
    Posts:
    130
    Yup, they also did it for determinism. ECS made things a lot more predictable for them that allowed them to easily solve a lot of issues dealing with building a multiplayer game (such network latency issues).

    But I do want say that though DOD does allow you to easily write performant code, the very same reasons why it is able to do this are also the reasons why it makes it easier and more predictable to make games with. By simplifying and making it more predictable for the hardware to reason about your code is why DOD is performant. And by solving this problem at the root level (hardware), the benefits naturally cascade upwards to the devs that have to reason about the code as well. In contrast, OOP overcomplicates and makes it very unpredictable for the hardware to reason about your code, and this naturally cascades upwards as well.
     
  11. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Did you miss me?:D
    Our first step in to really epic battles :) Siege machines! :eek: Just first iteration, without any cool VFX, enjoy!


    EDIT:
    Just a bit improved trajectories, rotations and now trees flying too :)
     
    Last edited: Dec 7, 2019
  12. Radu392

    Radu392

    Joined:
    Jan 6, 2016
    Posts:
    192
    That looks really cool. Been following this for a while but never posted anything. I’m doing something similar, only in 2D and another setting. A few questions.

    How many units can it handle on the screen at once? And on the map?

    Also, how many people are working on this project? Are you guys working full time?
     
  13. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Around 30-40k units total (army + enemy units + citizens + animals). Also on screen you see around 50k non static trees and 20k cliff tiles. On this screen you see around 15k, but I will improve this count, I have a room of places and ideas for optimizations, but currently we focused on gameplay features and stability, after that we will cleanup and prettyfy project and code.
    1 programmer (me), 1 artist, 1 game designer, and recently we hired one more programmer (as part timer) because I’m program not only gameplay logic but graphics to (shaders, etc.).
     
    Last edited: Dec 6, 2019
    phobos2077, Jes28, fopsdev and 2 others like this.
  14. June1111

    June1111

    Joined:
    Aug 11, 2017
    Posts:
    29
    Just thought I'd chime in for a second. Read your post back when I was playing around with ecs when you first posted it. Based on what I saw I decided to build my whole game on dots and it really really paid off. I won't ever go back to the old way of making games after learning dod.
    Just thought I'd give my thanks eizenhorn. Looking forward to playing your game someday.
     
  15. Radu392

    Radu392

    Joined:
    Jan 6, 2016
    Posts:
    192
    Once you go dots, you can’t go back.
     
    mkracik, Jes28 and MNNoxMortem like this.
  16. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,244
    @eizenhorn loving your flying trees.
    You need just explosion and flame :)

    Somehow deer animals stays safe, even under shadow of falling rocks from the sky.

    Well done ;).
     
    eizenhorn likes this.
  17. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Yep as I said it’s without vfx yet :) will be on next step. Animals already safe, just not in video, they running away from buildings player and enemy units correctly now, and they wouldn’t be in target area anymore :) Thanks :)
     
    Antypodish likes this.
  18. francois85

    francois85

    Joined:
    Aug 11, 2015
    Posts:
    936
    For the pathfinding and moving are you using steering with physical?
     
  19. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Not. Read on second page.
     
  20. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Just improved units collision :) Happy New Year and Merry Christmas :)
     
  21. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    266
    Looks cool. Are you using ECS Physics for that or some custom solution? And how big is your flow field cell relatively to unit size?
     
    Antony-Blackett likes this.
  22. Antony-Blackett

    Antony-Blackett

    Joined:
    Feb 15, 2011
    Posts:
    1,418
    You’re all giving me ideas for ecs driven games. This is dangerous as I’m only part way through my current project. XD
     
  23. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    411
    how are you planning on tackling vfx? as more ecs systems or using unity particles, or something else?
     
  24. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Custom physics. One flow field - location, is 40x40 (standard Unity units) whole map is 25x25 locations, one flow field cell is 1x1, unit size ~2-3 units in radius (of course siege machines bigger)
     
    Micz84 likes this.
  25. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Write my own shaders and systems :)
     
  26. bugfinders

    bugfinders

    Joined:
    Jul 5, 2018
    Posts:
    129
    @eizenhorn I just wanted to say how inspiring it is to see this. While so much of me would love to see how you coded it all up in dots, I respect your game and obviously if you showed us all how its done some of that magic would be lost.

    I am given great inspiration that great things will come from dots games as it will allow us far more complexities and quantities than ever before
     
    eizenhorn likes this.
  27. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Thanks. You can see some code on my talks (2018-2019 years) from DevGAMM conference. Latest was in Minsk at the end of 2019 (soon will be on YouTube). Where I detailed showing structure of our pathfinding and code how it implemented (19 jobs) and how our fog of war\ health bars implemented on JobComponentSystems (collecting data) and post processing shaders (visualisation of fog and bars). I'll post it here when it will be public. It's in Russian, but all slides and code in English and you can use auto translation for YouTube, I have strong and clear voice, it should be translated fine :D
     
  28. Miroslav-Zeman

    Miroslav-Zeman

    Joined:
    Feb 11, 2014
    Posts:
    1
    @eizenhorn Hey! Just want to say its amazing how much you progress with DOTS! Hope the presentation from Minsk will be on youtube soon! Im so interested in seeing your DOTS flow! :D
     
    phobos2077 and bugfinders like this.
  29. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Just small preview from our first story trailer :)
    preview.jpg
     
  30. LazyGameDevZA

    LazyGameDevZA

    Joined:
    Nov 10, 2016
    Posts:
    112
    Seeing how far this project has come in the 1.5 years I've been following along here it's really a testament to what can be done with DOTS. There's still loads of sharp edges developers can get caught out by, but if you're willing to do a little reinvention of the wheel the gains seem to be worth it. Keep up the good work @eizenhorn !
     
  31. bugfinders

    bugfinders

    Joined:
    Jul 5, 2018
    Posts:
    129
    @eizenhorn watching your game grow is very inspiring. I wish I had the time to devote to my game writing, but sadly my day job is what pays my bills
     
  32. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    164
    As PlayerLoop is out of experimental phase in 2019.3, I'd say that job system is ready to be used in production. Here's profiler timeline of mine AudioSource.GetSpectrumData replacement:
     
  33. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    As promised, my talk from latest DevGAMM Minsk 2019 "DOTS RTS Case Study" about our implementation of Fog of War, Progress Bars, Pathfinding. (In Russian, use subtitles for English :) )
     
    Sarkahn, RaL, eatbuckshot and 7 others like this.
  34. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,002
    cant wait for story trailer
     
  35. Kelevra

    Kelevra

    Joined:
    Dec 27, 2012
    Posts:
    58
    Hello @eizenhorn! Great talk and thank you for sharing!

    Noticed one thing in your FogOfWar fill system. Your algorithm is iterating over X coordinate and then over Y, considering your memory layout it's not very efficient from the point of processor cache. Because your index is always jumping between array indexes which leads to cache misses (x + y * MapSize). Try to swap cycles and iterate by Y and then by X it should give your some performance boost due to better utilisation of data from cache.

    Also, I wouldn't be so sure about "nothing bad" with accessing the same index from different threads because it will leads to "false sharing" and problems with cache coherence. Some times it could leads to performance drops.

    And you can switch from checking length to lengthsq :)
     
    Last edited: Feb 25, 2020
    eizenhorn and siggigg like this.
  36. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Yep I told about that on question time (not in video) I think I wrote this in a midnight, and just overlooked :) From time to time I'm founding this strange things and fixing that. But tbh even with this small ugly pieces - performance good and stable :)
    In this case it's fine :) it gives us full parallelization (without rethinking logic for calculation overlapped fog affectors) and false sharing cost not being so much (of course all is depends, but in our case and map size it's ok)
    Yes, good catch, absolutely agree that jumping through rows for every index it's not so good, yeah iteration over rows at first place it's much better, thanks! It's like with length lengthsq, in some places I'm following best pattern (rows first for good cache filling if accessing indicies) and sometimes I just forgot about that :)
    upload_2020-2-25_15-13-33.png
     
    Last edited: Feb 25, 2020
    Kelevra likes this.
  37. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,812
    Hey! Long time I was gone! Meanwhile, more work than ever! We decided to rewrite globally the core of our game (which was far from ideal) for better memory management and a smoother (for the user) boot process between screens. Now we use Addressables for absolutely all content in our game! They combine perfectly with DOTS! Memory management is now under full control, content loading is extremely fast and optimal, we got rid of all the excess junk by rewriting the kernel from scratch (instead of rewriting in the current version) thanks to the DOTS modularity, all other systems were transferred without any changes and they just work! A completely new Pure DOTS camera controller and with it a day and night cycle system have been written and all this works inside the systems and does not get out of a uniform code style (thanks AddHybridComponent)! Systems for serializing and deserializing the game world have been written, now saving the game works the way we want, preserving the fully current state of the game session up to commands for units, flying shells and even the state of animation continues exactly from the second on which it was saved. On the visual part - the cycle of changing day and night is accompanied by a very pleasant trifle - the lights in the buildings light up, which gives yet another highlight to our pleasant visual style. Сonstruction system has been redesigned, now it is more "saturated"!
    \
    And small secret frames of our first story trailer ;) This one finished but not yet released, and two more on their way! Soon we'll open our Steam page for wishlists.
    upload_2020-4-4_1-59-19.png
     
unityunity