Search Unity

  1. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  2. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  3. Want more efficiency in your development work? Sign up to receive weekly tech and creative know-how from Unity experts.
    Dismiss Notice
  4. Participate with students all over the world and build projects to teach people. Join now!
    Dismiss Notice
  5. Build games and experiences that can load instantly and without install. Explore the Project Tiny Preview today!
    Dismiss Notice
  6. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Unity ECS and Job System in production

Discussion in 'Entity Component System and C# Job system' started by eizenhorn, Sep 26, 2018.

  1. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    Hello everyone! As I promised @Joachim_Ante :) I wrote a little about our experience of moving to ECS and the Job System in a combat project, refactoring, the difficulties we faced and of course the pleasant results we achieved.
    (I apologize in advance for my English :rolleyes:)
    (And also you can read this post on Medium or Unity Connect)


    And so, for many it’s not a secret that Unity set a new course for the development of the engine (in my opinion very true, effective and really cool), innovations of the engine family of the 2018 engine brought a huge heap of features, additions and really important changes, one of the most “powerful”, and I think everyone will agree with me, these are new render pipelines representing new, extensive opportunities for graphic programmers, or rather slightly simplifying their lives, by extending the rendering capabilities from managed code and going under the auspices of SRP — Scriptable Render Pipeline, and the second global innovation management, about the use of which in production I am today and tell you — ECS and Job System.
    I think in the form of a small introduction, it is worth to tell a little about what this design pattern is, very popular in the gamedevelopment, and also about our project, which we successfully put on the rails of ECS and Job-s and continue to develop new ideas for this approach, because almost every week Unity developers bring fresh meat… in the sense of fresh changes, additions and opportunities (often these changes were rolled out very timely, making it easier for us and letting us not write our implementation of certain features, be it Culling, Hierarchy, and other).
    So, what is ECS? Entity-Component-System is an architectural pattern widely used in the gaming industry, as you can guess from its name, it consists of three key pillars: Entity, Component and System.


    I’ll just leave it here…
    Entity — is nothing more than an identifier, which determines and indicates the existence of some object, the entity itself, with which systems can be operated, which can be assigned certain properties by means of components. The component is a data container, without any logic, a certain set of parameters related together and defining properties that you can, let’s say, give to our entity. One of the examples taken from our game can be — resources, groups of entities with Resource component, which has resource quantity fields, its type, the entity with which this resource is associated, etc).

    Component group for injection
    The system is the part that is responsible for the operations on the data, in other words if the entity is an identifier, the component is data, then the system is the action to be performed on certain entities with certain properties (components), all these 3 pillars are independent of each other, which gives our code incredible flexibility, modularity and scalability, we take another set of entities, and the system will also operate with them, set other sets of components — again everything continues to work, this is one of those big pluses e gives us the ECS — great modularity and reusability of our code. Now, presenting in general terms what ECS is, you can ask yourself — “But, I can implement this pattern in my project on OOP, I can define component classes, I can abstract logic aside from all this, I can create my definition of entities and work with this!”, and to some extent this is true, and here comes on stage — Data Oriented Design (DOD) and Job System. An important feature of the new approach implementing ECS in Unity is that its implementation is built around the DOD principle, a good and simple explanation can be found of Mike Acton talk from GDC 2018:
    The essence of this approach, in simple terms and as I understand it, is to organize the data in such a way that they are cache friendly, ie went sequentially and to the maximum were placed in the processor’s cache line, avoid loading useless data (ie having a structure with 10 fields and using only 1 field, it would be better to take this field into a separate structure and load it), which reduces the memory accesses and thus gives us a performance boost.
    The second “kick for speed” is the Job System (Unity has already transferred the manual to the official documentation from github, you can find it Here, as well as basic acquaintance with ECS you can already find on the official site in the training section) and developed for it — Burst Compiler, which compiles the C # / IL code into high-performance native code, and as well as the new Unity.Mathematics library, also optimized for native code, SIMD and Burst, this system allows you to write multithreaded code is fast and easy (in comparison with the classic Thread approach of course), under the hood the system has a good security implementation that will save a lot of race conditions, all work is done on the Worker Threads, the Job System for us operates with available resources and distributes Job-s on them, organizes Lock-and etc. Summing up all of the above, we can distinguish for ourselves the key advantages:
    • Huge performance
    • Easy multithreading
    • Modularity and scalability
    • Available from the box
    All this can be perfectly described by the tagline which, lately, very often sounds from representatives of Unity - Performance by default.
    And so getting an idea about the new systems, let’s talk about applying them in practice. Despite the fact that Unity warns that use in production — at your own fear and risk, we could not bypass such significant and very steep changes that were very suitable for our project, and therefore began to refactor everything from the very first previews of versions ECS and the Job System. As I said above, these innovations are very well placed on our game — Elinor.

    Our game (early alpha gameplay)
    This is a single player real-time strategy in a realistic setting with a small amount of fantasy. The player has to develop his economy, train the army and build strong walls to defend himself against a raid of innumerable amounts (more than a few tens of thousands at a time) at night, and therefore the defending troops must be proportionate to the attacking horde, which leads us to the total army volumes up to hundreds of thousands of units at a time. The economy is not a key aspect, but it is extremely important at the initial stages of the city development in order to establish a continuous production of the army, it is implemented in the form of indirect management of resource flows with full visualization of the entire process of production of game values (and this, for a second, tens of thousands of objects in the game world with its own set of data). Hence again, we highlight the key requirements, and compare them with the advantages that the ECS and Job System give, and which have been described above:
    • Dozens, even hundreds of thousands of units
    • Tens of thousands of visualized resources, population, etc
    • 60+ FPS, by itself
    And as we can see, the bonuses from using the new approach are perfectly superimposed on our requirements, and therefore the use of ECS and the Job System is obvious (well, nowhere without my craving for learning new features of course).

    Perhaps the most difficult thing in applying a new approach is to reorient your brain from the classic OOD approach to DOD, working for many years as a .NET developer on projects built entirely on OO design, it was not easy enough for me, at that time, a sufficient amount of documentation, the forum was just beginning to come alive, and therefore many things had to spend a lot of time learning examples from the repository, reviewing the performances with GDC, Unite Berlin, and actively communicating on the forum. Therefore, “moving” started with a simple, try out the Job System, which, in principle, is quite autonomous from ECS and can be used in the classical approach.
    The first thing that was decided to carry out on parallel calculations is the generation of a mask for the grid construction shader.
    Initially it was necessary to receive an array of pixels in the old manner, then copy it to NativeArray and send it to IJobParallelFor to process the pixels in parallel, then reverse-order the NativeArray into an array and assign it to the texture, this approach gave a slight increase, but not as expected, since the conversion of the array back and forth covered almost the entire bonus of parallel computing. But after a while Texture2D had 2 wonderful methods GetRawTextureData / LoadRawTextureData working immediately with NativeArray and rid us of the dances with a tambourine with the usual array, from this moment the process of mask generation became almost invisible for us, according to the resources spent.
    But this is only the beginning, so it’s time to use all the innovations in a complex way and then we were expecting another underwater rock — in view of the fact that ECS, at that time, was in a very early preview (although now in the preview, but the functionality has already grown significantly) and it simply did not exist many basic things we used, be it the Colliders, NavMesh components, standard renderers, and so on. At first this is very unsettling, and therefore not all things could be immediately transferred to pure ECS. For this reason, many systems implemented in the game had to be translated into a hybrid approach. In a hybrid ECS, we do not get a significant performance boost (only small, on average 5–10% but it’s better than nothing), but we have the ability to work with classic components in systems (I’ll make a reservation, under the hybrid approach I mean using ComponentArray and GameObjectEntity, the rest of the implementation I use the same as with the pure approach, but the hybrid approach has limitations, we still can not use the classical components outside the main thread, so the path to Job-s is closed to them). On this approach, the system of citizens is implemented in our game. Every citizen is a regular GO with a set of classic components — Rigidbody, Collider, NavMeshAgent, etc. But it’s also worth mentioning separately the component Citizen,
    in this component we store different state of a citizen, determine whether he is a builder or a peddler or another type of employee, what resources he carries and whether he carries in general, etc. and we can use such a structure in the Jobs, which we actually did, thanks to such a “dirty” hack, our calculations of the number of citizens in the city, the transitions of their states, birth and death, verification of targets for transfer / collection of resources, etc., occur very quickly (Don’t forget that this is only for a hybrid approach :) )
    upload_2018-9-26_16-43-44.png
    upload_2018-9-26_16-43-53.png
    But this is only a temporary solution, now we are actively completing the core for navigation and physics on the basis of Spatial Hash Map and jobified RaycastCommand and NavMeshQuery (based on an excellent example from Nordeus) after which the civilian population will be transferred from a partially hybrid approach to clean ECS.
    The most interesting part begins with writing systems on pure ECS. At this stage, we have such systems: Displaying the surrounding forest (as we remember from the description, all the resources we have are real objects, which means the trees too, because they can be cut for processing, each tree is an Entity with a Wood component,
    upload_2018-9-26_16-44-16.png
    Wood system
    upload_2018-9-26_17-1-13.png
    Just for understanding world size
    displaying all resources in the game, displaying walls and roads (or rather the mode of their construction, in view of the fact that there is a lot of under the hood calculations, to correctly display the orientation of the walls, their rotation etc.). The pure approach has a large number of requirements and rigid limits in the writing of code, which, again, is due to the tagline — Performance by default. But due to these limitations, performance really becomes transcendental. Thanks to Burst compilation, iterating over all the resources in the game, displaying them, moving them, adding and destroying them becomes quite simple and very fast operation, even though we can have tens of thousands of different resources on the screen at once. its parameters.


    upload_2018-9-26_16-44-38.png
    As a result, after implementing a new approach, we can work with a much larger number of objects than before, performance has grown many times, it has become possible to quickly and easily parallelize multiple flows without worrying about threads, gameplay features that you think you can implement with less effort.
    In my opinion, the fact that ECS is now in active development does not in any way prevent us from starting to use it in production, for our example it is clear that we can now practically implement all the necessary functionality in the game, we did not notice serious blocking problems throughout the transition , and if they did, either in the next update of the package it was fixed and many new useful things were added, or you could find a temporary patch on the forum (or just come up with your own workaround). A huge plus that gave us early access to the ECS and the Job System is precisely the understanding of how these systems are arranged inside, how they evolved from the very beginning, why some systems are implemented this way, and not otherwise, why some things were rewritten or abolished, this allows you to better understand the new system and easier to navigate in it at a low level.
    If you compare the state of ECS when it was just born and now, the differences are huge. The new versions include the system of Culling, prefabs, Dynamic Buffers, Chunk Iteration, rewrited Transform System, Reactive System, Concurent EntityCommandBuffer, LODs and other amenities that make life easier for us, and without which it was difficult at first, and therefore to join a new approach and starting to study it now is much easier, and in view of the fact that behind this approach the future of the engine and its vector of development, everyone should already pay attention to the ECS and the Job System, and start gradually to delve into them.

    If I'm wrong in some points, let me know ;)
     
    Last edited: Oct 18, 2018
    pcg, rudypoo, pojoih and 54 others like this.
  2. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,193
    Thank you so much for writing this. It was really lovely to read.

    It really means a lot to us to hear what you are able to do with the new tech. The art style is really nice too and it's great hear this focus on performance is paying off for your game.
     
    Last edited: Sep 26, 2018
    rudypoo, pojoih, FROS7 and 7 others like this.
  3. SirStompsalot

    SirStompsalot

    Joined:
    Sep 28, 2013
    Posts:
    71
    Lots to digest here! Like you folks I'm switching over to pure ECS and this was incredibly validating. Thanks for sharing.
     
    eizenhorn likes this.
  4. daserra

    daserra

    Joined:
    Dec 7, 2016
    Posts:
    9
    Thanks for sharing that with us. I'm also using it in Production. Some people say you are crazy, there is no need for that, it may change a lot in the future. For me is not only about performance, i'm working in a 2D game and monobehavior can do it just fine. But i'm really enjoying the new way of coding in ECS "Data-Oriented Programming", I think it makes me more productive and my code is much more cleaner now.
     
    Seb-1814 and hippocoder like this.
  5. ComteJaner

    ComteJaner

    Joined:
    Jun 9, 2013
    Posts:
    9
    Great post, thank you fellas!
     
  6. LurkingNinjaDev

    LurkingNinjaDev

    Joined:
    Jan 20, 2015
    Posts:
    1,755
    Great post, this should be pinned in the forum. :)
     
    eizenhorn and hippocoder like this.
  7. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    23,793
    Amazing post, I agree it should be pinned.
     
    eizenhorn likes this.
  8. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    1,157
  9. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    Wow :) Thx :)
     
  10. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    2,442
    Well done. Very interesting to read and see, what was achieved.
     
  11. GameDevCouple_I

    GameDevCouple_I

    Joined:
    Oct 5, 2013
    Posts:
    998
    thanks for such a great post!
     
  12. PalmGroveSoftware

    PalmGroveSoftware

    Joined:
    Mar 24, 2014
    Posts:
    6
    really neat work !
    been reading more in detail now that i am a bit more familiar with ECS basics, and can't help but being a bit confused by something : do you use that citizen data struct in jobs if i may ask ?
    I was thinking even in hybrid mode structs passed to jobs can't have non blittable tpes ?
    still trying to wrap my head around how to separate pure data components as much as possible while syncing with gameobjects for presentation/UI/Input with minimum coupling possible..from what I understand in your implementation entities that will need visuals,physics,audio etc are in same struct with corresponding gameobject and iterated on with basic component system non jobified ?
    Thanks for any pointers !
     
  13. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    Yes as I describe in article this "dirty hack" allow me use Citizen struct inside job for Read\Write, but one moment - I absolutley sure than no one not read and write or write and write same citizen at the same time. And as I say it's just temporary solution.
     
  14. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    Just show-off :p:p:p Our DevGAMM 2018 Minsk trailer which was played on Unity stand with other great games:D And great meet with Valentin and Elena! They both super cool people!


    43422067_2290180817899998_3116834127423125413_n.jpg 44296650_329164854302381_4122957301860391110_n.jpg
     
  15. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    2,442
    Again well done. Keep up going ;)

    Can you share and judge, how many % of game, have you managed to port into ECS ?
     
  16. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    90% :) maybe little bit more, most part is pure ECS, now only physics - hybrid, oh and UI is mono (I think mono is enough for that, but it only for presentation, all data, behind UI, I mean which used in UI presentation is pure ECS data)
     
  17. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    2,442
    Oh that well done.

    For UI that makes sense. Probably by the time we will get ECS UI, it will be a bit late for project, to transit. Unless, if it wont be yet released :)

    I would thought, you don't need classic OOP physics in your game. Seams all can be done on ECS side.
    Is it something you haven't touched yet? Or any other reason?
    I mean, ground seams flat. There are no hills etc, for need to calculate physics.
    Can be physics applied in similar manner, as demo from 2017, with thousands of minions fighting?
     
  18. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    As I say many times here and there on forum, physics will be like in Noedeus demo :) Just now it’s in active rewrite phase :) besause we have some specific cases, ground flat but has cliffs, lakes and rivers and some other specific cases for physics. But it’s still in progress, because I rewrite navigation from nordeus-like to Flow Pathfinding, it’s more preferred in our “massive army” case, which moves like water waves flow
     
    Antypodish likes this.
  19. eco_bach

    eco_bach

    Joined:
    Jul 8, 2013
    Posts:
    1,022
    Thanks for sharing!
    Could you possibly share any performance benchmarks before and after switching to ECS on this particular project?
     
  20. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    655
    Yes, this is the first thing I wanted to show people but, unfortunatley, I write this many monthes after start rewriting, and then I did not know that I would write this post, thus I'm did not record previous performance counts, but, to be honest, compare with previous version is incorrect in our case, because previous game scale (I mean legacy OO approach) absolutlely different, with different count of objects, with different gameplay. Important thing in my post is - ECS can be used in production, ECS gives huge opportunities for developers.