Search Unity

Official DOTS Best Practices Guide 1.0 is here

Discussion in 'Entity Component System' started by SteveM_Unity, Mar 14, 2023.

  1. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    The DOTS Best Practices guide has been updated to match the latest Entities 1.0 pre-release. The old versions of the guide have been removed from Learn to avoid confusion, so only the 2022.3 version is available.

    Here's where to find the Guide:

    https://learn.unity.com/course/dots-best-practices

    The revised version of the guide covers a number of changes from previous versions, including:
    • A new section about ISystem, and why we recommend it over SystemBase for new systems
    • Updated code samples throughout, demonstrating usage of the newer API
    • Freshly-recorded performance data comparing different methods for structural changes, with a pretty new graph and an overhaul to the advice about structural changes.
    • Lots of minor tweaks and clarifications

    I hope you find it useful. And, as always, feedback is welcome.
     
    Last edited: Jun 22, 2023
  2. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    upload_2023-3-15_19-58-2.png

    Despite having the OS Language (and browser language) set to English, this page is quite the hodgepodge of German and English.

    upload_2023-3-15_19-59-35.png

    Workaround is to force the browser to US English. Not the best.
     
    Last edited: Mar 15, 2023
  3. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    As far as I know, there are no versions of the guide that we've produced in any language other than English. I assume that the German translations are being auto-generated by Chrome itself. All of this content should definitely render properly as English and not rely on US English. I'll pass this on to the Learn team who are responsible for this kind of thing.
     
  4. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    Blabla about locales, TLDR, no auto translate was used:
    Nope, no auto-translate at play here, I have set both German and English to "never translate" - this is a typical issue that crops up every now and then, for example commonly for people running Windows or Browsers under the en_DE "English (Germany)" locale, but the en "English" locale should always be supported, not just specific child locales such as en_UK and/or en_US.

    In my case, the browser was set to en, [already to circumvent the common en_DE probems, and as I work for a company that uses en_PK "English (Pakistan)" in many places interchangeably with en_UK, I just use the parent locale]

    Unity Learn would only display a non-mangled version if the primary language was set to en_US for me.

    Thanks for passing it on; I couldn't find a good place to give feedback on these things and your guide was quite egregiously affected. :)
     
    Last edited: Mar 16, 2023
  5. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    I've been asked to pass this on by a Project Manager on the Learn team. Let me know if it helps.

    Here are the images they shared of where to set the preferences:

    image (17).png Screenshot 2023-03-16 at 8.37.21 AM.png
     
    daniel-holz likes this.
  6. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    Thanks! I can switch with these links. The real bug here is that the webpage automatically prefers de_DE and defaults to it even though en is the preferred Browser locale. It correctly autodetects en_US, if present and higher up in the list than de_DE.
     
  7. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Okay, thanks. I've passed that on. But I guess it's also an interesting data point that I tried to reproduce this by copying your language settings (on Chrome, on a MacBook), clearing site data and visiting the site in incognito mode, and it's all in English for me. So I guess something else is at play here?
     
  8. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    792
    I have seen several times that many websites often use the country and not the browser language for automatic language selection, which is annoying if you belong to a linguistic minority in a country.
     
    Selmar likes this.
  9. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    More from the Learn team:

    For the record, I tried setting my language settings in Chrome to be the same as yours and opened the site in an incognito window and it rendered in English, which is what we'd expect. If Learn is trying to set a default by asking the browser its location and making assumptions based on that, that's unexpected and they'd need to investigate more.
     
    Last edited: Mar 29, 2023
  10. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    Interesting, it should be asking the browser its preferred locale, not location.
     
  11. CarlosAI

    CarlosAI

    Joined:
    May 9, 2023
    Posts:
    84
    Hi!
    Is there an updated version of the Entities 1.0 release?
     
  12. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    21,203
    Where the Unity version is selectable it says it was updated June 6th.
     
  13. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Ah, yes. I had a conversation with the Learn team recently about how the listed version should be 2022.3 to coincide with the full DOTS 1.0 release, and about how the older versions should be taken off-line to avoid confusion (because the experimental and pre-release versions were all surprisingly different in their APIs and the recommended best practices, and I don't want anyone to end up on the wrong version of the guide and getting bad advice as a result).

    Looks like some of those changes have been made, which has bumped the most recent update date to June 6th, but in reality the content hasn't changed since the version I announced back in March. I've edited the top post on this thread to better reflect the current versioning situation.
     
    mariandev and Ryiah like this.
  14. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    FYI, the Learn team removed all the old versions, so only the 2022.3 (i.e. DOTS 1.0) version is available now. I've edited the initial post in this thread again.
     
    PolarTron likes this.
  15. gabrieloc

    gabrieloc

    Joined:
    Apr 6, 2014
    Posts:
    49
    I have an extremely simple use case that I'd really appreciate some guidance on. I'm reading up on consolidating/minimizing structural changes, and having a hard time avoiding the "worst case scenario" example here.

    For the sake of example, let's say I have a pizzeria containing a PizzaOven system which updates the
    cooked
    state of a bunch of Pizza components. When
    cooked
    is >= 1, I attach a
    RemoveFromOven
    component to each entity containing the Pizza component and another job which queries for that component "removes the pizza".

    Since this is a per-entity structural change, how better would I design my jobs? Would all pizzas have a disabled RemoveFromOven component that simply gets enabled? Is there a better way that I'm just not seeing?
     
  16. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    As with anything else in DOD/DOTS, the answer really depends on your data and how you access it. How many pizzas do you have at any one time? How many of them are in the process of cooking on any given frame? How does the PizzaOven system know which pizzas are in the process of being cooked and which ones are in other states (being delivered, eaten, prepared, etc.)? How many pizzas, on average become cooked every frame? What does "Removing the pizza" actually involve? Does it involve an additional structural change?

    In a VERY general sense, unless you feel that the benefit of the structural change outweighs the cost (i.e. if you have many systems/jobs that will perform significantly better if they can use a query to include/exclude pizzas that are currently in an oven), I think an enableable component seems likely to be a good approach here. In fact in any general scenario where I don't know enough about the specific data or access patterns involved in a problem but I'm asked my gut feeling on whether introducing structural changes are a good idea or a bad idea, my default answer is that they're probably a bad idea. But the devil is in the details.

    If, for some reason, enableable components seem likely to cause you some specific problem (see "When IEnableableComponent is not the answer"), you should refer to "Build NativeArray<Entity> in a chunk-friendly way". Your job that advances the cooked state (or a job that follows it checking to see what's finished cooking) could be an IJobChunk that generates a NativeArray<Entity> with the entities to be tagged in the order they appear in their chunks, for the EntityCommandBuffer to execute as efficiently as possible.
     
  17. gabrieloc

    gabrieloc

    Joined:
    Apr 6, 2014
    Posts:
    49
    I was previously under the impression a Foreach job would query IEnableComponents by presence regardless if they were enabled or not, but some research reveals that actually isn't the case, which fits my use case. Awesome!

    So following the analogy, let's say I'm running the busiest pizzeria in the galaxy with a throughput of millions of pizzas a frame. I need a way to respond to state changes on in-oven pizzas as they're being cooked, and they can be either `baking`, `baked`, or `burnt`. I have a job for baked pizzas and a job for burnt pizzas which necessitate separate jobs. In this scenario, I could have a component representing each state, and a job corresponding to pizzas querying by state, and all pizzas could belong to a single archetype with various state components enabled/disabled. This would make sense, but the "When IEnableableComponent is not the answer" entry seems to imply otherwise:

    in chunks that contain a mix of enabled and disabled components, each component’s flag must be checked before deciding whether or not it should be processed. These additional checks create branches and interfere with Burst’s ability to vectorize code.


    Sorry if I'm missing something, if it helps my game is a pizzeria-like involving many structural and state changes for things like aggregating attack damage, triggering audio and effects, and many other things which seem to affect performance. Really appreciate the guidance!
     
  18. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    It sounds like you're describing a state machine for the pizzas. A pizza (presumably) can't simultaneously be "baking" and "burnt" at the same time, it will progress from one state to the next at certain transition points. So instead of multiple enableable components, it probably makes sense to have a single component called something like PizzaState.

    The next question is what to do with that PizzaState component. Our internal tests show that except in unusual edge-cases, the fastest way to process multiple entities with state machines represented in this way is just a single job with a switch statement inside it:

    Code (CSharp):
    1. switch(pizzaState)
    2. {
    3.    case PizzaState.Baking:
    4.      // Do baking logic
    5.      break;
    6.  
    7.    case PizzaState.Baked:
    8.      // Do baked logic
    9.      break;
    10.  
    11.    case PizzaState.Burnt:
    12.      // Do burnt logic
    13.      break;
    14. }
    You said that you need different jobs for the different states, but didn't really explain why. It's probably worth examining that requirement, because if they really do have to be different jobs you'd want to implement them as IJobChunk and have your system launch a job for each state with an EntityQuery matching all of the pizzas, and then have each job check each pizza to see if it's the required state for processing. This might be fine, but launching multiple jobs over the same set of entities seems kind of redundant in terms of job scheduling overhead and managing dependencies to guarantee job safety. To mitigate that, you would ideally combine states into as few jobs as possible, so perhaps you absolutely need BurntPizzaJob to be its own thing, but perhaps you can combine BakingOrBakedPizzaJob into one job that can process both states, and only launch 2 jobs rather than 3.

    As you point out, Burst vectorization failing due to branches is a potential issue in all of the approaches that avoid structural changes. Whether it's actually a problem will depend quite a lot on the nature of the work done in the jobs, whether it's heavy number-crunching or just incrementing timers, etc... In situations where the profiler tells you that the lack of vectorization is hurting performance, perhaps it would be possible to refactor the jobs such that the logic happens inside a branchless job that applies to all of the pizzas regardless of what state they're in. Whether or not the savings from removing branches and enabling vectorization (and/or explicitly making use of SIMD intrinsics) outweigh the redundant work that will be performed for pizzas in states that don't need the data will be very specific to your use-case, and something that really only a profiler can tell you.

    Finding a good solution to these sorts of problems is always going to be a fundamentally profiler-driven process. You make a best guess at a first approach, profile, and refine if the performance is not what you were hoping. That's why the Key Principles say you should embrace iteration. If I were building the first attempt at this, I'd probably use a PizzaState, and the smallest number of jobs required to process all of the states - ideally just one job, with the switch statement. I wouldn't worry too much about vectorization to begin with, unless the profiler showed that the initial approach needed improvement.
     
    bzor, daniel-holz and apkdev like this.
  19. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    867
    Like @SteveM_Unity says using a switch statement is a good starting point. If you have a PizzaState enum nothing is stopping a later Job or system from setting the state as needed. For example, in a later system, you can set the state as burnt using an if statement. You can also combine multiple states to get more complex behaviors.

    Basically, you should not have a component representing each state. Each state variable could be an enum. Not all states need to be an enun. But they should be data. A float or int for a timer state is best. They probably should be in the same component. It depends on whether your finite state machine can only have a single active state at any given time. Or if several different states are allowed simultaneously.

    In addition, state machines that depend on other state machines are possible. ECS uses data to define the current game state and is conducive to very complex state machines. Because ECS is primarily a way to organize and transform data it is generally best to explicitly define this date rather than using tag or state components. Instead of a tag use a bool if you are going to access that component anyway.
     
    Last edited: Oct 10, 2023
    apkdev likes this.
  20. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    I've been discussing this with colleagues and one of them had an interesting thought about this scenario:

    With a million pizza ovens on the go at once, it's not unreasonable to imagine that on any given frame, there could be an entire chunk (or more than one chunk, or close enough to a whole chunk that we can tolerate bit of wasted memory) of pizzas being put into ovens at the same time. If the baking process is deterministic then those entire chunks will change state at the same time, and that assumption can lead to a number of interesting optimizations.
     
  21. gabrieloc

    gabrieloc

    Joined:
    Apr 6, 2014
    Posts:
    49
    There are some excellent points being made here, I really appreciate the guidance from the two of you. It certainly makes sense to centralize state transitions. I'm worried I've mischaracterized my original issue with a faulty metaphor.

    Let's say in this pizzeria, there is a phenomenon in which anything reaching a certain temperature inside the oven explodes, including calzones and anything else I may be using the pizza oven for.

    In my project, I have a component called `ExplodeTrigger` which gets added to all explodable things. The component gets picked up in a different system entirely which reacts to explosions in a number of ways, none of which concern the pizza oven. This is a state which presumably need not concern our `PizzaState`.

    I could have all explodable things already have an ExplodeTrigger which gets enabled later on. But presumably there would be quite a few archetypes now, especially as I enable more and more things to explode.

    This barely sounds like a fringe use case, but maybe I have the wrong architecture entirely? Would it make more sense to instead invert the relationship and have the Explosion system query all explodable things always and check a flag to determine whether they need exploding? In my head this feels wrong because the system would be doing a lot of work for (potentially) nothing.

    Thanks in advance.
     
  22. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    I'm not quite sure what the new question is. Are you asking whether "ExplodeTrigger" should be:
    • A component that gets added to an Entity to indicate that it should explode this frame, or
    • An enableable component whose presence means than an Entity can explode, but which is disabled until it's time for the explosion to actually happen?
    If that's the question, I'd say go with the second option. You avoid the problem of introducing many new archetypes, you avoid the performance cost of the structural changes from adding the ExplodeTrigger component at runtime, and all it costs you is 16 bytes of memory per chunk and a small performance hit in your "MakeStuffExplode" System, or whatever you call it.

    What I'm saying is: Yes, do this. Make the flag you check be the whether or not the component is enabled. In cases where there's nothing about to explode in a chunk, the job will skip over the whole chunk in a single check. In the (I guess pretty rare) cases where everything in a chunk should explode, that's maximally efficient as well. In other cases, as the guide says:

    A system/query being (at worst) 2x slower because it uses enableable components compared to one that doesn't is a drop in the ocean compared to the cost of the structural changes you'd likely need to make in order to avoid that cost. If it turns out your MakeStuffExplode system is actually a performance bottleneck, the code can always be refactored: perhaps instead of enabling components you just populate some NativeArray<Entity> with the entities to be exploded in a given frame and have MakeStuffExplode iterate over that instead of running queries - although it's conceivable that the cache misses you'd introduce with that approach might make things worse.

    Honestly, life's too short to fret over every system unless the profiler tells you that you need to. Software development is always about compromises, and it's simply not possible to make every single system in a non-trivial ECS project all operate at maximum efficiency. So you design for the common case, focus on the hot code paths, and do whatever's going to move your project forward now, even if you have to revisit and optimize your solution later. DOD is about building something that's comparatively easy to optimize compared to OOP, but it's not about agonizing over every decision trying to get everything maximally optimal first time, because that's basically impossible.
     
  23. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    867
    Don't try to force even thing into one component. You need a way to query entities in the oven. This is different from a state machine. You could use an entity query or a physic query. Generally, when stuff is exploding, I am using a physic query and then setting states.

    You could add an enable component called InOven. This kind of solution will not scale well, imagine you have ten or twenty ovens. I would keep the state machines separate from your spatial partitioning. You may need to make an optimized data structure for spatial partitioning. The oven is a place. Baking is a state. You can be in the oven and not baking. For a simple oven game, you could use a NativeMultiHashMap or a couple NativeList.
     
    Last edited: Oct 12, 2023
  24. gabrieloc

    gabrieloc

    Joined:
    Apr 6, 2014
    Posts:
    49
    @SteveM_Unity
    This is a great takeaway.

    For additional context, I'm at the final stage of my game, where I'm experiencing dots telling me I'm using too much archetype memory (which is baffling as the Archetypes editor window reveals I'm using significantly less than the high water mark). I think I have a general idea for where to invest in next, thanks a lot for taking the time to clarify what was a big question mark in my head!
     
  25. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Warning: This is a pretty speculative response. Please take it with a pinch of salt, and correct me if you know I'm wrong.

    I'm not super familiar with the archetype memory handling, but certainly in older versions of Entities, every time a new archetype was encountered at runtime it would have to register itself in a data structure of all known EntityQueries in order to quickly identify which archetypes (and therefore which chunks) matched a particular query, so every archetype would occupy some memory for the rest of the runtime duration even if it was only ever used once.

    I don't know whether or not that's still the case with Entities 1.0, but if it is, perhaps those messages are warning about the high water mark rather than the number of archetypes that are actually present at any given moment?
     
  26. gabrieloc

    gabrieloc

    Joined:
    Apr 6, 2014
    Posts:
    49
    Oh god, that sounds highly plausible. There's a thread on it over here with no resolve. And yes I'm on Entities 0.51.

    I've been slowly removing all runtime prefab conversion (which also happens to be a prerequisite for upgrading to 1.0), with the hope that registering prefab archetypes at buildtime may alleviate my memory problems. But if indeed stale/unused archetypes never get cleaned up I'll only be prolonging the issue and not solving it.

    Now I'm really curious if also upgrading to 1.0 will actually solve things.
     
    bitgamestudios likes this.