Search Unity

Official DOTS Best Practices guide (0.50, 0.51)

Discussion in 'Entity Component System' started by SteveM_Unity, Jun 29, 2022.

Thread Status:
Not open for further replies.

  1. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    We've published a revised version of DOTS Best Practices. The initial version was based around Entities 0.16/0.17, so the new version brings us up to date, to Entities 0.50/0.51. Because the Unity Learn platform doesn't let you select different versions of a guide by selecting a package version, the revised edition is shown when you select the Unity version of 2021.3, or by default.

    https://learn.unity.com/course/dots-best-practices

    The revised version of the guide covers a number of changes, including:
    • Some more information on DOD fundamentals
    • Updated information on debugging tools
    • Changes in code samples to reflect changes in the API
    • Updated benchmark test results to show newer hardware and changes in performance characteristics in newer version of DOTS, and to hopefully present the information more clearly
    • Tweaked advice about structural changes, as a result of changing performance characteristics.
    • A new section discussing job/system granularity. Unity users have uncovered performance overheads in the job scheduler, in job/system dependency management and in repeated work performed by multiple systems. These issues had not been on my radar when I wrote the first version of the guide. Some of them will be addressed in future versions of Unity and/or the DOTS packages, but the DOTS team and I recognized that these are important issues when working with the current versions of the packages
    • A short new section with some useful resources on Burst Intrinsics
     
    SteveM_Unity, Jun 29, 2022
    #1
    chemicalcrux, JohngUK, Ng0ns and 19 others like this.

  2. Krajca

    Krajca

    Joined:
    May 6, 2014
    Posts:
    347
    Hi, I catch a few errors:
    @1.1 "Beach Ball Simulator: Special Edition - it’s more fun than it looks. Just wait until the player starts moving those green balls around!" - funny joke as it's a .png :D
    @2.1 "Breakout Data Design Worksheet" link is broken - a spreadsheet is private
    @2.3 " learning resources" link is also private

    I will add more if I found something else.

    EDIT1
    @2.6 "Key Principles" link is also private
     
    Last edited: Jun 29, 2022
    Krajca, Jun 29, 2022
    #2

  3. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    I'm glad you found the caption funny, but I'm not sure what the error is? It's meant to be just an image, not a link to a playable game or anything.

    Oops. I'll get this fixed. There's a PDF version of the breakout worksheet linked at the top of the page, under the "Materials" heading, which I think should also be linked from the text rather than the original source spreadsheet.
     
    SteveM_Unity, Jun 29, 2022
    #3

  4. Krajca

    Krajca

    Joined:
    May 6, 2014
    Posts:
    347
    Sorry maybe it's my English but it sounded like an invitation to watch short animation.
     
    Krajca, Jun 29, 2022
    #4

  5. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Thanks for the feedback, Krajca. The breakout worksheet link should be fixed to point to the PDF version now. From your other feedback we spotted a bunch of links that ran into problems during the upload/publishing process which are getting fixed as we speak. These links should basically just point to other sections of of the guide, so in the meantime it shouldn't be too hard to figure out what information the links are trying to direct you to.

    I'm sorry you got confused about Beach Ball Simulator - I tried to make it clear that it was an imaginary game that doesn't really exist, but I can see how you might think that's not a static image. I will think about how to present that better, probably in the next revision of the guide.
     
    SteveM_Unity, Jun 30, 2022
    #5
    Krajca and Anthiese like this.

  6. Krajca

    Krajca

    Joined:
    May 6, 2014
    Posts:
    347
    It made sense after reading the guide further :D

    It's a great guide. Especially the optimization example at the end. Such specific informations are in scarce in external sources and mostly in C++. Thank you for your work!
     
    Krajca, Jun 30, 2022
    #6
    SteveM_Unity likes this.

  7. Elapotp

    Elapotp

    Joined:
    May 14, 2014
    Posts:
    98
    Thank you, man! For me, it was really useful. A lot of gaps were closed and now I have more connected knowledge of how it works under the hood
     
    Elapotp, Jul 2, 2022
    #7
    JesOb likes this.

  8. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    306
    If I may add, on the section of "Know your memory allocators" it is recommended to use the [DeallocateOnJobCompletion] attribute on "pass[ed] Native containers to the job".

    I believe this is misleading. Specifically, this tag does not function for collections other than the basic NativeArray and nothing in the package Collections supports use of it.

    NativeLists, NativeHashMap, and so on must be disposed off using the 
    .Dispose()
    method following a job. For cases where disposal must be done following completion of a job without manual checks, a disposal job for the corresponding native collection can be done with 
    .Dispose(UsingJobDependency)
    . Using the attribute (or the extension in the ForEach lambda) will result in a invalid operation exception.

    Furthermore, IIRC several years back when this attribute was first introduced, it was determined that making it function for other containers was too much of a hassle and made debugging difficult since no explicit deposal method was called in the job scheduling code. I cant find the specific forum post from a unity employee (and this was back in the 0.15ish era so long time ago), but the use of the tag was not recommended and should be replaced by the Dispose methods with job dependency parameters.

    Is this no longer the case? Is this tag planned to be expanded to other types of collections? Will the disposal method of native containers be de prioritized to this tag when deciding how to dispose unneeded collections?
     
    Kmsxkuse, Jul 2, 2022
    #8
    thelebaron and Elapotp like this.

  9. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,759
    I personally would like DeallocateOnJobCompletion to be removed as it is not universally supported all it does is add confusion and obfuscation. It's a relic from the days of long past.
     
    Last edited: Jul 4, 2022
    tertle, Jul 4, 2022
    #9
    ElliotB, apkdev, Kmsxkuse and 1 other person like this.

  10. Arnold_2013

    Arnold_2013

    Joined:
    Nov 24, 2013
    Posts:
    285
    In section 3.2.4 it is stated

    "The graph above clearly shows that using an EntityCommandBuffer to make structural changes is slower than using EntityManager directly, especially when you’re working with an EntityQuery."

    Which in this graph it does, but in the downloadable "Materials Performance_Table_Example.png" the values of EntityManager and ECB are very similar.

    I guess its not the same test case, or one of the two is outdated?
     
    Arnold_2013, Jul 8, 2022
    #10

  11. happy253650035

    happy253650035

    Joined:
    Jan 8, 2013
    Posts:
    2
    Find a bug, tutorial Part 2 No7. the link of "FixedString32" and "FixedString64" cannot work upload_2022-7-8_17-14-26.png
     
    happy253650035, Jul 8, 2022
    #11

  12. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Thanks folks (and apologies for the delayed response, I was on holiday). I've submitted an updated version of the guide which should hopefully go up at some point soon. The revisions are:

    • Remove recommendation to use [DeallocateOnJobCompletion], and instead recommend Dispose() with a JobHandle and point out the problems with the attribute.
    • Remove the link to Performance_Table_Example.png in newer versions of the guide - it shows performance information from older versions of the DOTS packages and guide.
    • Changed the text and links about FixedStrings. I hadn't spotted that they have been renamed to FixedString32Bytes, FixedString64Bytes, etc.
     
    SteveM_Unity, Jul 14, 2022
    #12
    Kmsxkuse, zandalus, PhilSA and 6 others like this.

  13. Armitage1982

    Armitage1982

    Joined:
    Jul 26, 2012
    Posts:
    38
    Part 3.4
    I think it is in this chapter that you should propose 2 short examples for EntityManager and NativeArray<Entity>.

    Also, it would be nice to mark a clear division between this one and the previous one about "Settings Flags".

    I mainly use Flags in my Entities.ForEach(), which is the fastest solution (for the moment if I'm not mistaken).
    .
    However, as that previous chapter ends with "This approach has some downsides", one would be tempted to think that better solutions will be explained. When in fact it talks about Structural changes.

    I had to reread this part several times to clear up my doubts.


    I hope what I am trying to say makes sense.
    Otherwise, this course is a must. I need the same for Burst.
     
    Armitage1982, Jul 16, 2022
    #13

  14. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Although there aren't code samples here, I hoped that the table above contained adequate descriptions of the approach:

    EntityQuery (EQ): An EntityQuery is passed to the AddComponent() call to specify which entities should have the component added to them

    NativeArray (Array): A NativeArray<Entity> is passed to the AddComponent() call to specify which entities should have the component added to them

    The structural changes section was the hardest to write in the first revision, and was the hardest part to rewrite for this revision. I expect it will see another major rewrite in the next revision as well.

    Real talk: structural changes have turned out to be a bigger and more complex problem from a performance standpoint than we anticipated when the first versions of ECS were conceived. Many of these problems only became apparent when we started seeing DOTS in use in full-scale production code (as opposed to the "10,000 spinning cubes" type scenarios which are often used to test, verify and demonstrate behaviour as the API is being written). Seeing these issues "in the wild" was a big part of the motivation for me writing the guide in the first place, and the benchmark tests I wrote for this part of the guide have now been adapted as part of our internal testing framework to check for performance regressions between versions. The Entities team are still actively working on improvements in this area, which means that the performance characteristics and APIs around structural changes are still changing from version to version. For Entities 0.50/0.51 and for future versions until things settle down, I'll have to re-run these tests, arrive at new conclusions about the performance characteristics, and make the best recommendations I can.

    Often, these recommendations are written with very careful language ("Prefer", "Consider", "Understand", "Investigate") because the best approach is very situation-specific. Always, I have to consult with the Entities team to ensure I don't make recommendations that will land people in trouble when newer versions land, even though I can't make specific concrete promises/predictions about what will change in the future, or when. This version of the guide is in a particularly strange position, where mainly all I can do is point out the performance and the potential problems with each approach and try to persuade people not to do anything which will make their lives harder in the future.

    It is my hope (and from what I've seen, my strong belief) that future versions of the guide will be able to make this section shorter, and with clearer advice. I'm looking forward to that :)
     
    SteveM_Unity, Jul 16, 2022
    #14
    optimise, Armitage1982, Enzi and 3 others like this.

  15. Armitage1982

    Armitage1982

    Joined:
    Jul 26, 2012
    Posts:
    38
    Thank you for this clarification.

    It's clear that restructuring all this information is a huge job. After all, it is learning a new paradigm coupled with a new framework using a new compiler.
    I must admit that, as far as I'm concerned, I need examples as much as written explanations. Videos are not really my cup of tea so this guide is invaluable.

    Also, it's not a big deal, but could you or someone point out to the team working on Unity Learn that the site is not responsive (cf. image). The code in the code boxes is truncated, it's not ideal on desktop.


    Thanks again.
     

    Attached Files:

    Armitage1982, Jul 20, 2022
    #15
    zandalus likes this.

  16. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    I know the Learn team are working on new formatting for the best practice guide content that's on there. I'll pass this feedback on, because - yeah, it's not a great use of screen space.
     
    SteveM_Unity, Jul 20, 2022
    #16
    zandalus, Thygrrr and Armitage1982 like this.

  17. chemicalcrux

    chemicalcrux

    Joined:
    Mar 16, 2017
    Posts:
    720
    This was very helpful for me! I've got a much clearer idea of how to proceed with the ECS now.

    (also, i appreciated being able to read everything!)
     
    chemicalcrux, Sep 24, 2022
    #17
    SteveM_Unity likes this.

  18. djhatvr

    djhatvr

    Joined:
    Sep 22, 2019
    Posts:
    53
    Has anyone been able to get a .51.1 build to work on the Quest 2. I can't get a build to work even though it works in editor and on Rift S
     
    djhatvr, Sep 30, 2022
    #18

  19. hakankaraduman

    hakankaraduman

    Joined:
    Aug 27, 2012
    Posts:
    354
    Does this guide also apply to new experimental 1.0 release?
     
    hakankaraduman, Oct 1, 2022
    #19

  20. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,770
    Many will.
    But there are breaking changes in various areas, which affects how things work.

    I.e. with ISystem you will be able to burst system, or entity manager.
    Transforms changes.
    Entities.foreach stays for now, but is going to be replaced.
    Adding /removing components tags, don't imply structural changes. (Edit: I mean enable/disable components tags don't need structural changes).
    Some IJobs changes boiler plate. Plus there is coming mask.
    Etc.
     
    Last edited: Oct 1, 2022
    Antypodish, Oct 1, 2022
    #20
    hakankaraduman likes this.

  21. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    2,683
    They of course imply structural changes. Switching enablable components state doesn't imply structural changes.
     
    eizenhorn, Oct 1, 2022
    #21
    Antypodish likes this.

  22. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,770
    I mean you are right. I described this vaguely. What I meant more in the line, that there are components tag enable/disable state in 1.0.
     
    Antypodish, Oct 1, 2022
    #22

  23. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    2,683
    It's not only for component tags, but for any ICD and IBED :p
     
    eizenhorn, Oct 1, 2022
    #23
    Antypodish likes this.

  24. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    Short answer: No.

    Longer answer: I'm going to revise the guide for the 1.0 release when I have time (writing best practice guides isn't my main job at Unity, more a side-hustle). In practice this probably means early next year rather than before then. There are obviously a lot of changes in 1.0, and whilst some of them probably won't be mentioned in the guide at all, other changes might result in quite significant changes to the guide. I won't know for sure until I get time to sit down and get hands-on with the new version and book some time speaking to the folks who built it.

    That said, I was aware of a few of the 1.0 changes that were coming when I wrote the 0.51 revision, and although I couldn't talk about them directly, I did try to make sure that the guide didn't contain advice that would make lives more difficult when updating to 1.0.

    This list is a pretty good approximation of exactly the things I'll be thinking about when I go in for the next draft.
     
    SteveM_Unity, Oct 3, 2022
    #24
    Sirius64, calabi, Kmsxkuse and 3 others like this.

  25. SteveM_Unity

    SteveM_Unity

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    41
    SteveM_Unity, Mar 14, 2023
    #25
Thread Status:
Not open for further replies.