Search Unity

DOTS Sample

Discussion in 'Data Oriented Technology Stack' started by benjaminhwh, Dec 18, 2019.

  1. benjaminhwh

    benjaminhwh

    Unity Technologies

    Joined:
    Dec 6, 2016
    Posts:
    2
    At the Unite CPH 2019 Keynote we showed you a demo of several new DOTS features working together. If you want to take a closer look, we have now released all the packages from the demo and the project itself. Some of the technology is very experimental at the moment. This project is coming hot off the presses and provides a snapshot of where things are right now.

    You can get started by following the README file in the project.

    Have fun and happy holidays!
    Unity

    A few nice links to know:
    DOTS Sample GitHub: https://github.com/Unity-Technologies/DOTSSample
    DOTS landing page: https://unity.com/dots
    The keynote presentation at Unite CPH 2019:

    Unite summary blog post: https://blogs.unity3d.com/2019/09/2...te-copenhagen-keynote-highlights/#Performance
     
  2. AnthonyPaulO

    AnthonyPaulO

    Joined:
    Nov 5, 2010
    Posts:
    109
    Whoa! This is awesome! I really like the ability to make changes on the server and see it reflect on the client, so cool! Can't wait to play around with this, been waiting for this to udpate my project to multiplayer!
     
    Last edited: Dec 18, 2019
  3. pocketpair

    pocketpair

    Joined:
    Jul 7, 2015
    Posts:
    39
  4. pocketpair

    pocketpair

    Joined:
    Jul 7, 2015
    Posts:
    39
    Hi, thank you so much for all your hard work.
    I'd like to ask 1 question, is this project good example for practical usage of com.unity.netcode?
    Or is it better to refer https://github.com/Unity-Technologies/multiplayer example?

    Details:
    I've checked the code, but it looks some codes are from FPS Sample.
    It's fine but I'm worried that it's too old and including legacy codes in term of DOTS, especially compared to com.unity.netcode.
    For example, I took diff "Assets\Scripts\Networking\NetworkClient.cs" between FPS Sample and DOTS Sample.
    It's almost same.
    Scripts/Networking codes looks also similar with FPS Sample one.

    Additionally, Assets\Scripts\Game\Main\ThinClientGameLoop.cs looks almost same with FPS Sample one.
    com.unity.netcode 0.2.0 is already released and Networking codes should follow it even if ThinClient.

    In addition to that ClientGameLoop.cs is totally different from FPS Sample one.
    It should be good but strange since ThinClient code is almost same with FPS Sample one.
    ClientGameLoop.cs doesn't use NetworkClient.cs (It means probably uses com.unity.netcode?) anymore however ThinClientGameLoop.cs still uses original NetworkClient.cs. (It doesn't mean use com.unity.netcode)

    What is happening? This is still under migration from FPS Sample?



    Diff
    A: https://raw.githubusercontent.com/U...er/Assets/Scripts/Networking/NetworkClient.cs
    B: https://raw.githubusercontent.com/U...er/Assets/Scripts/Networking/NetworkClient.cs
    upload_2019-12-19_5-12-21.png


    Diff
    A: https://raw.githubusercontent.com/U...ssets/Scripts/Game/Main/ThinClientGameLoop.cs
    B: https://raw.githubusercontent.com/U...ssets/Scripts/Game/Main/ThinClientGameLoop.cs
    upload_2019-12-19_5-14-41.png
     
  5. Megafunk

    Megafunk

    Joined:
    May 17, 2019
    Posts:
    3
    This is really cool to see everything working together! Will this sample be supported/maintained with future iterations of the DOTS packages? I vaguely remember someone mentioning some of the netcode scripts in here getting moved to the Netcode package at a later date.
     
    jais_unity likes this.
  6. pocketpair

    pocketpair

    Joined:
    Jul 7, 2015
    Posts:
    39
    Also, do you have plan to put detail documents about DOTSSample ? I'd like to learn practical usage of netcode asap.
    Thank you very much for your efforts!
     
  7. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205

    GREAT WORK - Thank you for posting these questions... good luck on the reply
     
  8. filod

    filod

    Joined:
    Oct 3, 2015
    Posts:
    97
    when hit play button (only 1 client with burst on) , it runs at only about 15~30 fps on my machine (3900x+2080Ti), does this sound "Performance by default" to you or i miss something?
    upload_2019-12-19_15-36-19.png
     
    Last edited: Dec 19, 2019
  9. dispatch_starlost

    dispatch_starlost

    Joined:
    Nov 17, 2017
    Posts:
    30
    Running in editor I would get ~8fps until I turned off job debugging and leak detection - 25fps now. I mean its still not great but at least it's usable now. I imagine building a client and server and running outside the editor would be a lot smoother too
     
    mahdad_baghani likes this.
  10. filod

    filod

    Joined:
    Oct 3, 2015
    Posts:
    97
    Cynicat and JohnC_Unity like this.
  11. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Be sure to disable debug exporting, and also jobs and burst debugged per that performance tip in the readme...

    Also be sure you are on the correct unity editor version.

    After what I assume is the initial entity conversion and maybe lighting bakeing, I was seeing good frames on a 8700k/32gb ram/1080ti/m2s

    I have not done a full compile, but that would be a better way to check out performance. I will be diggin in next week ! (Friday, for the normals out there)
     
  12. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    12
    I was taking a look at the DOTSSample and I'm not going to lie, It has become more complex than the FPSSample.
    It is seems to be more modular than before. But everything just seems too much spread out.
    Also as ECS is evolving, I'ts even taking me longer to understand how the code works and what it actually does.

    Also the project should use URP instead of HDRP, as it is just a sample and to understand the FPS NetCode architecture also because I'm having long build times because of those HDRP Shaders, 13.50 min~
     
  13. Kolyasisan

    Kolyasisan

    Joined:
    Feb 2, 2015
    Posts:
    282
    Can I ask why all of the jobs are using .Run() instead of .Schedule()?
     
  14. desertGhost_

    desertGhost_

    Joined:
    Apr 12, 2018
    Posts:
    104
    I ran into a similar problem (Ryzen 5 3600, RTX 2060) in the editor, but performance is fine if you build a client and a server. We were able to get 32 clients and 1 server running across 4 computers at mostly playable frames.
     
  15. Kolyasisan

    Kolyasisan

    Joined:
    Feb 2, 2015
    Posts:
    282
    Strangely though, the built demo runs just fine, just about 7% for the server and 5% for the client CPU load on a Ryzen 3600. But it really is strange that editor has such poor performance, hope they'll improve it substantially.
     
  16. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,834
    Performance in the Editor comes down to a couple things:
    1) Job Debugger, Burst Safety Checks & Leak detection. They can all be disabled in the Jobs menu.
    2) By default all C# code runs in debug mode so that the debugger can be attached (Disabling editor attaching significantly speeds it up)

    (They are listed in the readme for the project)
    With those things configured I can get a solid 60 FPS in the editor on my machine.

    Also note in the editor we run both full server & client worlds & sending packets throught the transport player via sockets. This way in the editor you actually simulate the full setup with two isolated worlds but you go through all the network layers, this is super important for ensuring that you are always testing with the full netcode rig. You can even setup Lag simulation & packet dropping this way while running in playmode.

    In a real game the server & client is on two seperate machines...

    With all that said, it's clear we need to make our debugging tools faster. It's great that they exist and make it so we can safely write multithreaded code but we still need to make sure that all this safety tooling can run at 60 FPS in the editor.


    If you want to look at performance you really want to use the standalone player and run a game session with the amount of players that you want to have in your game connected. In our internal playtests we have done up to 50 Players with solid > 60 FPS. So far it was simply limited by amount of people to get into the playtest. Clearly we are going to scale that up a lot more. Make many moree optimizations etc.

    Also there is a million things we want to improve in the game code. We learnt & already improved a lot of things from going through the process of shipping this sample. This is just the first release, there is still a lot for us to do of course.

    We are not done, but we want to share early for those who want to see how the sausage is made and figure out how a larger FPS game can be structured with DOTS.
     
  17. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,834
    Our goal with this game in particular was that we would like to run the server on a single core. We want to prove that you can have very cheap server simulation code and that implies running multiple games on the same machine.

    For now we do this by using .Run(), in the future we want to use Schedule() and at compile time switch it based on a build setting. So on the client it runs multithreaded and on the server it runs on a single core.
     
  18. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    110
    Thanks for the hard work!

    So you are saying that the server should stay single core ? On a server machine, why would we limit ourselves to only one core ?
    A reason I can think of would be if we want to run one server per core on the same machine, is this what on the roadmap?
     
    AggressiveMastery likes this.
  19. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,708
    Probably. I think problem with multicore server is jobs will fight for resources and some simulations will wait resource freeing until other server instance already processing that, it of course can be solved, but not trivial I guess.
     
    dzamani likes this.
  20. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,834
    This really completely depends on each game. But generally speaking if you have a server with 16 cores, it would be great if we can run the game at 60 FPS on instance of the server pinned to each core. Server cost is certainly a massive problem that we would like to solve with DOTS.

    I am not saying every game should do that. Some games want massive simulation and would like to use all the cores on a machine. DOTS makes that easy. For now the game code mostly uses .Run. But we want to make it configurable at build time instead if you run multithreaded or pinned to a single core.
     
    Last edited: Dec 20, 2019
  21. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    PLEASE do not limit this sample to a single core on the server. That would be ... That would suck, to be limited on the server side. Please keep it at LEAST an 'OPTION' on the server side for multi-threaded server code. Its almost 2020, its crazy to have to say this.

    Thank you for the Great work you have started with the many sample projects, and the great work that comes from teams that use unity. :)

    Micah
     
    Last edited: Dec 21, 2019
  22. Danila24ru

    Danila24ru

    Joined:
    Sep 21, 2012
    Posts:
    10
    Can you provide some documentation on DOTS sample. It's really hard understand how it works. Add some explanation of project structure. How I can edit character, add new animations, game modes etc.
     
    Micz84, transat, filod and 2 others like this.
  23. filod

    filod

    Joined:
    Oct 3, 2015
    Posts:
    97
    DOTS Sample crashes&freaze a lot when i try to change any code... don't know trigger condition...anyone had bad/good luck with tweeking DOTS Sample? i can't even finish change player speed...
     
    Cynicat and AggressiveMastery like this.
  24. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Hello!

    After opening the project with 2019.3.0f3 on windows 10, i'm getting the following error:
    Library\PackageCache\com.unity.burst@1.2.0-preview.11\Runtime\SharedStatic.cs(36,28): error CS0103: The name 'Unsafe' does not exist in the current context

    Any solution?

    Thanks!
     
  25. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Removing, Unity PackageCache seems solved the problem!
     
    johnroodt and AggressiveMastery like this.
  26. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Hello!

    Any luck with 020_Environment element change?

    I added my own assets, its working fine in the editor but its hide my assets in build, seems colliders are at the same place but no meshes just a bunch of messed meshes in the center of the scene.

    Similar no luck with adding new character or weapon, its seems much complex than FPS sample was!
     
  27. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    12
    :D
     
  28. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Its not a problem at all but i can't understand why 10 prefab needed just for the character, its completly a mess without any docu, i understand its just a preview but with a little documentation would be good
     
    AggressiveMastery likes this.
  29. Filtiarn_

    Filtiarn_

    Joined:
    Jan 24, 2013
    Posts:
    130
    I'm having trouble understanding how the animation works.
     
    sybere likes this.
  30. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Same here, AnimationStream.cs GetLocalToRigTR gives: IndexOutOfRangeException: Index -1 is out of range Length 43, i changed all the AnimSources references to my character rig and setted the bone indexes.

    I'm trying to change the character, new character rig bones need to be the same as Terraformer character bones?

    Anyways even the exception, my character is in but stands in T-pose :D
     
  31. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    I was able to create my character and all error are gone, i can move, jump etc but the character still in T-pose so something not good with the anim graph.

    Seems the GhostCollection sequence decide which character load and not the Hero registry as was in the FPS Sample also the nextchar command are not working, its just reload the actual character, nextteam,show.fps, show.compactstats are not working too, they are drawing nothing and show.fps 3 throw exception.

    No matter how i change the Enviroment subscenes, its just put all the meshes into the center of the scene or if im just adding a simple plane its not showing the mesh, collider is there but no mesh in builds, in the editor its working fine.

    Nothing visible from the subscenes on runtime in the editor hierarchy so imposible to track character prefab or other things which are created in subscenes(because of this i can't check the anim graph with the PlayableGraph Visualizer).
     
    AggressiveMastery likes this.
  32. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Howdy Guys and Gals,

    I am starting a devlog into building a game, and using this sample. Seems like a good idea, right? ahHAHAHAHAHHA

    Well, check it out:


    I've been able to add meshes, have them colliding and drawing fine, along with new levels in the menu. Hopefully, it can help, as you are helping me with your comments in this thread tooo.

    Cheers
    Micah
     
  33. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    What i done is just removed the basic environment form the Whitebox_Basic_A 020_Environment scene and replaced my own assets, all the meshes are got the required Physics Shape script and parented to the 020_Environment object which has the Physics Body and Static Optimize Entity scripts, the images shows(first image from build, second from editor) how looks in the editor and how looks in the build, in the editor its playable and the enviroment colliding well but in build only has the colliders and the meshes are not rendered.

    I tryed to add terrain too but similar result.

    Still no luck with Anim Graph, i can't check the player prefab(with Playable graph visualiser) after its created because these subscenes are not visible in the Hierarchy in play mode.
     

    Attached Files:

    AggressiveMastery likes this.
  34. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Seems its a shader/material issue... standard HDRP LIT materials are not rendered correctly, its rendering the meshes in the center of the screen which is really weird, if i'm change my meshes materials to Whitebox_Chrome or Whitebox_Gradient its rendering well in the place where the mesh needs to be.

    There is few provided shaders but similar result with them, except that two what i mentioned so with Whitebox_Chrome and with Whitebox_Gradient.
     
    AggressiveMastery likes this.
  35. Grimreaper358

    Grimreaper358

    Joined:
    Apr 8, 2013
    Posts:
    613
    The default HDRP Lit shader doesn't use hybrid instancing as well as positioning for the world/scene in then Hybrid renderer but a shader made in shader graph can. The shader for the white box level was made in shader graph so that's why they work. but they are very specific to that artstyle so you will have to make your own.

    Open the shader and look at the top three inputs to the master node which shows the positioning setup and in the blackboard you can setup hybrid instancing (Per variable)
     
    AggressiveMastery likes this.
  36. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Agreed! With the prior issue with meshes being rendered in the center of the screen, if that is the camera view/POV, sounds like Instance-Indirect rendering without the adjusted positions needed in the shader. I've got terrain rendering fine, but collisions... i'm currently exporting the terrain to a mesh, then turning that mesh into the new collision shape object. Works ... and easy :) I'll have that in today's video.

    Cheers
     
  37. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Yes that was the problem, but seems there is a bug, because if im adding the meshes with the new shader to the Whitebox_Arena_A environment scene still not rendering well but on Whitebox_Arena_B exactly the same meshes rendering well in the build.

    With character i still had no luck, no errors but its still just in T-pose
     
  38. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Any way to inspect somehow what created on runtem in the PrefabAssetRegistry? I can't find the created character at all because what happening on subscenes are not visible in the hierarchy on runtime.
     
  39. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Hello Sybere,

    I had that issue I think, then I found that the Entity-Debugger-TAB has multiple scenes to it as well, to split up the entities apparently. Click the dropdown pointed to in the attached photo "Editor World" in the Entitiy Debugger Window and move between the optional worlds at runtime/play. I think I was able to find all the entities. You can select Server or Client worlds/groups of entities, as items that do not impact the scene (like a moon in the sky) may not be instantiated in the server-scene. As the server cares not for backgrounds.

    Cheers
    Micah
     

    Attached Files:

  40. sybere

    sybere

    Joined:
    Mar 26, 2015
    Posts:
    176
    Thanks! I already found the Entity Debugger but with this still can't select the player itself in the editor, when i trying to select the player from Scene view at runtime its pointing to the PrefabAssetRegistry object which is the SubScene.

    Anyways i found a solution for character problems, first of all the character meshes need a provided shader which is the Basic_Lit_ColorMask_Graph which has a Vertex Skinning option after that my character are started to move but because of bad bones it was a mess :D

    Another thing what i'm doing is to copy the entire Rig of the Terraformer(i'm doing with Autodesk Maya) to has the proper bones, after this i'm able to play with other characters.
     
    AggressiveMastery likes this.
  41. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Does anyone know, why this project takes forever to 'play' in the editor, after any changes are made, JUST the first time? Then following stops/plays are instant, unless any changes are made?

    I get a 7-14 minute hang when hitting play for the first time the project is opened, then if I hit stop, and play again, it loads almost instantly. I assume some asset registry is generated, but deleted and re-generated if anything is changed in the project.. I'd love to save the compile time between minor changes.
     
  42. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    6,919
    Does project use compiled asmd files? That may be the reason. In Unity 2019.3 and 2020 alpha, this issue seems quite apparent.
     
    AggressiveMastery likes this.
  43. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    eww, I am looking into this. Lots of .asm def files in the project.
     
  44. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    6,919
    From my experience these files slow down compilation. They apparently suppose to be improved, but seems not?
    For me, I avoid them as a fire, as they bring more issues than solve.

    Well, to confirm (as I may be wrong in this case), you can unpack files from asmd(s) and see, if that is indeed the case.
     
    AggressiveMastery likes this.
  45. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Your next post Damn well better be one Hell of one :D
     
  46. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    So I removed Unity Analytics package and it still compiles fine, and I also trimmed down the levels in the build, but I dont know if that had any effect.

    It looks like the Client and Server build when you press play, they say they connect and spawn a player, then the player freezes for 10 minutes. My guess is some time-out that waits to connect up to something for 10 minutes then gives up.

    Anyone have this long delay on play after any changes?

    lol
     
  47. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Turning off Burst, seemed to remove the play delay, but halved the FPS...
     
  48. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,834
    Turn off Synchronous burst compilation. It is generally not recommended to have that on.
     
    Antypodish and AggressiveMastery like this.
  49. AggressiveMastery

    AggressiveMastery

    Joined:
    Nov 19, 2018
    Posts:
    205
    Thank you Joachim, Synchronous Burst Compilation was the issue, and causes a 10 minute "play" freeze after any change.

    I was able to leave Burst on, with Synchronous Burst Compilation turned off, and get back to good performance with an instant editor play button. There is a reduction in FPS for the first few moments, then it goes back up to 70+ for the rest of the gameplay. I assume I could wait for the game to compile (the freeze) or play with reduced fps. Cool!

    I see now in the readme:

    "Be aware that the burst compiler is compiling code in the background so when you enter playmode it can take a while for full performance to be achieved. Turning on Burst timings (Jobs > Burst > Show Timings) will give you a log item for each completed burst code piece. When they stop appearing everything is bursted and performance is as good as it gets. If you prefer, you can also turn on Jobs > Burst > Synchronous Compilation which will wait for all burst compilations to finish before entering play mode."

    Cheers,
    Micah
     
    Last edited: Jan 1, 2020
  50. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    6,919
    @AggressiveMastery I am glad, this is not asmd issue, as per my initial suggestion.
    This spans across few threads now, as far I recall, people having similar performance issues.
    I think, there may be needed more prominent info, regarding this settings and not to be enabled by default (if is?).
     
    AggressiveMastery likes this.
unityunity