Search Unity

Unity NetCode 0.0.2-preview.1

Discussion in 'NetCode for ECS' started by timjohansson, Nov 29, 2019.

  1. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The first preview version of the Unity NetCode package has been released and is now available in the Unity package manager.

    Package documentation
    Samples (See Asteroids and NetCube)
    Unite presentation about NetCode

    What is it?
    The Unity NetCode package is our implementation of a server authoritative multiplayer model with client prediction for DOTS. It is the package we are using in the DotsSample from Unite Copenhagen. In this model all simulation is happening on the server and the NetCode is synchronizing networked objects - ghosts - to all clients. The clients send inputs to the server and it is also possible to send light-weight RPCs for control flow.

    We have been running playtests with 30-40 players from around the world with it regularly, and also a few playtests with more players. With the compression we have in place the bandwidth is usually well below 100kbit/s on the client in our playtests.

    What is it not?
    This is the first preview of the new NetCode package. As such it is not a full featured solution, the main focus has been on getting the core of it solid rather than making it feature complete or easy to use. There is also not a lot of documentation available yet.

    This is not the only networking model we plan to build, but it is the one we are building first. Other networking models will follow once this one is more mature.
     
    Mikael-H, RogDolos, Grizmu and 14 others like this.
  2. Soaryn

    Soaryn

    Joined:
    Apr 17, 2015
    Posts:
    328
    As my usecase for Unity is rather abnormal, I'd assume this doesn't have a DOTS solution to connecting to an existing TCP secure websocket correct? First glance of the documentation would indicate it has nothing related to that, but thought I might ask on the off chance there was something already built within the package :)
     
  3. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    962
    Yay! Thank you very much! Time to update!
     
  4. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    We do not have any TCP socket support right now so websockets will not work. Should be fine to use regular C# sockets as long as you do it from the main thread though. We are actually doing something like that for our NetDbg tool in the NetCode.
     
    tonialatalo likes this.
  5. Birgere

    Birgere

    Joined:
    Feb 8, 2016
    Posts:
    22
    I do not seem to be able to find the package in the package manager (even though I have preview-packages enabled).
    Nor am I able to find any release / pre-release of Unity 2019.3 in Unity Hub, is this an issue anyone else is experiencing?
     
  6. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    962
    Tim, can you give me a summary of the bigger changes that happened with 0.0.2?
    If you can't, no biggie, just interested.
     
  7. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    15
    There's no code for lag compensation, is there?
     
  8. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Most of the changes are listed in the changelog ( https://docs.unity3d.com/Packages/com.unity.netcode@0.0/changelog/CHANGELOG.html )
    If you want changes from the old github repo you need to look at both 0.0.2-preview.1 and 0.0.1-preview.6.
    We did develop most of it in the DotsSample repo and were not very strict about updating the changelog so I'm pretty sure it is missing a lot of things - but the most important ones should be there.
     
    BelkinAlex and Enzi like this.
  9. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The lag compensation is still part of the DotsSample code - but we will be moving it into packages (part of it will be in netcode, we do not have a final decision on if it will be all or if some part will live elsewhere due to physics dependencies). If you remind me I can point you at the code once it is released so you can get started before we move it in to the package, it is not a lot of code.
     
  10. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    15
    If I wouldn't have forgotten, then I will surely remind you :p. Thanks for that though.

    By the way, any ETAs on when the sample project will be released?
     
    Last edited: Nov 29, 2019
    AggressiveMastery likes this.
  11. SebLazyWizard

    SebLazyWizard

    Joined:
    Jun 15, 2018
    Posts:
    233
    I can't find the package either, I'm on version 2019.3.0f1 though.
     
  12. Kichang-Kim

    Kichang-Kim

    Joined:
    Oct 19, 2010
    Posts:
    1,011
    How about com.unity.transport package? Is that package is available via Package manager too? Also, Is it safe to use com.unity.transport alone without netcode package?
     
  13. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Yes, it is available as com.unity.transport 0.2.1-preview.1 and you can use it without using netcode.
     
    Kichang-Kim likes this.
  14. Jimmy_Lin

    Jimmy_Lin

    Joined:
    May 8, 2015
    Posts:
    10
    "com.unity.entities": "0.2.0-preview.18",
    "com.unity.netcode": "0.0.2-preview.1",
    "com.unity.rendering.hybrid": "0.2.0-preview.18",
    "com.unity.test-framework": "1.1.8",
    "com.unity.transport": "0.2.1-preview.1",
     
    SebLazyWizard likes this.
  15. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    962
    I have the new netcode up and running and it was a much better process than the last time. I like most new changes. Less code-gen, no seperate prediction systems.

    Sadly I still ran into an older problem I couldn't quite figure out. There seems to be a 1:1 relation between a type of ghost and its snapshot to a (converted) model.
    If I want to have several moving platforms with a generic mover ghost I don't know how to make them have seperate meshes. There doesn't seem to be a pipeline for this and I'm wondering if it's better to wait because there is something up and coming that solves this or roll my own implementation.
     
    Mikael-H likes this.
  16. Wobbers

    Wobbers

    Joined:
    Dec 31, 2017
    Posts:
    55
    GameObjects with GhostAuthoringComponent are converted twice in each of my client worlds, so I have two Entities per GameObject per World.
    Is there a way to stop that from happening? Do I have to uncheck all the boxes for Interpolated Client / Predicted Client manually?
    Why would a component need to be interpolated and predicted at the same time?
     
  17. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Any plan to add support for hybrid ECS (i.e. Convert and Inject Game Object conversion mode) for GhostAuthoringComponent? At least expose Convert and Inject Game Object conversion mode selection option just like how Convert To Entity does. Currently I have project implement in hybrid ECS way and I not able to use this netcode package as it needs to be in pure ECS. I think until there is most of Unity features available in dots, it's must have feature.
     
  18. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The easiest way to solve this is to have system on the client which changes or adds the mesh based on other data in the ghost.
    So you would add a MeshTypeComponent or something like that which you create yourself to synchronize a mesh index. On the client you run a system which looks at the mesh index in the MeshTypeComponent and changes the mesh based on that.
    You could also mark the MeshComponent as not present on client and only run the system once for entities which have a MeshTypeComponent but not a MeshComponent and add the MeshComponent with the correct mesh based on the type.
     
  19. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The ghost is convert to two entity prefabs, one for interpolated and one for predicted. These are just prefabs and not actual entities, they will not show up in entity queries. When you receive a new ghost it will be instantiated from one of these two prefabs based on which type of ghost it is. A single instance of a ghost is never interpolated and predicted at the same time, but one ghost type can have both interpolated and predicted ghost instances.
     
  20. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Convert and inject is not something we are currently planning to do. I am not ruling out that we might do something like that in the future - but not short term. Ghosts are converted to entity prefabs which are instantiated on the client side. That means the prefabs themselves cannot be hybrid.
    It is just the synchronized data and prefab that needs to be pure ECS. You can still create hybrid games with it, but it requires some extra code. You would need to have a component on the ghost which a system on the client or server looks for and creates a GameObject or other hybrid data based on. So instead of creating a prefab with hybrid components on it you would create it with a NeedsHybridComponent and run a system which creates the required GameObjects / components for entities with that component.
     
    optimise likes this.
  21. Flipps

    Flipps

    Joined:
    Jul 30, 2019
    Posts:
    51
    Tried to manipulate the child data of a GhostAuthoringComponent but the checkboxes are disabled.
    I also disabled the root objects translation/rotation first in case there are any depdencies.
    Is this a bug or i am doing something wrong?
    Using the newest packages.

    upload_2019-12-1_16-0-45.png
     
    Mikael-H likes this.
  22. Wobbers

    Wobbers

    Joined:
    Dec 31, 2017
    Posts:
    55
    Oh, I see. The Problem is that one authoring component on my ghost prefab adds a singleton to the entity and thus I get an error that the singleton exists twice.

    Maybe I am just using the authoring components wrongly. Probably should add that one on Instanciate then. Haven't used entity prefabs yet. Thanks for the clarification!
     
  23. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Removing components from child entities is not supported. The checkboxes will strip the component from the entity when it is generating the entity prefab, so instance of the ghost does not have that entity at all if it is unchecked. We only do that stripping of the main entity.
    If you just want to disable sending of it you can check the checkbox for "Manual field list" and remove "Value" from the list. That way it will not send any data to synchronize the translation / rotation. The components will still be on the entity, but in the case of transforms they would if you were not using the netcode package too.
     
    Mikael-H likes this.
  24. Gorgor

    Gorgor

    Joined:
    Apr 28, 2012
    Posts:
    51
    Me neither. Can someone explain please? Is this package present in 2019.3.0f1 or it is only in 2020 alpha?
     
    Birgere likes this.
  25. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    962
  26. Gorgor

    Gorgor

    Joined:
    Apr 28, 2012
    Posts:
    51
    I have 2019.3.0f1 which should be the newest version. Yes, I have checked "Show preview packages". I can see there for example:
    "com.unity.entities": "0.2.0-preview.18",
    "com.unity.rendering.hybrid": "0.2.0-preview.18",
    "com.unity.test-framework": "1.1.8"

    ....but
    "com.unity.transport": "0.2.1-preview.1",
    "com.unity.netcode": "0.0.2-preview.1",

    is nowhere to be found.
     
  27. Djayp

    Djayp

    Joined:
    Feb 16, 2015
    Posts:
    114
    2019.3.0f1 >> I've been able to install these packages through the manifest.json but I couldn't find them in the Package Manager too.
     
    Last edited: Dec 2, 2019
  28. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    962
  29. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    15
    There's still no implementation of support for fragmentation
     
  30. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The package works with 19.3 b11 and later. It seems the package is not listed in the package manager window right now, I'll investigate that and in the meantime you can get it by manually adding it to Packages/manifest.json
     
    pal_trefall likes this.
  31. PrisedRabbit

    PrisedRabbit

    Joined:
    Aug 14, 2012
    Posts:
    63
    Just tried the samples and it all unusable on MacBook Pro 15 2012 (non retina) (10.14.6, Metal)

    upload_2019-12-3_21-37-22.png
     

    Attached Files:

  32. pocketpair

    pocketpair

    Joined:
    Jul 7, 2015
    Posts:
    72
    @timjohansson
    Hi, I'd like to ask 1 question.
    Is DOTS netcode for all people who creates networking game with unity?
    or just for existed/future DOTS users?
    We are currently working open world TPS game with unity but didn't use DOTS a lot.
    I'd like to know whether we should use dots netcode or we should find another way.
     
  33. Jawsarn

    Jawsarn

    Joined:
    Jan 12, 2017
    Posts:
    245
    Not sure if helpful to anyone. Trying to extend the package with some of my own worlds for other servers (like Login, Communication, Matchmaking). Internal servers only use RPCs. I generate simulation/initialization groups, and extract the RPC network part of the NetCode and add it in specified folders. Parse and remove unneeded code (which is very prone to what they might update it with). (But hopefully this will reduce my refactor time in future x)) This is the window script https://pastebin.com/TLSek3F2

    Some things I've noticed for extending are that "Runtime/ClientAndServerWorld" could almost be it's own package which netCode is dependent on, could make using your own worlds and bootstrap a bit easier by simply replacing it with your own package. "UpdateInWorld.cs" should seperate the systemGroups into their own scripts.^^ I don't like the "UpdateInWorld" attribute as a compilment of "UpdateInGroup" as it can express same thing, but I can see how it is problematic without it as well. I've yet come to overwrite the bootstrap yet I'll see how that works out with combining it with my "UpdateInExtendedWorld".

    Thanks for the awesome work! : )
     
    Mikael-H likes this.
  34. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    I have never seen it be that slow on any machine, what does the timeline view of the unity profiler look like? Does it get better if you enable synchronous burst compilation from the jobs menu?
     
  35. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The netcode is built as a part of dots so you will need to use dots for parts of your game to use it.
    You can still write most of your client with game objects, but you need to keep the things you synchronize and the prediction code in entities.
    The server needs to be all dots if you want to take advantage of the new workflows running both client and server at the same time in the editor.
    It is still in early preview so you should expect breaking changes, and when you intend to ship also affects the decision on using it or not.
     
  36. Flipps

    Flipps

    Joined:
    Jul 30, 2019
    Posts:
    51
    Thanks for the nice explanation.

    I am struggling at creating a build for client only. What is the best way to do this (for now)?

    My thoughts:

    1. Using the new Build Pipeline with the new NetCodeConversionSettings. I think this is not implemented yet because i get a build where client + server worlds are running

    2. Doing it like in the asteroids demo:
    - import Settings to create a client.dll and server.dll
    - delete client or server dll after the build run

    EDIT:
    Just figured out that you just have to declare UNITY_CLIENT or UNITY_SERVER in PlayerSettings Scripting Define Symbols
     
    Last edited: Dec 4, 2019
  37. laurie71

    laurie71

    Joined:
    Aug 15, 2018
    Posts:
    41
    The quick start guide in the manual needs some attention; all the images are missing...
     
  38. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    I'm at that point too, see my post here: https://forum.unity.com/threads/dots-netcode-copenhagen-updates.760112/page-3#post-5242976

    I've tried something else: creating a custom bootstrap, 2 boot scenes (Client/Server scenes), creating a MB with a boolean that will create the corresponding worlds and then using the build pipeline to make a build (Client / Server profiles with corresponding scenes).
    This should be working but it's not and there is something weird with Netcode gameobject conversion.

    If I try to create the server / client world before the scene where I have my ghost collection it will not convert correctly my GO. I've looked more deeply into that issue and I've seen something that I find weird: in GameObjectConversionSettings when it will fork settings, the BlobAssetStore isn't kept and since it's a fork, you will get a null ref after that if you try to use it for the conversion which the GhostAuthoringSystem does.
    Is this by design ? Like you must create the worlds in a scene where the GhostCollection already exists in order to prevent this issue ? (even then you would miss the awakes of the ConvertToEntity scripts I think).

    Feels like I'm not understanding something about conversion here but not sure what.

    I've also tried to change the netcode package directly to set a BlobAssetStore (after forking the settings and modifying the BlobAssetStore property to be able to set it inside another script) but the build just spam things like these warnings or it just crash (with a memory access error).


    Internal: JobTempAlloc has allocations that are more than 4 frames old - this is not allowed and likely a leak
    (Filename: C:\buildslave\unity\build\Runtime/Allocator/ThreadsafeLinearAllocator.cpp Line: 554)

    Internal: deleting an allocation that is older than its permitted lifetime of 4 frames (age = 10)
    Unity.Collections.LowLevel.Unsafe.UnsafeUtility:Free(Void*, Allocator)
     
  39. laurie71

    laurie71

    Joined:
    Aug 15, 2018
    Posts:
    41
    [EDIT:] Problem solved by skipping the "Update component list" step and generating the code, then updating the components afterwards.

    I tried to get a test project set up, using 2019.3.0f on Mac (Mojave, 10.14.6) but the editor crashed while setting up the Ghost Authoring Component after generating code. After that it continues to crash whenever I try to work with that component. I even tried deleting the test project and starting over, but as soon as I hit the "Update component list" button the editor crashes, every time.

    The only clue I can find as to why is this assertion error in the logs:

    * Assertion at metadata.c:1117, condition `idx < t->rows' not met

    I've checked that all required packages are installed; I've tried both with and without installing updates to the couple of packages that have them (and yes, I fixed the invalid Collections package version to make the Jobs package happy).

    Packages:

    upload_2019-12-4_21-8-37.png
     
    Last edited: Dec 5, 2019
    johnroodt likes this.
  40. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    In order to make a client only build you set UNITY_CLIENT in the PlayerSettings. To make a server build you check the Server Build checkbox in the build settings and it will add the UNITY_SERVER define for you automatically.
    It is ok to leave UNITY_CLIENT defined also when making a server build, UNITY_SERVER will take precedence and override UNITY_CLIENT.
     
    RogDolos and Flipps like this.
  41. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    This is a bug in Fork which we are working on a fix for, it should set a valid BlobAssetStore. I have only seen it cause issues when using unity physics 0.2.5 instead of 0.2.4, but it is possible other converters can cause the same problem.

    When using ConvertToClientServerEntity the worlds must be created before conversion runs or it will not convert anything. The other option you have is to use Entity SubScenes - the NetCodeConversionSettings is used together with the new BuildSettings to convert a subscene to a client and a server version. You also need to set the BuildSettingsGUID (World.GetOrCreateSystem<SceneSystem>().BuildSettingsGUID) for the world to load the correct subscene.
     
  42. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,052
    I'm having really really long stalls in the editor when trying to run the NetCode sample project, upwards of 1+ minute when simply hitting play - editor consumes one core fully and completely freeze, if i leave it for ~1 minute it will eventually start.

    Completely clean project of the sample repo, on 2019.3.0f1.

    Edit: The same happens if i install the netcode package into a clean project, and it goes away if i remove the netcode package.
     
  43. Husain136

    Husain136

    Joined:
    Sep 13, 2015
    Posts:
    15
    I guess they really want us to use the experimental "Enter Play Mode Options"
    :p
     
  44. PrisedRabbit

    PrisedRabbit

    Joined:
    Aug 14, 2012
    Posts:
    63
    With synchronous burst compilation - same behaviour. Also first run takes 30 sec + increase memory usage up to 4gb.
     
  45. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,052
    I have a similar problem, takes forever to start (30-60 seconds) and when started it's really really slow, getting like 10 fps on a threadripper 2950x
     
  46. PrisedRabbit

    PrisedRabbit

    Joined:
    Aug 14, 2012
    Posts:
    63
    upload_2019-12-5_21-40-23.png

    upload_2019-12-5_21-41-48.png
    0.9 Gb of Video RAM usage? While I have 512mb. That's the reason of low fps on my machine.
     
    Last edited: Dec 5, 2019
  47. Jawsarn

    Jawsarn

    Joined:
    Jan 12, 2017
    Posts:
    245
    I feel this is this counter intuitive, why does one have to take precedence and not just _ADD_ them together by default? Simpler to throw out a build with server which can be playtested, and let custom build options enabling/disabling the defines and thereby what worlds are used?

    On a parallel note ,rewrote (extended) the bootstrap for multiple different server worlds; Barely any comments and weird code of using "foreach" when there is only max one element in array. =/
     
    Last edited: Dec 5, 2019
  48. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Is the slow entering playmode only when using synchronous burst compile? That is known to be slow the first time and it cannot use more than one cpu core since the compilation is synchronous, compilation should be cached after that so it is faster.
    10 fps on a modern machine sounds strange though, could you post a screenshot of the profiler (preferably timeline rather than hierarchy)?
     
  49. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The setup right now is that if you pick headless server in the build settings you always get a server - because it is setting UNITY_SERVER. In headless, server is the only thing that makes sense so it seems like a reasonable default.

    If you build a non-headless you by default get both client and server code. In a build like this it is possible to pick which worlds to create at runtime based on command-line parameters, and that is probably what you want to do for a playtest.

    If you want to build a pure client without any server code in it you set the UNITY_CLIENT define in the player settings. If you do you will still get a server when you pick headless, but if you do not you will get a client-only build. This setup would not be possible without custom tools if we added the defines and the reason server takes precedence.
    There are not a lot of reasons why you have to ever make a client only build over a client and server build, which is why client and server is the default.
     
    RogDolos likes this.
  50. pal_trefall

    pal_trefall

    Joined:
    Feb 5, 2019
    Posts:
    78
    Would it not be possible to build headless thin clients for stress testing? Or is thin clients handled differently?