Search Unity

Other Experiences from making my own advanced networking package in DOTS 0.17 - 0.50

Discussion in 'NetCode for ECS' started by PolarTron, Nov 28, 2022.

  1. PolarTron

    PolarTron

    Joined:
    Jun 21, 2013
    Posts:
    94
    For a long time I wanted to make my own advanced networking package filled with all the cool features from the Overwatch Networking GDC talks, Glenn Fiedler articles, FPSSample Deep Dive video and countless other references. DOTS was the perfect framework to build such a library in so I did it and then I lost interest as 1.0 was released and I wanted to focus on the game making parts of game development. I want to share some of my thoughts as some kind of a brain dump in case it is useful for the development of Netcode for Entities.

    The following are some main features I had or was going to implement in my package that maybe differ from Netcode for Entities. There are some features I miss, like private snapshots, but I might not know about the proper ways to do them yet. I’ve only used Netcode for a little bit so I probably need to do some projects first to properly understand all the features.

    ---BEGIN BRAIN DUMP---

    Client/Server/Shared assembly definition files. I had one for each purpose. Made it easy to exclude server code from clients and vice versa. The main problem I had with this was that generated code had to be put in the same assembly because of the partial keyword.

    I had a spatial hashing system for snapshots. You only get snapshots for entities in your current area and areas linked to this one, like a graph. This was inspired by the Source Engine’s visleaf system. I had a basic spatial grid system in place but it would be easy to make your own with a bounding box system. This makes it possible to avoid getting updates in areas you are not capable of rendering visually and adds another layer of security because cheaters don’t know what’s behind certain walls, because they don’t have that information.

    I didn’t have a Priority Accumulator/State Synchronization system in my code so a “public snapshot” feature was possible by generating a single snapshot packet for multiple clients at the same time. Every client who had the same base snapshot index in their input packet and were in the same area would make the code generate a single snapshot to send to these clients. The public snapshots were for information everyone should know of, like transform updates. Private snapshots were added as a way to send “private” information tailored to each specific client like the result of clientside prediction and information about teammates outside of the public snapshot range.

    Instead of a state synchronization system I was planning on making an “overflow” snapshot for information that didn’t fit inside the MTU. This snapshot was going to be sent at a lower frequency than the network tick and only be “consumed” by the client when a main snapshot arrived. Steam Datagram Relay and the fragmentation pipeline made this idea kinda redundant but it could be used to limit the bandwidth on snapshots instead of having a priority accumulator.

    I had a Pre & Post simulation update group for systems outside of the prediction loop. These groups were incredibly useful to move certain systems away from the prediction loop. Input gathering should be done in Pre and Logging/Debugging should be done in Post.

    I copied the Vector3Int struct and made my own GameUnits struct to quantize the Translation and PhysicsVelocity’s floating point data. I had to make a wrapper to convert these units temporarily before and after the physics simulation. I don’t know if this was clever at all but I felt pretty clever while doing it. I guess I absolutely wanted to be certain there wouldn’t be any precision errors.

    Only when there was a prediction error there would be a rollback. Predictions+input were stored in a history buffer and would be used whenever there was a rollback. I know this differs from Netcode for Entities where it rolls back every frame but I think I saved some resources there.

    I had a Floating Origin system in place and planned to extend the Unity Physics package with code which allowed me to update chunks of physics data by moving the origin around. A problem on the server is that there is a single physics update but the players can be far far away from each other so a floating origin doesn’t make sense there. Which group of players get the lowest floating point precision physics update? Multiple updates are required there. That, or not use floating point precision for physics updates.

    ---END BRAIN DUMP---

    An early version of the package can be found here: https://github.com/polartron/open-netcode. I spent a few months after this release upgrading the package in a project I did but I can't be bother trying to extract that code. It got messy as I gave up trying to keep game code and Open-Netcode separate.

    I hope my incoherent tech brain dump helps Netcode for Entities in any way. I’m looking forward to the future of this package because I can finally understand most of the features in it.
     
    PolarTron, Nov 28, 2022
    #1

  2. NikiWalker

    NikiWalker

    Unity Technologies

    Joined:
    May 18, 2021
    Posts:
    316
    Hey PolarTron! Nice writeup, thanks for sharing. For completeness (or to point people in the right direction), I wanted to give our approach to each of these features:

    Yes, this is our recommendation. The Asteroids sample uses this exact approach (Mixed == Shared), with an additional assembly for Authoring.

    We use a similar approach (Relevancy), but perform the culling while creating the per-connection snapshots. Relevancy allows you to denote which ghosts are relevant (or not relevant) to a given connection.

    +1 on this being a much needed feature. See our implementation here. Docs are being worked on.

    You can also use "Distance-Based Importance Scaling" (again, more complete documentation is coming) to change how often entities that are far away are replicated.

    "Public vs Private Snapshots" can be achieved using the above Relevancy. You just mark ghosts - which should be considered "hidden" from a specific clients POV - as "irrelevant" to that client.

    As for generating a single snapshot packet for multiple clients, our equivalent is "PreSerialization". PreSerialization (a toggle on the GhostAuthoringComponent) forces a ghost to be serialized once, then that serialization data is passed into every snapshot generated for every client on that frame.

    Example: In a 10 player game, if you expect to send the 10 players in every snapshot, you can preserialize the player ghostType, and thus all 10 snapshots (one sent to each client) will contain the same preserializated data. You therefore only serialize them once (per frame). It's only really suitable for ghostTypes where you're typically sending the entire ghost state in every snapshot (i.e. for extremely high importance ghosts). It also scales with connection count.

    In our case, we'd prefer the user solved this with Importance and relevancy, although snapshot fragmentation is supported if absolutely neccessary.

    This is a neat idea. AFAIK you can use `[UpdateBefore/After(typeof(GhostSimulationSystemGroup))]` (in the `SimulationSystemGroup` to achieve something similar, although I need to re-examine the recent changes to NetCode SystemGroups to give a more concrete answer.

    You can write a Variant of the `PhysicsVelocity` component - with whatever `GhostField` quantization value you require - and apply that Variant globally via a system derived from `DefaultVariantSystemBase`. Thus, every time the `PhysicsVelocity` is serialized, it'll use your Variant, and thus your quantization value. Again, documentation improvements are coming that'll make this a more transparent workflow.

    This is a big item on our list of prediction + physics optimizations. I can't give an ETA, but we're definitely thinking along the same lines.

    Just to expand on this: You are correct that - by default - snapshots are sent at the same rate as the `SimulationTickRate` (and thus once per frame). I.e. Rollbacks occur whenever a snapshot is received, and the user does have control of how often snapshots are sent (via the `ClientServerTickRate.NetworkTickRate` field).

    I'm actually not sure what our (Entities) recommendation is for huge-scale simulation is. I believe Unity's Component-based Transform throws after 100km on any axis (1unit = 1meter), but there are issues even as close as 5km out. I'll dig into this.

    Thanks again, mate!
     
    NikiWalker, Nov 29, 2022
    #2
    Neiist, Kmsxkuse and PolarTron like this.