Community project for physics based games and networking (VR use case)

Davinet is a (work in progress) proof of concept for a Unity multiplayer networking framework (it is not remotely production ready). Its primary goal is to allow players to interact with physics objects with the same level of responsiveness and precision as they'd have playing offline.

This proof of concept is created to both survey if there is interest in a community driven networking package of this type and as a request for feedback on the general architecture. If there is community support, I promise it can be renamed.

GitHub Repository

Current Status

• Working well in LAN conditions.
• Main goal right now is to get the authority transfer scheme working nicely, to allow players to interact with objects responsively on the client end.
• Other goals to get the package to prototype level are listed in the repo.

Original Post Below
Hey Unity networkers! This thread was inspired by a recent post by @JoeStrout.

I'm currently working on an asymmetrical VR vs PC called Davigo that is heavily physics driven, and a major feature we want to implement is networked multiplayer. I did a survey of various networking packages to see if there was anything appropriate. Some of the more battle-tested packages like PUN didn't quite fit our requirements, and while Mirror looks very well supported, it's still based on the deprecated HLAPI, which I found very opaque with its code generation and tight coupling.

One option we've been considering is to roll our own networking package. This would by definition eventually fit all of our requirements, but that would be out of scope for our team alone. Another option would be to build an open source community project, under the idea that enough people need a solid networking package that is conducive to VR/physics problems.

I'm interested to hear from anyone else's experiences running into the problems of networking physics heavy games (not even necessarily VR!) in Unity, and if you're interested in a community driven package what kind of requirements your specific project has. This way, we can see if there's some key features the community aligns on. For Davigo, I'd be looking for:

1. Responsive physics on the client side—players can pick up and thrown objects with the same level of responsiveness they'd have playing offline (similar to how conventional multiplayer games attempt to offer players client side responsiveness for their character). Here's one example of how to accomplish this in VR.
2. State synchronization design philosophy. State synchronization runs a mostly deterministic simulation on the client that is frequently reconciled with the server's state.
3. Solid (SOLID?) engineering principles, including: low coupling, high modularity, programming against interfaces rather than implementations, composition over inheritance and whatnot. I find many packages out there have extremely high coupling and very low extensibility—extensibility in particular would be essential for a community driven project.

Things I don't personally need, but your project might:

1. Full server authority. I'd be looking at having players self host, so server authority simply reduces the amount of potential cheaters to one per session rather than everyone in the session. This is a nice to have for us.

So let me know if this comes off as interesting or if you have any input. I have lots of engineering experience and solid networking fundamentals, but I wouldn't be a subject matter expert like Gaffer. There are many open questions, like how far something with the above requirements could be taken with PhysX, and what an MVP could like like. Sometime this week I might throw together a small repo this week using LitNetLib if there's enough interest in this.

 

My own needs, currently, are for something that can scale readily to large games like MMOs. That means a couple of things:

• Work should be pushed off the server onto the clients as much as possible (though this is in a basic conflict with full server authority, so I'm not sure what to do about that).
• The world must be divisible among multiple servers, with smooth hand-off from one server to another.
• Network traffic must be kept to a minimum, so the server isn't overwhelmed with messages when hundreds of players are on at once.
• For the same reason, sending and receiving of messages should probably be non-blocking (done on a separate thread) so the main thread can be doing things like physics while sockets are doing their thing.

But like @Iron-Warrior, it's important to me that players see a responsive world; they should be able to poke things and have them react immediately, without waiting for a round-trip to the server. That means client-side physics and other state updates. Possibly we can have a "trust, but verify" policy where clients update the game state immediately, but the server checks their work and makes sure it's believable.

 

Made a quick demo yesterday evening using LiteNetLib.

This follow's Gaffer's state synchronization model described above. Obviously this is in ideal conditions (playing it locally!), but it's pretty easy to get started with, and improvements (bandwidth, error fixing, etc) can be added iteratively. I'd like to implement a rolling cube character for each player (with listen server support) and some form of client authority before posting up a repo to see what everyone thinks. As well, hopefully it can demonstrate how to keep the game logic disconnected from the netcode.

 

mostly deterministic ? isn't it two words that can't go together ?

but still looks good and neat demo you did like Gaffer's one
very interested to see it out

 

Yea, "mostly deterministic" is a bit of a contradiction in terms :O

Gaffer uses the phrase "perfect determinism", but that almost seems redundant. I suppose "the simulations will mostly stay synchronized" is the most accurate, but doesn't roll off the tongue.

 

Players are now spawned/controller via the client. Added host (listen server) support. Clients are authoritative right now with their own character (big cube), though authority (calling it ownership) can be granted to any object. When a client owns an object, the data flow is reversed (the client writes its state to the server, the server accepts client states on objects it does not own).

 

This seems like a very solid start. For something like a hockey game, you would probably assign ownership of the puck to whatever player touches it, so that as you're dribbling etc., there is no latency for you even if there's a bit of latency for everyone else.

My own needs at the moment are to support an MMO. That means, among other things, that there's a lot more state than just physics state; every character and many of the objects have a fair number of stats (health, mana, XP, current buffs, etc.) that must also be synced. Probably those should be server-authority. Any thoughts on how those would fit in to what you're building?

Another issue that just came up for me yesterday: you can grab a hat/helmet and put it on your head, and it works like this: when grabbed, we reparent the hat to your hand, and when you wear it, we reparent it to a transform on your head. I use an Rpc (which through Mirror is an annoying multi-step process, where a client first sends a message to the server which runs a method that then calls an Rpc on the clients) to notify other clients about the grab or wear, so they can update the transform locally. And more messages are sent when you drop something.

Annoying, but it works fine for players that are logged on... but of course it doesn't work for players that log in after you grab or wear an item. They didn't get the message, and so their version of the hat is not properly parented, and everything breaks. To fix that, I will switch to using a SyncVar to keep track of what any grabbable item is parented to. But identifying that parent in a network-savvy way may be a little thorny.

So. In our spiffy new networking framework, can we streamline this process any?

 

Thinking about this a little more deeply, it seems to me that any physics object is always in one of these high-level states:

1. Non-kinematic (i.e. being simulated), and awake.
2. Non-kinematic, but asleep.
3. Kinematic (and quite likely parented under something else).
4. Currently hidden/disabled (e.g. inside a container, killed and waiting to respawn, etc.)

Handling all these states in Mirror is a real PITA. I would like to see a new networking system handle these smoothly and painlessly, with a minimum of network traffic (but still working correctly under client drop-in and drop-out).

Thoughts?

 

So this is the next thing I'm working on: synchronizing infrequent data (whereas everything so far, like you said, is just physics states that need to be updated very frequently). The first use case I'm working with is:

When the player presses "T", they can transform between a ball and a cube. The code for it looks like

Code (CSharp):

1. using UnityEngine;
2.  
3. public class CubeSphereTransformer : MonoBehaviour
4. {
5.     [SerializeField]
6.     GameObject cube;
7.  
8.     [SerializeField]
9.     GameObject sphere;
10.  
11.     [SerializeField]
12.     Animator animator;
13.  
14.     private StateField<bool> isCube;
15.  
16.     private void Awake()
17.     {
18.         isCube = new StateField<bool>(true, IsCube_OnChanged);
19.     }
20.  
21.     private void IsCube_OnChanged(bool value)
22.     {
23.         cube.SetActive(value);
24.         sphere.SetActive(!value);
25.  
26.         animator.SetBool("IsCube", value);
27.     }
28.  
29.     public void Transform()
30.     {
31.         isCube.Value = !isCube.Value;
32.     }
33. }

The key here is that a variety of stuff happens during transform: two game objects (with sphere and box colliders) get toggled, and an animator (with a blend shape) also gets toggled. However, since all this occurs during an event callback that gets fired whenever

isCube

is changed, we only need to send one piece of data across the network (a bool). Re: your hat example, you can see in the second gif in this thread that when the client joins the server immediately syncs all of the states. This would include syncing the current

isCube

state, which would drive the local changes (colliders and animations).

So all we need to do is ensure all of our objects state is encapsulated either in

StateFields

(low frequency) or high frequency data we manually write (physics). For your problem above, this could mean making a custom component (StatefulRigidbody or something) that has a

StateField<bool>

for Kinematic/non-kin that would drive the actual rigidbody's state.

Regarding server authority, it would be nice if certain things simply could not be owned on the client. This could introduce a bit of latency, but for low freq stuff like current XP it wouldn't matter. I can't imagine most MMOs would predict that on the client. As usual, making it streamlined is the most important part!

 

I like it. Your demo is really coming along!

I like the StateField generic you've got there. Do changes to such fields get sent immediately, or are they batched up and sent with others after some interval? (I can see pros and cons both ways.)

Managing the state of a physics object is more than just wrapping isKinematic in a state var, though. When an object switches to kinematic, it often needs to change its parent and perhaps its local position/orientation at the same time. Failure to do these things all together, or in the right order, causes Bad Things to happen. Similarly, when an object is released, it has to become non-kinematic at the same time that its parent changes back, and often it's imparted an initial velocity too (think of things being thrown or flung off).

Perhaps a good next step for your demo would be to have a "sticky" button that, when held, changes the behavior of the ball/cube such that the little cubes it touches are picked up (reparented to the ball/cube). And then when the button is released, they fly off with appropriate velocity. This would be analogous to picking up items and carrying them in your hand (or mounting them to some other part of the player hierarchy).

And then (as a separate trick?), have a "vacuum" button that causes any cubes touched to disappear (i.e. become both invisible and not interacting with other objects in the scene — perhaps tucked away in the scene hierarchy somewhere). And then when that button is released, the vacuumed-up objects reappear, perhaps popping straight up out of the ball. This would be analogous to picking up items and stashing them in inventory.

I feel like if the library is going to be specialized for handling physics well, it ought to handle these cases too.

 

Those two are defs good use cases, especially making sure it's easy to synchronize complex operations with simple data (where, like you said, order matters).

 

Finished working on the infrequent (

StateField

) sync system. On Awake, the networking MonoBehaviour component (

StatefulObject

) will search every component attached to its game object (recursively) to find MonoBehaviours that contain

StateFields

(using Reflection). This keeps the game logic components firmly decoupled from the netcode.

Fields are only sent when they are dirty (or if the player has just joined to get the state of their world up to speed). Thinking it over, StateFields (combined with the frequent raw state updates) should be powerful enough to express any state of object—except when it wants to refer to other networked objects. In theory this shouldn't be hard, since all of the networked objects have an (int) id, so just gotta pass that over the network. But this is pretty key to get right, so it's next on the list.

 

One thing I've run into is when a state field wants to refer to some other object in the scene that is not a networked object, but some sub-object in the hierarchy. For example, an object is reparented to the player's left hand. The left hand is not a network object, but the player is. (As it must be, with Mirror at least, because it does not support nested network objects.)

I handled this by sending over the net ID of the player, plus a transform path. Then I just walk the transform path by name (which of course assumes the names are unique at each level) starting with the network object.

Something to think about when trying to get this right!

 

Following this!
Your StateField approach seem very interesting!

 

Working on local authority and ownership of objects, as described in the Bidirectional Flow section of this article. In short: by default the server has authority over all objects. When a peer has authority over an object, it will reject any updates that come in for that object and send out their own. When a peer has ownership, they by default also have authority, but other peers cannot claim authority over an owned object. Objects that are owned by the server can be claimed by all.

Below the player has a new ability to grab objects (by right clicking them) and throw by right clicking again. Grabbing consists of setting the object to be kinematic and updating its position every frame to be inside the orange sphere (I could have used parenting here, but since the player rolls around the held object would roll with it). This is all done with a single variable of the type

StatefulObjectReference

. Code below for the event callback when it's changed.

Code (CSharp):

1.  
2.     private void HeldObject_OnChanged(StatefulObject current, StatefulObject previous)
3.     {
4.         if (current != null)
5.         {
6.             current.GetComponent<Rigidbody>().isKinematic = true;
7.             StatefulWorld.Instance.SetOwnership(current.Ownable, GetComponent<OwnableObject>().Owner);
8.         }
9.  
10.         if (previous != null)
11.         {
12.             previous.GetComponent<Rigidbody>().isKinematic = false;
13.             StatefulWorld.Instance.RelinquishOwnership(previous.Ownable);
14.         }
15.     }

...and I kind of like programming like this. Feels like it prevents errors by tying side effects directly to the object state. Since in perfect conditions (zero latency), none of this is ownership stuff is necessary, below I have the latency set to 200ms. Note that I have not done any testing under less than perfect conditions for now, so all sorts of goofy stuff happen (there is no jitter buffer, so some updates will arrive in the same frame, out of order, and so on—future work!).

The key here is despite these nightmare conditions, the client's ability to precisely throw the object is unaffected, since they have authority over it. This does mean that other peers would have a reduced ability to react to/dodge the throw, but this is the same with most first person shooters (where attackers have the advantage in that hits on the local client), minus the server authority. However, since the server is still the intermediary for all clients, there's no reason it couldn't reject updates it didn't like (at the expense of responsiveness on the clients).

On a slightly different topic, something I'm really keen on is making sure that the package is easy to code against. The idea is that gameplay code should never have to directly talk to the server, and try to reduce how much it even needs to consider it. Right now, all gameplay code only directly interacts with the following class:

Code (CSharp):

1. namespace Davinet
2. {
3.     // Can't spell Singleton without sin.
4.     /// <summary>
5.     /// All objects that are part of the game logic should be a part
6.     /// of the stateful world.
7.     /// </summary>
8.     public class StatefulWorld : SingletonBehaviour<StatefulWorld>
9.     {
10.         [SerializeField]
11.         IdentifiableObject[] registeredPrefabs;
12.  
13.         public event System.Action<StatefulObject> OnAdd;
14.         public event System.Action<StatefulObject> OnRemove;
15.         public event System.Action<OwnableObject> OnSetOwnership;
16.  
17.         public Dictionary<int, IdentifiableObject> registeredPrefabsMap;
18.         public Dictionary<int, StatefulObject> statefulObjects;
19.  
20.         public int Frame { get; set; }
21.  
22.         private void Awake()
23.         {
24.             registeredPrefabsMap = new Dictionary<int, IdentifiableObject>();
25.  
26.             foreach (IdentifiableObject registeredPrefab in registeredPrefabs)
27.             {
28.                 registeredPrefabsMap[registeredPrefab.GUID] = registeredPrefab;
29.             }
30.         }
31.  
32.         public void Initialize()
33.         {
34.             statefulObjects = new Dictionary<int, StatefulObject>();
35.  
36.             foreach (StatefulObject statefulObject in FindObjectsOfType<StatefulObject>())
37.             {
38.                 int id = statefulObjects.Count + 1;
39.  
40.                 statefulObject.ID = id;
41.                 statefulObjects[id] = statefulObject;
42.             }
43.         }
44.  
45.         public void Add(StatefulObject o)
46.         {
47.             int id = statefulObjects.Count + 1;
48.  
49.             o.ID = id;
50.             statefulObjects[id] = o;
51.  
52.             OnAdd?.Invoke(o);
53.         }
54.  
55.         public void Remove(StatefulObject o)
56.         {
57.             statefulObjects.Remove(o.ID);
58.             OnRemove?.Invoke(o);
59.         }
60.  
61.         public StatefulObject GetStatefulObject(int id)
62.         {
63.             return statefulObjects[id];
64.         }
65.  
66.         public void SetOwnership(OwnableObject o, int owner, bool silent=false)
67.         {
68.             o.SetOwner(owner);
69.  
70.             if (!silent)
71.                 OnSetOwnership(o);
72.         }
73.  
74.         public void RelinquishOwnership(OwnableObject o, bool silent=false)
75.         {
76.             o.RelinquishOwnership();
77.  
78.             if (!silent)
79.                 OnSetOwnership(o);
80.         }
81.     }
82. }

With the netcode listening in to events and reacting accordingly. My plan is to hopefully finish a very minimal MVP and then post it to GitHub before doing any kind of optimizations, to get some feedback on the (extremely rough!) architecture.

It depends—one way with the above approach would be to have a StatefulObjectReference for each hand that can hold objects. Then, the hand transform would be setup by the developer, making the object reference the only necessary thing to send over the network.

 

As we saw in the previous video, poor network conditions will lead to a poor replication of the simulation if not handled correctly. Even in very good network conditions, packets may arrive out of order or not at the same rate they were sent. To handle network "jitter", we can use a jitter buffer. Instead of applying state packets immediately when they arrive, the packets are inserted into a buffer to wait for a pre-specified period of time. This creates artificial latency, but it gives the network time to re-order out of order packets or spread out packets that arrived in the same frame. The example below has the server inducing simulated latency of 0-200ms (so very jittery!). For the first half, no jitter buffer is used. In the second half, a buffer of 100ms (much larger than we'd normally want) is used. Even in very jittery conditions, we can see the simulation is quite smooth (at the cost of latency, but a pretty good tradeoff!).

Jitter buffers are explained in a bit more detail in the state synchronization Gaffer article in the OP.

Next up is cleaning up the architecture a bit, getting some good debug tools running, and some finishing touches for this to be a proof of concept that I can throw up on GitHub.

 

I'm skeptical of this jitter buffer technique. It seems like something you would need only if you don't know how to extrapolate from any given point in time.

But if you do know how to pick up from any point in time, then why shouldn't you just always use the latest information you have, ignoring any packets that arrive with older info than that?

 

That's a really good point. My assumption going in was that it was mainly to allow for objects that are controlled by players to have less pop, since they're not something that the client can really predict (since they're influenced each frame by the user's input). The jitter buffer allows all of those inputs to be played out, rather than discarding any that arrive out of order.

That's my take, anyways—I'm not remotely an expert on this stuff. Something that is super key for me is to make sure that any features like this that have potential drawbacks (in the case of the jitter buffer, less responsive) are not only toggleable, but when toggled off they don't interfere with the code (i.e., if the jitter buffer is disable, no buffer would be created, rather than simply having a buffer of length zero). Optional features should do their best to not increase the complexity of the code. Making this stuff easily testable is also really important, so that a developer can clearly decide whether it's valuable or not.

 

Just caught up on this thread, seems very interesting -- looking forward to, hopefully, participate in the thread and with the code when released.

 

Finishing the ownership-authority system. The video below (the client, under 1000ms of latency) shows how authority propagates between objects: when the force sphere pushes objects away, it takes them under its authority. When those objects hit other ones, they also take them under authority, causing a chain reaction.

Once the objects come to rest on the server (not the client), the server will send a message notifying them that their authority has been revoked. If the clients were permitted to immediately revoke authority when objects locally fell asleep, they'd immediately start picking up "old" data. In this example, they'd snap back 1 second in time, so we need to wait until the server and the client have the object in the same place (where it fell asleep). Of course, other clients can come and grab authority in the meantime.

(Note that some objects seem to fail to give up authority—this is since the server is constantly getting updates from the client on the object's rigidbody state, these state updates are waking it up on the server. I do check if the incoming values are similar, but they're still enough out of sync sometimes to stay awake. It's not something that would be visible to the player, but will prolly need a custom check for awake/asleep.)

Getting all the bugs ironed out of this is fairly tough! Since it's core to keeping the simulation responsive for all clients, important to get it right. As well, have to make sure it all works correctly with host clients (listen client), since I'm a big fan of self hosting games.

This is the last feature I'd like to get in before making the repo public, so hopefully done soon.

EDIT: Got bored of dealing with authority nonsense, pushed a repo up.

 

Wow this framework looks extremely promising and works really well already. I'm currently making a physics heavy coop-game and so far no network-solution I played around with fit my needs without drawbacks. I nearly gave up on finding something useful, but your solution really looks promising.
I hope you keep up the good work!

 

Definitely looks interesting. A physics-focused networking Library would be very useful for me. A bit unfortunate that it isn't entirely server authoritative, but given the circumstances, it would be pretty hard to implement.

 

@Iron-Warrior Yo,
Doubt, is this technique also used in moba/brawers? Would it be a recommended technique for this type of games? Because of the state synchronization. I am following the project, I see that your focus is on physics, I am trying to learn these networking techniques but it is for a game structure in which it does not involve physics.

Cheers.

 

The repo no longer exists, is the project still going on?

 

Nada, I moved on to using Photon Fusion for the project (currently in beta), which has client side prediction and integration with PhysX.