Data serialization for multiplayer games

I am looking for guidance on what type of serialization to use for a multiplayer sports game. The game can have up to 8 players and will use TCP for server to client communication. I have never written a multiplayer game this complex, but I suppose that the clients and server will need to be constantly communicating this data:

- Location of players and game object (ball)
i) Vector3 for their transform
ii) Vector3 for their current velocity

So roughly this is up to 18 vector 3s, plus and ID for each. If I use JSON, this is about 760bytes, or we'll say 1kb. This update probably needs to occur 2 times per second. So the server must be able to push up to 16 updates per second, after of course receiving client inputs and doing its game calculations.

First of all, I don't really understand if this is a lot of data. I don't know what a game like NHL16 or Rocket League is pushing, but I am guessing they don't use JSON. I researched Google ProtoBuffs a lot but have read about the problems associated with Unity's version of .NET. At the same time, I have 0 intention to support any mobile devices, but I would never look at supporting them as a downside.

So to my question: What is the best choice for data serialization for a multiplayer Unity game? I am using C# and the main intent is to support Windows. I am fluent in JSON and already have JSON support, but would want to move to something much more compact.

 

You don't really need to worry about any of that if all you're sending is transform and velocity. Once you start implementing unity's networking you'll see how things are done but it's basically all handled behind the scenes for you. The only time I've ever had an issue is if I need to send some non-standard data types that Unity does not support sending via rpc. In those cases I use something like this:

Code (CSharp):

1.  
2. MemoryStream stream = new MemoryStream();
3. BinaryFormatter formatter = new BinaryFormatter();
4. try
5. {
6.     formatter.Serialize(stream, objectToSerialize);
7. }
8. catch (SerializationException e)
9. {
10.     Debug.Log("Serialization Failed : " + e.Message);
11. }
12. byte[] objectAsBytes = stream.ToArray();
13. stream.Close();
14.  

To turn any simple c# class into a byte array that I can send via rpc. You have to make sure to mark your class as [Serializable].

On the other end after you receive the byte array you can transform it back into an object using:

Code (CSharp):

1.  
2. SomeObject objectThatWasDeserialized;
3. MemoryStream stream = new MemoryStream();
4. stream.Write(objectAsBytes, 0, objectAsBytes.Length);
5. stream.Seek(0, SeekOrigin.Begin);
6. BinaryFormatter formatter = new BinaryFormatter();
7. try
8. {
9.     objectThatWasDeserialized = (SomeObject)formatter.Deserialize(stream);
10. }
11. catch (SerializationException e)
12. {
13.     Debug.Log("Deserialization Failed : " + e.Message);
14. }
15. stream.Close();
16.  

I don't know if it's the most efficient way to serialize and deserialize data but it's certainly one of the easiest since it works for pretty much anything you could need. For the most part though you would just use unity's built in rpc system or syncvars.

[Edit]
I also experimented with protocol buffers and I was able serialize faster and send less data over the network if I remember correctly but I didn't end up using them anyway because it just wasn't significant enough of a gain to warrant bringing in a third party library, and the set up was a little more difficult. If you find yourself doing a lot of serialization of complex data types I could see it being worth while though.

 

Well my TCP/UDP clients and servers are built far outside of Unity. I am really using Unity as a demo for my personal game related networking products (like a chat server, matchmaking server, game server, game client). Therefore my game server won't understand Unity's serialization (unless there is some library I can use). That's why JSON or protobuffs were my two choices.

As a side question, any idea on what a "normal" amount of data-per-message is for a multiplayer game? Is 2kb/s a lot?

 

There is no general rule about how much the server or clients send per second, except one: the more you reduce it, the better it is. 2kByte/s for your scenario seems way too much.

Let's repeat, it's way too much data to be sent. I wouldn't recommend to use JSON for that.
There are alot of techniques which reduce the data.
If you really want to optimize this alot you sometimes have to roll your own use-case specific system.

Anyway, a few easy things to look into and to get started:
- Data reduction: What do you really need to send in order to achieve the effect?
- Data truncation: Don't send a complete int or float for convenience, truncate the data
- Apply the truncation wherever it is possible: Vectors, Quaternions, ...
- DeltaCompression: Offsets (or let's say difference to the previous state) are most-likely small, more truncation and reduction possible
- Don't send data that hasn't changed unless it is important for the game
- Use bit-flags to represent and interpret data, e.g. subsequent IDs
- .... list continues to more advanced techniques. I might also have missed some obvious ones.

On top of that you can even compress that.

Let me throw in a quick example for a completely different and unrelated, yet network-heavy genre:
Popular MMORPGS manage to only send ~ 20000 to 25000 Byte/s for hundreds of players and other network entities (in crowded areas). With a tickrate of 10ticks/s that's 2kByte per Update which is send to one client. On top of that, there's alot more to sync, not only 2 values

This is completely different to your case, but it helps to illustrate what's possible with a proper solution.

I've been working on my own networking solution and studying everything I can find for quite a while, it's still a fun project and not likely to end up anywhere near the professional ones. But there's so much to learn and it's a great topic to learn networking, train skills and find out about efficient techniques and algorithms.

 

Great info, thanks. What have you found as a good reference to study? I am having trouble just finding something to learn from. For example googling "writing a data serializer" doesn't really help.

 

Well, first of all I tried to come up with own ideas. Things got very complex pretty quickly.
When I ran out of ideas I started to search scientific papers and just googled lot's of things, I can't really remember what I had actually googled and I'm afraid I haven't saved all the resources (many things you're either already aware of or can be remembered just fine, because they are so obvious once you heard about them).
Just try to search for game networking. Don't hesitate to spend time on it, it's totally worth it. I had spent days and nights reading so many resources, it's incredible.

Much info can already be found on quite well known sites such as Gamasutra, GafferOnGames etc. They cover the basics I have mentioned above, but you may want to personalize all that for your specific games.
For the networking part itself I looked into asynchronous socket examples from MSDN, they've got a pretty basic example to send trivial text messages. However, the example itself won't do the work, you have to adjust quite a bit, add events, keep track of connected clients and wrap everything in order to have a great balance between an flexible, yet easy and convenient interface.

For sensitive data that you don't wanna send to a simple webserver, you may also integrate an SSL/TLS layer on top of that, which is not much of an extra work.