Multiplayer action - amount of data?

I was wondering how much data being sent is common for a fast paced multiplayer game.

Right now I am sending 12 vectors per player from the server to each client (delta compressed - only 2 of the objects move at once) at a sendrate of 15, and the keypresses of the clients back to the server via RPC (one for key down, one for key up).

With one player as the server and another 2-3 players as clients it already runs unplayable, but I'm not sure if it's the network or something else (I don't get it when I run two or three copies locally).

Is this already too much for a LAN game?

How many kb of send/receive data per second are you guys having?

 

It's probably the network. Try connecting just one player on a remote server and see if it lags also(not locally), or emulate Dial-up within Unity editor.

You should notice the same behavior.

The problem is that 15 packets per second just aren't enough to keep the game smooth, 15 packets = 15 updates, meaning it will seem as if the game is running at 15 frames per second.

You should either increase the sendrate, or implement client side prediction. Client side prediction is a bit complicated but if you're a good programmer and know some math you can have it up and running in a week(starting from nothing). You can find the basic client side prediction in the Unity Networking example(available for download from this site).

 

Yeah I'm already using prediction to some degree. I derived my code from the network example.

Like I said, if I test locally, a sendrate of 15 totally does the trick. I shall try emulating Dial-Up. Maybe the issue really stems from our LAN.

But I'm stil curious what amount of data/s is common with other games. What works for you guys?

 

I have a maximum of 75 updates a second with a possible drop rate of ~20. The prediction takes care of any drops > ~30. Target is 40 FPS update for stability. Not every update sends the same type of information either. As in, I do not design the same packet for all sends. I have my packets modularized.

 

A sendrate of 75? That sounds like a lot, but you seem to have a smart way of going about it.

How much data do you send then within one update?

 

Player basic data module sends < 48k, battle module sends < 128k, zone heartbeat information 128k ~ 256k (no more than 4 per second push only), there are a few other modules depending on the area (indoor vs outdoor, singular buildgs etc), if you have ever played Dungeons and Dragons Online, you would understand the type of system I am working on. All broadcasts are limited to the area the players are centralized in. The global update happens ~2 times a second since the global update is for time keeping and synchronization along with global chats.

Most people make the mistake (not saying you are, just from watching general threads about networking) of trying to shove everything in one serialize object. Taking the character data for all characters and broadcast it to all other characters, this is bad practice. You have to know how close one person is to another person to decide what to send to that person. If a person is in the same game level area (what I call zones, and what almost all MMO's call zones), all you really need to know is how close they are to each other, so basic vector information only across the zone so the other clients know about the players location, not what they are doing.

Now, you have 'visual range' this is the desired distance that additional information gets passed around, this is both server and client defined, first the clients machine capabilities, this you have to determine what the machine can handle, PLUS you have to take into consideration server load, see, another common misunderstanding is that each level (zone) in an large player game (not even talking MMO here, talking >64 player base), is that each level is all taken care of by the same server, in truth, this is server banks (up to 8 virtual servers per bank depending on the processors, but most have Itanium or Xeons and about 16 ~ 32 gig of ram and 8 quad core), each virtual server handles just 1 zone, this way when a zone has an issue, they can just reboot that single virtual server, thus only causing issues to those players in that particular zone. In a single game heartbeat they force save all characters that will be effected and reboot, worst case, they reboot without that save and the players are stuck with the prior save heartbeat.

That said, I save character info in incrimental saves (bits of changed data), if need be I rebuild the character data in up to 12 chunks. If you understand SAN backups with snapshots you get the idea, same concept since frankly it takes too much time to save all of the player data on every save, I only look for changes between saves, its the difference between a character file being 25 megs and 128k, think nano-second shots vs a second or two per player. This is never discussed (atleast no one has ever brought it up around here).

Large player based games are more than just sending data about the players around. All you have to do is break everything up into chunks and make the chunks as small as humanly possible. Otherwise you end up with sparatic movement, battles that bog down to nothing and lost information.

 

Thanks, that was very insightful!

What I'm working on is not even remotely as complex as what you seem to be doing. It's a 4-8 player "arena" game - so essentially 80% of the time every player "sees" every other player (even if only their xz position on the radar). Apart from a few RPCs I only need to sync position and rotation of the player's stuff. So I have to synchronize rotation and position all the time - plus each player object has a subobject with rotation as well - they can fire up to two projectiles which need to be synchronized too >> 12 vectors.

That's 32*3*12 ~= 1 kbit per player.. for 8 players ~8kbit / update, and that's not yet compressed.

At a sendrate of 30 that'd be around 280 kbit/s for 8 players, or ~35 per player.

Monitoring Unity for 2 players, I get 60-80 kbit/s per player. So there seems to be an overhead of 25-45 kbit/s (which includes RPCs for like starting and stopping animations and such).

If I emulate dial-up or broadband, I pretty much get the same bad performance (which is still better than what we had on LAN).

Curiously, if I double the sendrate, the amount of data sent per second barely increases, but the performance hardly becomes better either.


I use Unity's delta compression. Could that be the problem? Sending the raw data turned out to shove heaps of data through the network, so using that turned out to reduce the data/s a lot. Could it be that it turns ugly when packets are late and not in order anymore?

 

Tackle your problem one piece at a time.
First thing, setup a network send of just the player location, broadcast without verification. Attach this code to the player so that on every update loop, it sends an RPC packet to all connected players that only includes vector data about the players location not orientation.

Monitor the players movements and bandwidth used.

Have the structure you use pass the following data

[PSEUDO CODE]
struct MyPacketData
PacketType as single
VectorData as 3DVector
TargetID as integer
end struct

PacketType Enumeration:
0 = player location
1 = player facing
2 = player orientation
...
10 = ammo type 1
11 = ammo type 2
12 = ammo type 3


Now, this single packet structure holds up to 255 different types of defined information and that objects location, can be used for players or weapon ammo data. So you use the same packet send command, pass it the structure as the object to send and have the structure hold what you want.

This is used on a broadcast event, once the rpc is sent, the other end gets the packet to an EGO that says what to do with the incoming data based on the PacketType enumeration. Then it determines if the TargetID already exists (aka a player ID, if it does it throws the values to that object and updates it, if it doesn't exist it clones a prefab and places it where the object location was (aka ammo fired from weapon)

 

I'm using Unity's NetworkViews for serializing and deserializing the data being sent, so that takes care of getting the right packet to the right object and I can just stuff into the packet whatever needs to be sent - as long as the order of putting stuff in and taking it back out is the same, this seems to work fine - so there is no real packet type, as every receiving object knows what data it is going to get.

So the packet structure is roughly like
struct MyPacketData
{
data as Vector3[]
targetID as int
timestamp as double
}


As there are no bullets or other objects that need to be instantiated on the fly, I send everything that can differ from the prefab in an RPC when an object is being created. The amount of data used for this is so small (e.g. 5 calls every 2 minutes), that there is no need to optimize that.

Every now and then I call a parameterless RPC like "StartRunAnimation" on an object. This is neglectable as well.


I shall see if I can send the data more efficiently with delta compression off so the packets can arrive unordered.

 

I handle everything independent of the internal networking components of Unity. Since I didn't write there network interface, I don't know every bit of code that is working behind the scenes, so to avoid any unwanted unknown overhead, I use my own implementation of RakNet and SmartFox. Hopefully you can locate the leak in what is killing your network interface.

 

Yeah, I had to make the decision in the beginning whether to trust Unity's implementation or to write my own.
As this is the first realtime network game I ever attempted, I took Unity's approach.

Even though I can't implement it as you recommended, it was already very helpful to just talk about and to hear how others do it.