is TCP viable for an MMO?

I'm using a c# tcp socket for packet sending, and I'm curious if TCP will work fine for an MMO.
Keep in mind this is a click to walk, slow paced MMO.

I'd be fine with mixing UDP and TCP, and using UDP for just entity movement/positions but I keep hearing that's a bad thing to do. Also if I use both, would I only set up a UDP listener on the client to receive movement updates and send to the server in only TCP? or should I allow the client to also send input over UDP?

I really prefer to use 100% TCP, because I need reliable data transmission for 99% of things. I'm just worried TCP will make the game unplayable if someone has some packet loss.

Thoughts?

 

I will just say that fast paced FPS games, where high accuracy is critical, they manage well with UDP.

On other hand, without hard comparison data (UDP/TCP) from your case, we can only speculate.

 

It's definitely not FPS. It's 3rd person, top down click to walk. 300 MS ping wouldn't matter. I'm just worried about TCP causing players to have 1000MS+ latency.

 

We use TCP for uMMORPG. It has significantly reduced stress since switching from UDP.
Reliability really matters there.

Imho TCP is fine for MMO movement too. If you use rubberband movement then you'll have 0 latency for the local player anyway - doesn't matter if you use TCP or UDP then.

TCP has downsides. But if you use it for an MMO, then you will waste 0 more hours on worrying about networking. Anything you send will be received on the other hand. No race conditions, etc.

 

Thanks .

I think I'm going to stick to my Async TCP then.

 

Since you mentioned MMO, I assume that you are targeting >50 CCU so I'll save you a huge amount of pain here: Unity's mono version has some serious issues with anything 'async', you'll end up with 120+ seconds latency after 50 CCU if you use async. Even if you disable Unity's async thread context.

 

Age of Conan used TCP. Occasionally it lead to backlog blockages, but mostly it was fine.

 

I'm just using Il2CPP, and using c# System.Threading namespace.
Will it still have issues in unity? Will I have to go to synchronous?

 

Might be off-topic but can you please post more details about this?

Is this because they use a custom context?
I checked their "reference code" for their custom context (not sure if this is still up to date) but it seems like they store all async callbacks in a queue and execute them every frame, so that shouldn't case such a high delay, unless the locks around the queue cause the slow down?

Is this a "bug" which will be "fixed" or is this a known issue without a solution?

What else are you using? Custom read/write threads for each connected client are not very scalable, so the other .NET Tcp options would be the BeginX/EndX methods or the SocketAsyncEventArgs (the name might be confusing, but they don't use the "async keyword" features, the class is older than the keyword, but they are "annoying" to use)

Also "bonus question" no matter if you use the BeginX, AsyncArgs or custom threading in all cases the methods will "finish" on a different thread, how do you sync back to the game/main thread, do you also use a custom sync context which stores the callbacks in a list/queue?

 

If you use async then you are affected.

Initially we thought it's because of the custom context, but we were able to turn it off and it's just as bad.

It happens for anything async: BeginX/EndX methods, SocketAsyncEventArgs, async-await, all of it. We had to try them all and it always results in 120s+ latency.

I use simple threads for Telepathy (my TCP library) at the moment.

I don't know if this is a known bug. We didn't submit it, because everyone in the Mirror team who submitted networking bugs in the past was either ignored by the UNET team, or received a 'by design' reply.

 

Hmm good to know, must be a "Unity Thing" then, because I didn't notice anything like that in a "normal" C# server application yet. Which .NET backend version did you use?

 

Yes, it's a Unity thing. happens in all the backends.

 

I recommend disabling https://en.wikipedia.org/wiki/Nagle's_algorithm (TCP_NODELAY) for TCP. Everything will be a bit more responsive, especially if you use a lot of small packets.

 

Will do. Thanks

 

I would advice against using TCP for any games, because as soon as something gets dropped all packets are on hold till the missing one gets resend, that is not a way to create fluent gameplay.

There are great UDP libraries around that support both Reliable and Unreliable sending over UDP, I don't see a reason to go for TCP at all?

@vis2k, i checked your Telepathy library, it is absolutely horrible, you are starting 2 threads PER CLIENT on the server, do you have any idea how many context switching your server cpu is going to do with that?? This is not at all how you should work with threads. my god.

 

Look at the expected network traffic, and compare the messages that need reliability to those that do not, and whether pauses to resend those messages that don't need reliability are acceptable or not. If most of your traffic needs reliability, and pauses in processing your traffic are acceptable, then TCP is probably optimal. If very little of your network traffic needs reliability, and your game will be significantly impacted by waiting or resends, then TCP is probably a poor choice.

Most games probably fall somewhere in between those extremes though. So you need to look at the pros/cons of all the options you are considering.

Personally, I think a UDP system implementing channels, with reliability configured per channel, is probably the most optimal networking solution for the majority of games, though that certainly doesn't mean it is best for all. There's some games where TCP will be the optimal choice. Implementing a lot of higher level functionality on top of UDP introduces a lot of complexity which TCP won't have, which could impact CPU cycles directly or introduce extra garbage collection that a simple TCP solution wouldn't have as an example, which would be the trade off for the greater flexibility UDP allows.

 

Did you read the rest of the thread?

 

What is this question?

The sadest thing here, that most of the time you give people false information in order to advertice your 200$ asset which is not worth its amount of money.

What you did in your libs is basically took some features of deprecated UNET hlapi and put TCP on top as well as some strange SafeQueue. Your benchmarks are inaccurate and completely inconsistent.

The servers that use your library will suffer from GC spikes and bandwidth problems just because you ignored any PRs that fix allocations as well as some bandwith optimization strategies.

 

I was just surprised about Vincenzo's post about threads, after explaining why async TCP doesn't work too well in Unity

Which part was false information?
I don't like two threads per connection either, but it is the fastest option for TCP in Unity at the moment. If you know a better solution then please let us know.

 

I can't imagine Unity introducing a bug like you describe and a bunch of people not noticing. 120 seconds sounds an awful lot like something is blocking you don't know about, holding up data from being processed and sent along.

But that aside, take a look at the Poll/Select api's. That with a small thread pool should actually perform fairly well up to I don't know a couple hundred or so sockets. That's what everyone did before we had kqueue/epoll/io completion ports.

 

DarkRift is based on asynchronous sockets and utilizes this API, then just dispatching stuff to user if I remember correctly. Everything works fine in Unity and outside of it. @Jamster

For many descriptors you need scalable event interfaces as @snacktime said.

 

I haven't read the whole thread but I'll respond to this part since I think I was tagged for this bit in particular!
​

DarkRift has a 'Dispatcher' which is essentially just a queue of callbacks. If the developer doesn't want to deal with threads then they enable an option and DarkRift calls every event after passing it through the Dispatcher.

Put simply, the Dispatcher is given events from multiple threads and then when Update is called (or just in a while loop on the main thread when outside of Unity) the Dispatcher is emptied of those callbacks. The threading code on it isn't too bad until you come to more complex scenarios like performance optimization and object pooling (all my objects were being created on the thread pool and recycled in the main thread which doesn't work with the DarkRift's object pooling!)

Hope that helps a bit!

 

I must answer this question a bajillion times. There's something fundamentally absent from the Unity docs LOOK HERE MODS that could explain to beginners how to share data with the Unity main thread from networking threads.

Use a ConcurrentQueue. Push items onto the ConcurrentQueue in your networking thread. Poll until empty in an Update method on a MonoBehaviour. That's it!

You can also use UniRx, which internally does something similar. But there, you have to learn "subscribeOn" and "observeOn" semantics which can be quite obscuring.

 

I might have worded that part wrong.

I know that you can use a (Concurrent)Queue to pool (work)items from one thread to the main thread.

But my question was if they use their "own" queue or if they use a SynchronizationContext, because Unity sets a custom SynchronizationContext which already does something similar (they use a queue with locks), so it would be possible to "Post" your item to their context instead of implementing your "own" queue/logic https://docs.microsoft.com/en-us/do...zationcontext.current?view=netframework-4.7.2

(Btw the ConcurrentQueue allocates a new object everytime you add a item, so it could cause some GC traffic if you add many items)

 

World of Warcraft uses TCP

 

I really wouldn't do that. But generally, since Unity already tries to run Update as fast as possible, and presumably the things you get from the server will need to be rendered, your dequeuing inside an Update will occur in the call stack of the fastest path anyway. You don't want to contend with Unity's render events, just put your work inside them!

Anything that doesn't eventually appear in an update you can do on your networking thread.

I think if you want to build something as ambitious as an MMO, and you're looking at ones that people have actually delivered with Unity, you're going to get out of the reductive frame of mind of working about how a few pointers being allocated will impact performance. Besides, I'm sure you can find a ConcurrentQueue implementation that pre-allocated nodes.

Although I'll credit that you probably know enough to reuse a buffer for your TCP reads! There's a lot of basic stuff that can go wrong.

 

Yes it does and it will always have acceptable performance doing so because Blizzard have a shadow internet of ISPs in each region which address a huge amount of problems which the humble indie will not be able to address. I think TCP is fine, and many games do still use it, however for an MMO, (cluster/shard/etc dealing with thousands of people) don't expect anywhere close to Blizzard's performance with it. It's good... but not a good example to be using.

I have made multiplayer games in TCP before with packets being sent as late as once per second but it will drive up engineering and not suit all game types.

Indies never find out how bad TCP can get because it's pretty much never that an indie gets a successful mp game going with hundreds of concurrent players. So TCP is fine for most indies.

So for those using TCP, turn off your delay and send packets as infrequently as you can, while simulating everything locally by default - just correct when important things happen. This is the best way to work with TCP.

 

Thanks for the info! I definitely agree with the last statement.