First Impression on the new network stack

Hi, guys! I just had a chance to take a look at the new network stack and in one word, I'm very disappointed. It's not that I expected much for the initial release but I'm utterly disappointed that there is basically nothing to look at. I wrote what a good network stack should support but I really feel ashamed that I was asking. The current state is nothing more than just basic network transport and streaming raw data. Given several months of work, I expect a lot more, at least automatic Class to message serializers with special attributes and some simple filtering setup but at the current speed of development, it will probably take several years before we see something sophisticated. You guys all know Unity's track record; a lot of things can happen before that.

Honestly, I really don't know what you are guys are thinking right now. There are many who are waiting on you ever since you mentioned that you will drop UNet. Is this another pet/experimental project like UNet? Do you even have a roadmap for yourself? Before we spend more time waiting, I would like to know what your design goals are and the roadmap with timeline. Sure it's not an easy task and if you are short staffed, please hire bunch of top network developers pronto. I think this is far more important than any other non-core project as I think the network system is the weakest link in Unity.

Unity mentioned the development Transparencies many times but so far, this network project didn't show any of it. You never shared what you are working on until today, and you never mentioned your roadmap and it even seems like the development is done on your internal repository and you push it into github once in a while so that we don't know the progress as it's happening. I think I finally understand it why, unfortunately, I was right. There wasn't much to share.

What I would like to see, in the end, is that the new network stack can scale up to large number of players using as little network resource possible. Optimize hack out of it using ECS and we will have the best performant network libraries, even better than what Epic has. Without it, what's the point of making another me-too network stack? In order for it to happen, you will need set the goal very clearly from the beginning.

Thank in advance for sharing your mind and I hope it really works out this time.
Cheers!

 

I posted my own comments in the network feedback thread, maybe they should start a new thread just for this repo so people know where to post.

Part of being transparent is putting out stuff early. So I don't think you can fault them for not having something more complete, and also not being transparent.

What they did put out is really bare bones yet complete in the sense of a complete loop. It seems like a pretty good starting point to me.

 

I agree. I wasn't expecting a finished product at this point but I was surprised at just how thin the wrapper around pushing raw bytes is. But, let's look at what they've said:

"The capabilities provided in the HLAPI will move from a generalized API to more-specific game archetypes. For example, we will soon release an FPS sample that will include open-source implementations of lag compensation, forward prediction, etc., that can be reused for similar games." (link)
...and...
"we will release a full source FPS sample game that includes production-quality sample code for client-side prediction, interpolation, and delta compression built on the new transport." (link)​

So, this is them saying that what you see in the new repo is exactly how they expect people to make FPS games with Unity, by manually serialising everything manually into a byte buffer. Create an interface with Serialize and Deserialize functions, override them in every single thing you want to put on the network, and iterate though the things you want to write or read. This IS their idea of production-quality code to use instead of the HLAPI.

This approach is efficient and works well, which is why we were writing code like this back in the mid 90s. (Admittedly it was less messy back then because it wasn't spread over a ton of components.) It's also about 10 times more cumbersome and error-prone than marking up properties for replication and RPCs, which is probably why Epic started using that in about 1998, and why HLAPI had a similar approach. But Unity is now going for "performance by default", which this new approach delivers, but which clearly implies ease of use is secondary.

 

@Kylotan Didn't know that you are reading this forum. Your fan since the days when dinosaurs roamed this planet.

 

@Kylotan, thank for your reply. I'm glad someone is interested. ^^
Here is the link to the post(in the reply),https://forum.unity.com/threads/game-server-hosting-feedback-and-questions.555076/,

Basically what I was asking were agnostic low-level functionalities. We can argue that there can be different usage but they are all relevant to all games. I strongly believe that having a sophisticated filtering support is a must for any performant network libraries, otherwise, why do we care about the network stack? Transport layers are easy to write and many other libraries can do equally well.

Yeah, network stack can have several layers and the current library is nothing more than a transport layer. We should have at least serialization layer, and filtering layers, and then application layer(such as FPSSample) can sit on top of it. I don't expect we all write the code on top of the transport layers ourselves. (I just start taking a look at FPSSample and I'll comment on it later)

Right now, we just don't have any clue what Unity is going to do with the network stack and I'm not even sure they are reading this. Large company such as Unity can do much better job than this and many are using Unity very seriously. I just don't see that Unity is taking it very seriously, unlike what they always say at the keynotes. Oh well..

 

Ya your idea of filtering is basically the equivalent of pipelines I mentioned in another thread. They are almost ubiquitous in networking stacks although less common in the game industry.

 

When you say "filtering" do you mean what you said in the other thread about "distance, priority, visibility"? I'm used to that being termed 'Interest Management'. I think it's a useful thing to have but normally it's layered on top of entity replication, and we don't even have anything resembling that in this architecture. Maybe they'll share something regarding that in a future example, but I'm just disappointed that they didn't think they could make an FPS that way.

You also mentioned delta compression - to be fair, they have provided code for that in the FPS Sample. Look at DeltaWriter.cs and DeltaReader.cs for the packet-level approach, and the [Read|Write]Packed*Delta functions in RansOutputStream.cs. and RansInputStream.cs for the per-field code. It is fairly general so you can probably cut and paste it into your game - providing you're happy using the low level transport as-is, because it's entirely low level and appears to examine the previous byte[] you send and just smashes the delta data into another byte[]. Very 90s.

Now, to have that as part of a general purpose replication system would be nice, but even then I wouldn't have it in by default, because it incurs costs (remembering prior state at both ends), needs handling differently depending on the transport (unreliable will require more prior states), and not every game would benefit much anyway (slow-paced MMOs can send position messages infrequently enough for it to not matter).

 

I took his filtering idea to look more like what you see in Netty. Where the network stack is chain of filters/pipes that transforms the data as it goes through, passing the transformed data to the next level along with optional state information if needed.

So you can then just layer in whatever you want. The protocol itself could be a layer, serialization/deserialization, branching for reliable/unreliable. Becomes basically a plugin system more or less with just a line of code or two to add any existing plugin.

 

Sounds interesting, but I've never heard of it being used in games. It's hard to imagine what role these layers would play. Having said that, I took one look at the Netty documentation and it was all in super-verbose Javaspeak and the user guide doesn't mention the word 'filter' even once, so it's possible I am misunderstanding the concepts.

 

DotNetty is probably easier to parse if you aren't used to java. No docs since it's basically a port but the examples are fairly easy to parse. Pipeline is the term they use not filter.

Pipelines are not common in the game industry. But then in the area of client/server architectures generally the game industry isn't exactly leading the way in best approaches to anything.

 

I'm used to Java, just not interested in wading through tons of verbose code to try and understand the concepts. Without the docs I'm unconvinced that it adds much value. I've actually worked on networking outside of games (in geographical information systems and telecoms testing) and obviously you can get some deeply-nested protocols but that's mostly about effective reuse rather than being some sort of 'proper' way to handle communications.

EDIT: I read the Netty docs and examples some more and although I can see that it's okay for implementing arbitrary business logic on top of arbitrary network connections, most of what it's providing isn't going to be useful for games. It's hyper-flexible but still basically kicks you out to manual byte-to-message translation at the bottom so you don't gain much.

 

@Kylotan, I have to admit that I'm coming from Unreal and released a FPS game using it. If you want to know more about unreal network system, here is a good doc describing how their network subsystem works. You will see what such a "filtering", aka, relevancy system are necessary to minimize packet usage. The mileage can vary depending on the type of the game but you can still take an advantage of it. http://cedric-neukirchen.net/Downlo...twork_Compendium_by_Cedric_eXi_Neukirchen.pdf

I've dealt with many engines but I haven't seen anything that comes close to Unreal network system and they are battle-proven and light years ahead of everyone. It's been around for many years and the recently made a huge upgrade again. Thanks to their hugely successful game and they optimized the network engine quite a bit supporting up to 100 players. Without the game, the probably never bothered optimizing it. We can at least learn something from their experiences.

The doc is not the latest but it gives you pretty accurate picture of their network system and for the latest info, you might want to take a look at this video.

They talk about Replication Graph how to send packets that are relevant at any moment. It is a part of low-level network and independent of games you are making. I believe it should be part of the new Unity network stack.

Cheers!

 

In the context of games I think the advantage is an actual architecture where how to extend it is part of the design. Netty is no more or less useful for games then any other industry. Most uses of it are actually just the protocol or protocol + serialization. Deep pipelines are not how people tend to use it.

 

Exactly my thoughts.

Unity team, if you are reading this, I repeat, hire this developer
https://assetstore.unity.com/publishers/743
He is developer of NGUI and TNet, he already worked in Unity and created UGUI for you. Now hire him again, improve and integrate TNet right inside Unity. Stop reinventing a wheel.

What a terrible way of development a multiplayer game.

While old Unity network (RakNet) was live, Unity redefined a "multiplayer game is hard", Unity made it simple.
When I was very new in Unity, I managed to create multiplayer game in a few hours using RakNet and I was surprised how easy it is. Seems like Unity going to completely ruin this "easy to use" aspect. "Multiplayer is hard" again : (

Fortunately, we have a solutions from an Asset Store, but it is sad that accessible multiplayer no longer out of box in the Unity. I no longer can say to people "With the Unity you can create multiplayer game (demo) in a few hours, this is not hard", at least, not so easy as before.

Seems like Unity team is not understanding that developers don't want a "super optimization", they just want a easy to use network solutions which just works.

 

Hi Chris - I'm quite aware of the Unreal networking system. I just wasn't clear what you meant because you said 'filtering', which means some different things in a networking context (as you can see above) whereas Epic used the terms relevance and priority (https://docs.unrealengine.com/en-us/Gameplay/Networking/Actors/Relevancy). I agree that these are useful things to have, if you have an actor replication system, to decide which actors to replicate to which clients, and how often. If Unity decides to create a new HLAPI (or even just a working example like the HLAPI) which deals in actor/entity/whatever replication, then I too hope it includes something like this. But this would be the icing on the cake, and right now I don't even see cake. :|

 

As it turns out, "performance by default" really means "usability as an afterthought"

 

Oh no, definitely sign me up for high performance code. Having to patch some unet components via reflection was an absolute pain, so it's definitely nice we'll have proper source code access now

This is definitely true though, I don't want to have to reinvent the wheel for every project and figure out lag compensation, local prediction/reconciliation/interpolation. There's so many subtle ways to get things wrong. I'd much rather have a basic framework that forces you (as in the person writing code for it) into the patterns required for this to work nicely.

I'm hoping they'll get to that kind of level. We haven't heard anything official from unity aside from just tossing the repos out there yet, right?

Still optimistic anyways, you do need a low level transport for higher level code regardless and hopefully the FPS project was simply started before work on higher level things

 

Well said.

I hope that Unity's third networking attempt works. I hope we can use it as UDP backend in Mirror. But it seems like either no lessons were learned from UNET, or I am missing something obvious here.

• LLAPI failed because it was way too complicated and written in C++, instead of a lightweight C# library that should ideally be so simple that it's free of bugs, without us ever having to worry about it.
  • The new low level layer was written in C++ again. What's wrong with C#'s built in UDP? It's easy to use, maintained by Microsoft and battle tested by millions of people in production.
• HLAPI failed because while the architecture was easy to use, the code behind the scenes was too complicated and full of bugs, never production ready. Sean Riley left and no one knew how to work with that insanity. The lesson learned was that it should be more simple, easier to maintain, more stable.
  • FPS sample is at 60k LOC again. Not all of this is networking. But it is a networking demo after all.

Also, how performant does it have to be? We can handle about 500 players per zone when using Mirror for uMMORPG. When benchmarking only C#'s built in TCP sockets, I can run 1000 connections on my 2015 Macbook easily. That should be at least twice as many for UDP. And yet again twice that if I release my game 3-5 years from now, on a decent server.

So I am wondering, do we really need 10x more performance with the new networking? Wouldn't 10x less bugs be more valuable?

 

Garbage collection and latency. Using .NET sockets in Unity is a path to the trap that has been formed for many years because of the stalled progress on GC upgrade and replacement of the Mono in general. It's logically right to utilize sockets in C and parallelize stuff on top because one of the primary goals of Unity right now is performance. And this is why the new stuff is full of unsafe code.

Yes, because future is interactive and performance is one of the keys of a successful product in our days.

The process of improvement of a product takes time but now the source code is open, and the community can make it better without waiting months for a fix like it was with the UNet.

 

Ouch

 

@vis2k Hi, I know you are doing quite of bit of good work for the community and I really appreciate it. I know you decided to make your network stack based on TCP (I read the reason somewhere) but UDP with reliable/unreliable protocol is the right approach at least in FPS games. TPC will work very nicely in LAN environment but it is basically unusable in real environment. I think you will know what I mean when the socket connection drops randomly without warnings and you cannot prioritize important data. Also TPC consumes much more data and it always want to send the data reliably unnecessarily. A simple example is when you want to send burst of movement data unreliably and at the same time enemy hit message reliably with higher priorities over the movement data. Sure there is more work involved making UDP reliable but it's not that difficult to implement. That's why Unreal uses UDP for their network engine for good reasons. Mixing UDP(for unreliable packets) with TCP(reliable packets) can be another approach but your are also increasing liability for the failure at the same time. It adds more headache than it solves the problem. ^^

A good network library should be easy/orthogonal to use but making easy means, it does all the dirty work behind. It is inevitable for codes to become more complicated and I have no problem with that as long as it does its job.

In my opinion, they just started with a basic transport layer but the big concern I have is that we have no idea where they are going. Can we depends on Unity to give us library that are really usable in production game? Or they just want to make some gesture that they care but not really solving the problem. They said they want to make the best and performant network library but I simply don't see how with what they have now. Even if they want to do it, I don't see that they can do it fast enough looking at the current pace of development. I say it because Unity's track records are not something I can write home about. If they are really trying to make a difference this time, they should be more open and share their plans so that we have something we can build our game on top of it even though it can take some time. In the end, I hope they have easy to use HLAPI-like layers but it seems like they don't even have LLAPI figured out yet.

Unity, please share some of your thoughts if you are reading this.

Thanks.

PS. I think we should have a forum for discussing the new network stack. This forum is filled with many different topics and they all seem random.

 

@chrisk TCP vs UDP is a non issue. You can easily change the transport in Mirror. By default is uses Telepathy (TCP), but it also includes LLAPITransport if you want UDP. Use the one that works best for you.

 

I didn't mean to argue about TCP vs. UDP in this thread, sorry if my post made that impression. I think I mentioned UDP too. Feel free to use what works for you

Also, I hope I didn't step on any toes @nxrighthere @BHouse . Having an official open source UDP transport now is of course very valuable, regardless of different visions for the high level stuff. It's too easy to forget to mention the good things when discussing. An option is better than no option.

 

@BHouse, Hi, if "multiplayer" is important project why can't we have a forum of its own under "Betas & Experimental Features"? Having a sticky thread would not be enough for various discussions. Track of discussion can easily be lost and people will stop reading the long thread. And I think it's more important than "Unity Hub" for sure. From what Joachim mentioned at the last Berlin United, I know you want to make the best performant network library but what I really like to know is that how. Do you have any implementation plan with timeline? A highlevel overview would be ok for now. You are asking us to give feedback but there isn't much to give feedback on other than just bunch of wishes. There has been few months since the project inception and I'm sure you have some ideas how to tackle them. I think it's the best if you can show us your plan and we can start discussing about them.

Thanks.