Game Server Hosting: feedback and questions

[Edit: removing old sticky threads as our new SA team begins to manage this forum]

In a September ‘18 blog post, we shared more details about the long-term vision for Hosted Dedicated Servers at Unity, as well as some details about our first steps forward that will be available in alpha soon.

This sticky post will stay up-to-date with blog posts and talks that are relevant specifically to the the future of server hosting and orchestration at Unity. Here’s the list that exists today:

Talks

• "Connected Games: Real-time Multiplayer" - Unite LA 2018
• Unite Berlin Keynote - June 2018
• "Hosting AAA Multiplayer Experiences with Multiplay” - Unite Berlin June 2018

Blog posts

• “Multiplayer Connected Games: First steps forward” - Sept 2018

If you have feedback and questions specifically about our new dedicated server hosting and orchestration solutions, please post here to ensure the support team and developers dedicated to this topic see and respond.

 

in this video :

its explained that there will be solutions for turn based (level 2) mobile games server aka not realtime, . is it possible to know any plan about this ? what solution will be used? server within unity ? open source or baas ?

 

Hi Tiny-Tree; we're focused initially on solving challenges for the "L3" space, and we haven't decided exactly which method is best to implement L2 games.

The typical set-up is a fast / real-time database to pass game-state back and forth, with potentially some on-demand cloud functions that run to provide server authority for commonly-hacked logic. That said, we are open to other possibilities.

I'll turn the question back to you - what are you hoping we do?

 

i would hope for an open source level 2 server, that is made by industry specialists as a boiler plate for any kind of social games. same as the matchmaker developed with google and unity.
for api something like having a basic rest api with db connectors that we would call from client through sockets/websockets to make it fast and full duplex and allow to receive notification from server to client.

having an administration panel would be out of scope and restrict its development for user by fear of breaking things

it could be run on unity and ecs but i guess it is better in the end to have that project written in go or for node js.

No need for extra features like: clans, daily reward, achievements, because its really specific per game and out of scope and it would be better to focus on modularity, plugin implementation to help user made ecosystem of server plugins that would be sold on asset store for example

• simple example/boilerplate of how a successful game would have made its social backend that we could build upon
• something that could scale vertically or horizontally without too much effort for at least 10 k concurrent users on a single instance which is easy to do with a single node server
• an unity sdk that is ready for ios/android/webgl ( extremely important)
• can be run on our own cheap vps

here is few example of what is available as open source:

Parse Server:
its a good solution, cheap to host but overly complicated, too much features, not good for small indies.

Nakama :
ready for prime time, its cleaner and more easy to work with it than parse but to get the multiple instance feature require to buy it at an undisclosed price, simple to maintain and expend upon

 

Hi, I'm really dying to know what the new network library will look like. We should have something on our hand by now as it was promised during United Berlin but there is hardly any information.
I'm really concerned if it's developing so that it can scale up to ~100s of players. The followings will not matter much if we are making network game for ~10 players also. Any brute-force method will work.

Joachim said that Unity wants to make the best networking library and I was really excited to hear it.

Anyway, before we can give our feedback, we need something to review first.
I hope you guys can share it before it's too late, once it's out the door, it will be much harder to change.

Without more information, I can only ask what I expect from the best network library. The followings are what I expect from what the best dedicated network libary should do when sending packets.

• Whom to send: Everyone, Owner(client can own the object), Team members and etc.

• What to send: Various Filtering Setup depends on distance, priority, visibility and etc. (delta bit compressed)

• How often to send: Depends on the importance and moving speed, the frequency of sending the packet should be configurable.

• Reliability: Both reliable and unreliable. Basically, movements are unreliable and everything else needs to be reliable.

• When to send: Once at the initialization?, or whenever it becomes dirty?

If it supports the feature above, it can be used not only for FPS games but probably all games. Many games will not need such sophisticated filterings but they can all benefit reducing the size of network packets.
Yeah, there is deterministic stuff but I heard that you guys having little problem supporting now and I fully understand. It's much harder to do with the heterogeneous network environment. And of course, people want MMORPG but with exception of the persistency, the fundamental network layers are the same. And the persistency has nothing to do with the network itself.

ECS has lots of potentials and I think it can get us the most performant network library, period. I just hope it also flexible enough so that it sends the optimized data that are absolutely necessary at the each sends instead of sending duplicated data blindly because reducing the size of network data will play a very crucial role in scaling up.

Thanks for taking my opinion into consideration in advance I'll wait for the good news.

Cheers!

 

1) Will free Personal license Unity user able to use free tier to try and test before shipping game? And what's price for each plan?
2) Is there currently any server available now that able to store and load persistent player data meaning that super fast storing and retrieving player data?

 

IMHO, two things are important to almost every project.

1. BuildPipeline for the dedicated server. Editor should be able to generate a slim build, excluding unnecessary textures and shaders safely.

2. Clear roadmap about when new features are coming and when it's production ready.

 

Thanks for the feedback - this all sounds great. We're gearing up for the preview/alpha release of all real-time multiplayer tech very soon (weeks not months), and we completely agree that we need to release early and often to make sure we meet the needs of real games. The initial releases will not have all the features you list above, but we do hope that many will follow in frequent ongoing updates through next year.

You also note that ECS / Job System integration is critical, and we also agree - to reach the scale of 100's of players, performance is key (on both client and server code). More concrete details about this integration will be discussed in some upcoming Unite LA talks.

 

1. We haven't announced any pricing plans just yet, and we know cost is important to all devs (especially the indie community). We'll announce something soon once we've explored all options to keep costs down for developers.
2. This is a great question; persistent player data is important for many different types of games (real-time or async), and we don't have concrete plans in place. However, your feedback is helpful as we're prioritizing 2019 plans.

 

1. Agreed completely - in the short-term, we've been redoubling efforts to optimize and fully support the headless version of Unity runtime for server. In 2018.3, we will also introduce workflow improvements that include a managed server #define, so the code separation can be respected at compile time. Longer-term packages and a very minimal player will provide more and more modularity to ensure the server is only running exactly what your game needs.

2. It's a good point that clarity is important, AND we also believe that adapting to user feedback is equally critical. We don't have a long-term roadmap that is completely locked-in and shared because we know that there is so much we just won't know until we begin working with devs directly on the new tech. We also won't call anything production-ready until we've seen sufficient evidence that real games can succeed using the tech, so again, no set dates - just a goal of frequent updates and iteration.

 

Thanks for replying, I really like your definition of "Production ready", my respect!

I understand how hard it is to make a roadmap for the unknown. As a developer, I plan technical solutions for my team ahead of time. A whole game may need a ridiculously long time and the worst thing could happen is that a close sourced core solution has been obsoleted or delayed, totally a nightmare. I think every part of the upcoming network system are fundamental to networking games(especially L3 and L4), therefore something like a roadmap could provide stronger confidence to developers and help the iteration. Just my personal suggestion

 

Hi,I want to know if unet change to game server hosting,will the prices go up?Currently,it is just massively expensive for level 2 (turn based)games compared to phonton and smartfoxserver.So,you should lower the price.Thanks

 

When this goes live Unet will be removed, not changed. If you want to use Unet on a rented server, you can already do that through any number of 3rd party server providers.

 

Unreal 4.21 is released today with huge updates as always. Notably, it includes the Replication Graph and here is the brief description why it's necessary for performant network libary. I really hope Unity study it and has something to offer... soon, not in 2 years.

 

Hello,

For L2 tier. Since you're pairing with GCP, how about GAE (google app engine)? The product is very reliable and scalable. It comes with many out of the box features such as Authentication and others.


I'd prefer to have an App Engine logic written from experts, to manage most of the common issues with games, and adding some features such as matchmaking. Also it should be controlled from Unity Editor.


For example developers can write all serverside logic in #if_server and only libraries used will be compiled and uploaded to a GAE version to scale on demands.

 

Hi!

I'm curious as to how you're planning on leveraging a solution where you can utilize the navmesh and colliders crafter in the editor on both the client and the server to make sure users don't cheat. Because that's something that would be really nice to have, not having to export navmesh and then through black magic use it on the serverside like some do now. Any thoughts on that? And how close are you now to showing off some serverstuff? Couple of weeks has passed *eager*

 

Anyone who integrates with Steam gets access to a whole bunch of other features there. The game gets a distribution platform, beta builds with separate download alternatives (several can be juggled at once), and chat (lobbies and DM) + direct game joining. So any of that should be roll your own or just use Steam like sane people do.

Ranked matchmaking is something that can be specific to each game, so it's probably not always straight-forward. A first-/third-person shooter may simply rank you on win:loss ratios, while other games rely on progress and acquisition of powerful upgrades.

I think all a system provided by UT needs for Steam games is unranked matchmaking (random player matchup), ranked matchup (developers assign ranks to players based on their criteria, UT's system only finds players in the pool at that rank) and maybe toxic player segregation (scum and villainy get their own ranked and unranked pools).

For mobile games you kinda need some sort of account and lobby system. It's very convenient to just let the client use a soft account registration by device IDs of some sort, then optionally register the account to an e-mail address later. If not part of a service provided by UT it would be useful to give people examples. Indie game developers are NOT generally well-versed in backend server programming

If using UT's server hosting, some control over it in a management panel is necessary. If you're going to run instances as needed based on capacity, all of that has to be made stupidly simple. But if a game is in testing you don't want it to run too many instances because of costs.

We don't know what sort of prices are intended, but if I were to run a small beta I'd like to limit the number of players so it doesn't get too costly. A turn-based strategy game would presumably also be less of a stress on servers than a 60 ticks per second FPS, so equal prices there wouldn't be fair. I foresee a whole mess there

A lot of this has already been mentioned, but they're things I consider necessary, so I repeat:
- The server should be able to build from version control.
- Slim headless server builds - drop textures and sound for small uploads (you should only need scripts, mesh and scene data at most).
- The replacement high-level networking API needs to be finished

A lot of the things typically ask for can be done by running a VPS or dedicated server with some backend services, but the two things which could make it less ideal are ability and proximity. As mentioned above, knowing how to make games doesn't necessarily prepare you to write server-side software, and a random VPS out there might have connection issues both to the game server and to some or all players in a match. Many indies just want to concentrate on the game, and as few stops as possible to get all the services working together is best.

Personally I'd write (and have written) matchmaking/account systems myself. There isn't much to it for a dumb matchmaker that just finds open games - you just need either a web API that returns JSON or a gRPC API with compact binary data, then the only slight effort is if you want to make a live-updating webpage to go with it.

UT could probably provide something like that fairly easily, with an SDK to quickly connect clients to it. With gRPC you run a CLI command to export the client code automatically, and all you need to do is write actual server code.

 

I'll leave it to Tim and others to chime in about navmesh/colliders.
On the server front, do you mean access to the alpha for server hosting? The roll-out has begun *very gradually*, and especially with the holidays coming up, there likely won't be another significant round of projects added until after that (since we know most devs will be OOO and not on-hand to investigate challenges). Thanks for the patience!

 

Thanks for all the feedback. The good news is that much of this we are actively discussing today, and while I can't promise anything, I can say that we are at least aware as we plan for 2019.

 

Any update on L2 connected games or others? It seems to be there is only one team. But for such a big project like Cloud, doesn't it seem better to have a team for every type of networking L1, L2 ..etc?



Also for L2 can we expect something like GameSparks? (Amazon's BaaS for games)

 

I'm reading threads and watching videos and i have one question that I can't find answer to: Will I be able to do my own dedicated server i.e. player hosted server with this new unity networking or will I be stuck with cloud based paid option?

 

With the new Network framework how easy will it be to run an "in proc" server on the client? So I can build a game with the client/server architecture but still be able to run a complete offline session on the local client (that would run client and server in the same process and with faked network communication)

 

The thing is, anything (including Unity shipping today) can be used to write "isomorphic" code. But if you click on that link your head is going to explode. You'll think like, oh man, what does this web S*** have anything to do with answering my question? It absolutely does. The problem you describe has been solved actually multiple times in the networked communications (i.e. web) space. But those other people don't use Windows, they don't use C#, they don't use Visual Studio, they interact with the command line a lot, they're basically doing stuff that makes a typical game developer have a headache and hit the home button and rage quit. They're also making websites, sure, but it takes a certain amount of technical maturity to realize web sites and video games have a lot in common.

It's not really about what Unity can do for you here, because if you know that your game logic has to be "embedded" (another way to describe what you're talking about) and you write it that way, you already know the answer to your question is yes.

I'm not a Unity employee or anything. I'd go into detail responding to some other engineering questions, because it's fun, but I'll pass for now. It's more interesting to talk about the why, not the how, of multiplayer gaming. It takes us to more interesting engineering questions, which is the the purpose of this thread.

Like ECS and Networking and whatever isn't really about making a multiplayer framework. We're getting that as a side effect of the primary objective, which is to make Unity capable of developing a networked first person shooter.

Networked first person shooters have been the top of the Steam charts pretty much for all of Steam's history. So Unity was pretty happy occupying its niche despite the PC market, not because of it. Its inability to make FPSes took on new urgency when all of a sudden the biggest iOS games are Fortnite, a clone of Team Fortress 2's Zombie PvE mode, and Honor of Kings, a licensed LoL clone in China. While there was plenty of evidence that real life human beings were willing to play MOBAs on their cellphones (an utterly insane proposition to a normal person like me), nobody could have anticipated that Fortnite, with its utterly grotesque on screen joysticks, would be such a big title.

You couldn't look at Minecraft, another networked FPS, and say "Well whatever, you don't need to aim in Minecraft, there's no time sensitivity, its networking stack can be as simple as a 2011 Java application by some billionaire rightist recluse," and anticipate that Fortnite would be as compelling on a mobile device as it is today.

So what does this have to do with the technology we're maybe going to see? You're going to look at the new framework. It's going to LOOK super appealing. And then when you try to do anything with it, it will be an INSANE HEAD SCRATCHER.

Like go ahead and try Improbable's GDK. (1) Writing a poorly-specified programming language via conventions and C# Attributes like they do (and like how many frameworks operate) and (2) operating it via a bunch of GUIs, like a website, is a losing proposition. It demonstrates a fundamental immaturity of technical design that, while I'm not really the most qualified to comment on, is endemic in the game services industry. It's endemic to engineers who overpromise and underdeliver, that's for sure.

Part of it is that the multiplayer framework problem is very product manager driven. That should have been apparent with this logic of, "Well look at the top of the Steam and App Store charts." It's more fundamental than that.

Like you can't alienate your Visual Studio, C#, GUI people. A headline feature basically has to be "It's still C#!" TinyMode (bless its heart) launched with TypeScript, which was a technologically necessary (and wise!) move, but I'm sure it's seen as a faltering experiment in introducing users like me to a bunch of new programming paradigms (including the language) all at once.

Another headline feature has to appeal to that GUI part. Like "It's gotta be GameSparks!" Listen, nobody who ships a multiplayer game people actually play wants to be awake at 3am clicking UIs to non-disruptively deploy their games. Anyone who has any experience running a GOOD multiplayer game knows how disruptive (in the bad, not Silicon Valley sense) GUIs are and how much better full, bash-script-specified automation is.

Then there's this obsessive focus on memory management, righting some wrong of C#'s original Garbage Collection sin. Users obsess over allocations in a way that I've never heard helps a Unity game get delivered, and we already have all the resources (pun intended) in the world in C# to preallocate. Yet here's a framework bending over backwards to give you even more nativier buffers.

Attributes! Interfaces-by-convention! GUIs! Listen, the reason we have this stuff is because you people (the users) care and think it's super duper #1 important, even though it really isn't pretty much ANYWHERE. Except in first person shooters, of course.

Let's summarize then, because this is going to reveal the key insight into how this technology works:
- People who make FPSes always choose tools purpose built for them
- The Unity Network stack is "purpose built" for FPSes
- But it LOOKS like a general networking stack for normals
- People who make FPSes sometimes freak about the same stuff normal Unity developers freak about, like memory management
- But people who DON'T freak at attribute-encoded programming languages will FAIL at making an FPS anyway.

Maybe you disagree. Maybe you point out that UE4 uses macros everywhere. Macros are practically the attributes of C++. Those macros are an essential way their Blade Runner-inspired Replicant multiplayer concept works.

But then listen to yourself. You'd be defending C++ macros. Literally the most horrible feature of that technology. Who's going to trust someone who defends C++ macros to make a safe, easy-to-use, flexible and legitimate multiplayer networking framework? So don't even think about talking about UE4.

Simply put: There exists no person who is (1) capable of making an FPS correctly, (2) has a favorable opinion of an attributes-flavored multiplayer framework, and (3) doesn't work for Unity.

A similar thing played out in the web space many times. XSLT and SOAP. Horrifying! It's playing out again now. AMP. GraphQL. If you're a cynic like me, React. These are ALL ill-specified things that are secretly code / a new programming language disguised as something else. Because they LOOK like something else, they happen to be appealing to the sick patients / zombies of the web world, the front end developers with no back end experience. Unfortunately, those developers are incredible numerous. Adoption of these bad ideas SKYROCKETS, making product managers look INCREDIBLY GOOD, while people who actually use these things to ship wind up totally replacing them if they make anything super popular and long-term (like 3 years old) anyway.

Blizzard had many years to evolve StarCraft into WarCraft 3 (the grandfather of their current networking stack). That system is so flexible but so difficult to write! Truthfully, I'd be delighted if Unity releases a multiplayer framework as polished and flexible as 2007's StarCraft 2 and takes 10 years to do it, instead of pushing out a bunch of C# attributes in 1 year.

But I also know that the SC2 editor is a huge pain to use, and god help you if you try to make a serious game in the Overwatch workshop. I get that a lot of these things are driven by factors totally beyond your control, like Apple's maddening decision to require all code to be AoT compiled despite not being able to deliver a fully-specified AoT compiled applications programming language even by 2019. il2cpp and its serendipitous bonus feature, the emscripten WebGL pipeline, are truly miraculous technologies. So I have faith in both the technical chops and luck required to deliver on a networking framework.

So keep at it Unity. Please be judicious about your technical choices. I'm a happy user but definitely concerned about Multiplayer Try #3 here.

 

This is the big question.

My literal reading of the news from the company is that we will NOT be able to do this. If so, then the company is choosing to not reply to your post. If so, this is the end of free gaming using Unity, as the "free" personal licenses will have no value if the thin-server-P2P option is deprecated. Many coders who dream of writing a popular game do not have the income and a credit card to pay monthly fees to use a commercial server to build and test their game.

 

The networking alpha is available for everyone to download and see. It is not some conspiracy. The fact that Unity's cloud solution is not yet available, yet you can use the networking alpha for a dedicated server game right now, makes the answer obvious.

 

Joe, the presence of an alpha version when the company is in testing does not answer Krajca's question. Unity needs to be clear as to whether their plans right now are for the platform to support 3rd party and, in particular, personal home-based servers. I take as much information from their refusal to answer the question as you do from the alpha version.

 

Absolutely you can run your server runtime wherever suits your game best; the ecosystem will remain open.