Game Machine - Massive multiplayer demo

This is still a work in progress but I now have a hosted public cluster and a working demo to show everyone.

The primary reason for Game Machine existing is because I think the core tech for this kind of thing should be open source. I happened to have spent some amount of time working on very large scale games, and decided about a year ago to make this happen. I'm been working on this full time now for the last month.


From the github page you can scroll down and there is a link to play the demo that will take you to a page with prebuilt clients and information on how to login. The demo is a 4 server cluster running at linode on low cost vm's.

https://github.com/gamemachine/gamemachine

The demo is primarily to show off the server side tech, the client is still fairly rough.

The demo game was built in just under 2 weeks entirely in ruby. The platform is based on the JVM with the idea that when building smaller games or prototyping early stages, it's highly desirable to stay in high level languages, and drop to java/scala only when performance dictates. I have used this approach on games handlilng billions of requests per day, and it has worked well. Most of the underlying framework and performance sensitive pieces are in java and scala.

The longer term goal is to keep Game Machine inherently scalable yet also a simple platform to work with. Building things as complex as an mmo is never simple per say, but IMO it's currently much more difficult then it should be.

The server has been tested on windows and linux, should work fine on OS X also.

I'm currently working on some demo's of how to use Game Machine for simpler tasks such as FPS games, where little to no server side coding is really required. But for the flagship demo I wanted to tackle large virtual worlds.

The full server source is GPL2, and the client is MIT.

I'm also working on a low cost hosted solution that will include a managed Couchbase server and other goodies.

Cheers,

Chris Ochs

 

Nice to see your back Chris,
and nice to see you finally delivered Game machine to the public.

I will certainly take a look at it right away to test out.

one thing just right out of the bat that cought my attention as i was reading this thread is the license model.

I like the globally known and most popular up to the date MIT license but for the client only? why not both? any particular reason you chose GPL for server? personal reason? or to retain full control of rights and distribution?

Iam not fun of GPL licenses, i mean not any more since it is quite restrictive as of you can look at me but no touch kinda deal, I mean dont get me wrong GPL was great but back in the days where early open source software distribution was starting where everyone who wished to use the software will needed to contribute back to enhance the main distribution source.

not sure, since i havent really looked closely at the full license but sounds or what i have known about and heard it is not that popular or lesser used now adays.

I think if you give full MIT for both client and your server will become really popular like some other great game engine under MIT unless that is not your intention atm.

but anyways, you are the creator of the server and you have the right to do what ever is right for you, it is nice to see you back in unity forums and glad you really delivered what you promised last year since i followed your initial thread last year.

regards and thank you for your contribution to the community

 

Quick feedback on the demo client we know it is not the best looking demo yet, but the background sound? whats up with that?

sounds like a horror movie, or like psycho 2, not cool at all. :-(


anyways, besides the cosmetic and non of my hearing pleasure sound stuff, is there any reason why in the standalone binary client demo, whatever you type in the chat input it always send the word "Red" instead of the actual typed message?

also i downloaded the unity package demo for the full world demo but when i play it in the editor i dont see no NPCS or monsters around as compare to the binary demo?


anyways, will test a little bit more and post more feedback or questions.
keep up the good work

 

Seeing the word 'red' is a bug. Just looked and that is what happens when you type in an invalid chat command. If you use a valid command it works as expected.

Are you saying you see no npc's, or just a lot less? If you are seeing none at all that would be a new one. I'd appreciate a copy of the debug logs in a quick git issue in that case.

One thing if you are running against the local server, I have the default configs there set to not spawn as many npc's. You can change those numbers by editing games/example/data/game_data.yml. zone1 is the default zone that is loaded when running a single server.

 

Ya as for the sounds and other client roughness, I'm a server guy by trade, it's a struggle for me making things look and sound pretty

 

As for the license choice. I chose the GPL because I do not want someone just taking my work and distributing it in a closed source product.

The GPL isn't really what people think it is for server side stuff. My license gives you a choice between GPL2/GPL3. GPL3 is actually better for you with this kind of software, as it has explicit protections that make it so the distribution clause does not kick in when you work with third parties such as contractors, producers, or people running the game you made. That is one reason why made the choice optional, even though most people will probably think GPL3 is worse because of all the press. GPL3 also specifically gives you a course of remedy in case you do distribute with a closed source product. GPL2 does not, and you would be subject to the whims of the copyright holder.

 

yup, that is the reason GPL sucks,

at least for me i dont like it, too much restriction, and if one were to use it or modify one will need to submit changes to the original developer, now adays people run away from that.

well, i assume some wont mind, but i find it kinda 10 years ago license restriction from back in the days when all the open source software started to boom.

MIT is the way to go nowadays, reason is popularity, even the license is very flexible people still will look for the original developer which is you, for advise, so lets say you create a public website forums for this new server, people will come to you to seek help or comments than going to otherone who is not the initial developer.

i am a true believer of why going to someone else who is not even the creator, if i can seek support from the original creator himself.
plus you get the popular vote for the flexible license. but i am just saying, just looking it at a marketing point of view,.

that is all, so not trying to make you change your mind or anything, like i mentioned before, i dont like the license in particular, i usually turn around and move on from those software under that license model, even tho i do not plan to take it and use it as my own closed source product, noone will follow me since there is the open source original creator product available at my very own fingertips for free.
why buy the cow if you can get the milk for free sort of deal,


anyways, i am here to test since i love testing new networking assets for unity since pretty much is what i like to do.
going back to my recent testings, I downloaded the full demo unity package and played it from within unity editor and i see no spawns at all, not even 1 single NPC or mobs.

the only time i get to see mobs and NPC lots of them is in the downloadable binary unity .exe demo client.

one more question, does the server support server side collision detection? i did not see that feature on the description.

hope this helps

keep up the good work.


P.S Yup, usually coders are fairly poor artists and i know that since i am a coder myself and a crappy artist. so the client demo is very understandable so no worries there we all understand, but the background sound is a No, No, you'll be better off without it., seriously, just a friendly suggestion.

 

GPL does not force you to commit back changes. It only prohibits you from distributing with closed source software. And even if you do distribute GPL with your closed source software, it does not make your software GPL. That's old FUD, but a lot of people still think that's what happens.

I'm not an idealist on this issue at all. I honestly think that in this industry, someone taking my work and republishing it under their name is a real risk, and I just don't want that happening. And like I said, I'll probably offer an optional commercial license for like really cheap, bundled with support. One way or another I'll make sure people have access to source code under licenses they can work with, as I have no intention of making money from licensing per say. I just don't like that model.

That's really strange on the issue you are having from running it in the editor. Please do post a debug log on that if you can in a github issue. I'll go take another look at the full package and drop it into a new project to see if I can duplicate it.

No server side collision detection. That's probably a bit later down the road. I've looked at it, and I think the most I could really do is just expose something like bullet, and do the work necessary to make it run concurrently and all that and then you could access the api from within Game Machine actors. Exposing a C/C++ shared library in java/ruby is no problem that's pretty easy.

 

Ok I figured it out it's bad default settings in the package. In the world_login scene in the Login script properties there is a use regions checkbox. Check it and it will work. What's happening is you are connecting to the cluster so you see chat and stuff working, but you are not connected to the region server where the npc's are. The client is assuming that since you are not using regions, you are just connectiing to zone1 and that the server you are connected to will have the region running.

I'll put up a new build with the right settings asap.

 

ahh, i see what you mean when you choose the GPL,

you want to go commercial optional providing full support to who request commercial license.,
since you doing this full time why dont you just go full commercial close source instead, after all it is your creation, if It was me and i would want to make my full time job, i would of just go commercial all the way, I would get a partner Artist who can make the levels looks pretty and you the coding part, I would of create a full mmo framework with plugins to sell, a full mmorpg demo game with most of mmo basic features , that can be a real hot selling product.

that is what most of the unity mmo server frameworks are doing, so far ive see a few of them popping up going full commercial and full support for plugins for basic mmorpg demos.


ok, going back to the demo:
ill try the regions check box.

other question:
in the server description you mentioned matchmaking, can you elaborate on that please?
what kind of system is there for matchmaking? can it be used for a moba style matchmaking where the users are matced to join some sort of battle?
or is this something different.?


thanks again

 

As far as a full mmo kit, the idea is to have kits for all the common multiplayer game types. But I want to keep the focus on making good api's and the best server platform out there. Going back over the current demo and reworking the client api to be dead simple to use is my next main task.

The real strength of this platform is that for games that do actually have a lot of users, it makes writing the server code simple. You never have to deal with threads, concurrent data structures, none of that. Regardless of scale you are not left having to architect the hard stuff yourself with Game Machine. Game Machine was never designed for people that don't want to write any code. Other solutions are going to be better for the simple things and I'm not going to try and compete with those products. I would say my solution is best for when you need a good amount of server side logic, and you need it to scale from several hundred to several hundred thousand players. That is where Game Machine will shine,

Matchmaking, chat, and grouping are all built on the distributed pub/sub system. I need to put up a ui for matchmaking, but it's reallly just a messaging application at heart.

The flow of a matchmaking system would be once you join a game, you wold probably switch those clients to the same server, and then create an actor on the server that runs the game and pass it the list of players that are playing.

This is a good example of the kind of stuff you get with Game Machine. Say you have a large moba or world of tanks style game. We have features such as consistent hashing which let you take a group of players that want to play a game together, pick an actor in the cluster to run them on in a way that distributes all these games evenly over the cluster, and connect those players to that node so they are all local. That is just out of the box pretty much. There is literally no architecture that needs creating to make that work at hundreds of nodes thousands of concurrent games scale.

 

I like what you saying, especially the last paragraph regarding the moba features since that is what i would like to test.

any chance to get more detail info about this particular features in GameMachine,

something like player login, to take him to a lobby where he can chat with other clients then he can invite them to join a group then to start queue for battle where the server matchmaking system will match other group in queue or randomly select players that requested the server to blind matchmake them automatically regarding some preset variables like lvl, or hierarchy level etc.. then one the server finds match to teleport them to an instance in some available cluster all together to start the match etc...


so you saying this feature is already out of the box in GameMachie?
if so, can you point me to the right direction of the code so i can check it out and see the actual flow?
or any documentation available?

thanks in advance

 

The UI for matchmaking is not in place right now but I'll go over how to implement what you describe.

Note that when I say out of the box, I don't mean already written for you, I mean the api provides all the tools to do the job without having to create any core functionality yourself. I will most likely be creating a simple Matchmaking system based on what I outline below here in the next week or so. I'm going to keep this at a fairly high level here because I'd rather take my time to put more detailed information on this kind of thing on the wiki as opposed to email.

The starting place in the client code is Example/HelloGameMachine.cs. That is the base for any game that gets you connected and sets up all of the internal systems.

The next thing to look at is GameMachine/Chat.cs. You will see the following line in Start() that subscribes to a chat channel:

messenger.JoinChannel("global", "subscribers");

That is how you join a chat channel. Also notice the callbacks that are setup where you receive messages and handle private group invites.

A Matchmaker UI is basically going to be a lobby that's tied to a chat channel. Players could then create other channels for people to join that are public or private. Chat.cs and ChatCommands.cs have examples for how to handle most of the messaging stuff. There are private channels as well as public ones. The messaging system is basically your toolbox here, and the chat examples in code are the best available documentation currently until I have the chance to create an official matchmaker ui.

On the server side I would create an actor that handles the matchmaking flow, while the selection process would be abstracted out into some type of pluggable system that it calls but doesn't care about the implementation of which is left up to the developer. To obtain the information and persist it on the server you could just create another actor that is responsible for handling that. I view that as orthogonal to the core matchmaking workflow and best decided by the specific game.

When some/any client sent a message saying we are ready, the matchmaker actor would ask for a selection of an opponent from the plugin, then persist a key for each team, with the value being the other team. Every time the actor gets an update message from the client it checks to see if the team that client is on has a key persisted, and if so it then notifies the client of what opposing team they will play, and also sends them the address of the actor that will run the actual game.

The actor that runs the game would be a distributed actor that actually creates a child actor for each game. That child actor can then run the game itself, create other children if it needs to, whatever is called for. The key used to get the distributed actor could be a hash of both team names combined.

Some exact implementation details might vary of course once I started writing it, but the above approach should work fairly well.

 

Hi Chris,
Ive been away for the week, too much work, glad weekend is just starting to test servers again.

hey, i see you uploaded a new demo for matchmaking, is that true?
https://github.com/gamemachine/gamemachine/wiki/Matchmaking-&-Teams

but when i try to download the demo i cant download, it throws me an error
https://s3.amazonaws.com/gamemachine/gamemachine-example.unitypackage

it throws some sort of xml error in any of my browsers:

any advice?

regards

 

also i am trying to build the mono server

can we use this server instead of the jruby or java server?


if so, do we need to port all jruby game play files to c# and then recompile server?

or how does it work exactly?

can you elaborate a little bit on the mono server mechanics? or how does it work, etc...

i ve changed the config.yml file to use mono:

then i run the server.exe and it crashes, what else do i need to set, i see there is a test_actor.dll in the config as reference, what exactly is this and how or where do i compile this .dll?




also, i got the jruby /java server to run locally in my pc, and tested the demo world client on it it work ok, but..

the chat still dont work, when you type any world in chat fields it displays the word "red"


also, i created the new players from http in the admin panel on port 3000 but where is this databased stored at?? i thought it was a mysql driven database??

but now i know it is not, so where exactly is the users database, or persistent information stored at?? is it a flat file db? or some other ?


thanks in advanced

 

The more time goes by, the more I'm considering just dropping the mono pieces. I'm a huge advocate of using the right tool for the job and not just the tools you know, and the mono server goes against all of that in a major way. Game Machine is really designed to maximize the synergies of JVM languages. The fact that C# is hugely popular in the game community did not escape me. But for what I wanted the platform to do, the JVM was the right tool for the job by a large margin.

That said I'm happy to assist if you would post the errors you are seeing either as a github issue or on the mailing list (link at bottom of main github page). The mono server is just a simple socket server. It's designed so you have one instance per Game Machine node, and Game Machine starts it up for you when it's enabled and stops it when it stops.
You talk to it by sending messages to it like so:

Endpoints::MonoGateway.find.tell(message)

And in mono there is a similar way to send messages back to the java side addressed to the actor you want to receive it. I haven't tested it in a few weeks so it might be using dated message formats and bombing because of that. But other then that it should just work.

The default database is a java database called mapdb that is local only. There is also built in support for redis and couchbase. There is no relational database support right now. I've been thinking about the right way to support that. It's one of those things where everyone will want to use it, even though it's not usually the right choice. Game Machine is entirely non blocking by default. You can do blocking operations but you can also easily shoot yourself in the foot with them. I view relational databases as useful for loading up static game data, talking to external databases to load things like user credentials, or for when you really need full transactions (trading, etc..). That said it's not really hard and it's on my short list to provide a solution there.

FYI the latest version up has a full teams/matchmaking demo. It's in the example Unity project (the small one).

Chris

 

like i stated before, I still can not download the demos, neither both,

https://s3.amazonaws.com/gamemachine/gamemachine-example.unitypackage

or

https://s3.amazonaws.com/gamemachine/gamemachine-full.unitypackage

i get some error when i click on the links to try to download:

I think your download links are corrupted or disabled by your amazonaws.com web server

so am not able to test any of them as of yet.

any chance you can create a mirror somewhere else?

please advice.

 

Damn sorry about that I didn't notice the previous message. I forgot to set the permissions on the last update. FIxed now.

 

Thank you,
i just downloaded the unity package and about to start to test.


now, regarding the mono server, you saying that you think it will be best to just use jruby instead?
i mean for performance reasons? so I assume even better will be to code all your game logic in java instead?


what i am trying to understand is on how the server works, who calls what and when, how the messaging system works,

for instance one is used to RPC, to talk to the server or vise, so i assume here there is not such a thing?

I downloaded the server and followed install directions and it all worked fine, but during the installation noticed that one need to build /compile etc..then create a new server. so I assume the server is not a standalone one, where one can just move to other hardware node and fire it up? so i have to go thru the whole installation compilation again?
I guess what i am trying to say is that is the server bound to its original compilation environment hardware?

how about if I update the server files, do i need to recompile all back again and follow the installation instructions?


jruby is new to me, seems like python but with some strange syntax, java am not that big of a fan altho I have used it before when I was playing with multiverse server, but i can live with it. so hence wanting to try to use mono to be familiar with the coding., but still am not able to get it going, , maybe i am not understanding how it really works yet.

If mono were to not be viable, id rather use all in java, is it possible? but seems that all is configured by jruby.
, like a mixture of both.

as for the database, id rather have a system like mysql to store persistence data, how hard will ti be to do my own mysql db manager, or will it be like a complete rewrite of the auth system??

or you think the existing db system will cut it for a medium to large project?

 

So look at jruby as your scripting language. IMO it's a superior solution to other engines that embed something like lua, because it's native to the platform, it's a first class citizen. Especially on the server you don't always need the fastest language for everything. You are not working with confined resources like you are on the client. There are areas where you can make tradeoffs on the server that you just can't on the client side. There are things I keep in java and things I keep in jruby. Generally speaking I use the most productive language I can at all times, until tests show that I need to use something else.


For deployment you have a number of options. The simple approach I use for my small cluster is to just copy the whole directory out to the servers I am deploying to using capistrano. I preinstalled jruby in my home directory on the servers. The following page on the wiki has an example Upstart script I use to manage the server:

https://github.com/gamemachine/gamemachine/wiki/Game-Machine-Cluster

You can also bundle jruby with the app using an all in one jruby jar that is available. There are also tools to bundle any jruby app as a stand alone jarfile. Or you can use Puppet or Chef.


Explaining how the whole system works is bit much for an email. I think it would probably be much more productive to help you accomplish something specific. I could wonder all over the place explaining stuff at a higher, non specific level and it probably wouldn't help you much.

As to databases and auth that's something specific I can answer more easily.

The auth system is pluggable. In config.yml there is an auth_handler setting that points to the class that handles authorization. The default one points to one I made up for the example game. You can basically just copy that class and implement the authtoken_for method and that's it. That method returns the authorization token (a string) that your authorization system gave the user.

Also, if you wanted to write it in java you can do so. In games/preload.rb you would put this to pull your class into the global namespace:

java_import 'namespace.MyClass'

Then in config.yml just set auth_handler to MyClass

Then just implement authtoken_for in java.

Any code you put in java/src will get compiled and stuck in java/lib when you run game_machine build. Everything in java/lib is automatically loaded by jruby, the java_import is just to put it in the global namespace.

You can do everything in java if you want. Just create java actors. All actors inherit from UntypedActor, and they all work the same regardless of the JVM language you choose. There are a number of wrappers on the jruby side that make life easier, but other then the json models which are not implemented in java, the other stuff is just there to make common tasks less verbose.

For databases the way it works now is that there is a distributed in memory object store that persists to the database you have configured, by default mapdb. It's a layer over any key/value store that makes accessing it lightning fast and is able to handle high volumes of updates. It was designed for things such as game state that change frequently. If you are persisting game state every second or so, that's what you want to use. I suggest using couchbase as the backend store for a real game. I use mapdb for development so I don't have to run couchbase.

For stuff where a relational database makes sense you can certainly use mysql. I would create a connection pool at startup, using preload.rb as the place to call into your java class and create the pool. Then just use it inside the onReceive method in the actor. Another approach is to create one connection per actor inside preStart. Game Machine can't really offer a full database abstraction layer, it just doesn't make sense. What does make sense is to put up an example of where to create a connection pool and how to access it from within an actor. But beyond that there is really nothing Game Machine specific going on there.

I'm going to be working on a lot of specific how to/getting started stuff here soon, getting a lot more into concrete, specific examples of how to do basic stuff.

 

I think this will be the most healthy way to help the ones interested in this server backend.


reason is, it all sounds good, but even being a game programmer, knowing and used most common used programming languages in popular game engines,
ie: c#, c++, python, lua,
when I look at GameMachine, i am very puzzled, that do not even know where to start, jruby simply i never thought of this combination for making a server backend for this kind of setup.
most common java servers plugins for unity use mostly java server plus python as scripting language..

so I look at your existing documents and look at the server files to understand what the hell is going on behind there , but still am lost


so I think documentation and examples will make a lot of difference to make it less painful for interested people to get started.
because believe me when i say that right now working is "painful" to get started with the current state and docs.

1 language programming is more than enough for hair pulling and debugging experience when developing a game meaning c# in this case, but learning c# plus Java plus jruby, plus understanding a non familiar/common db structure, it is excruciating experience imo.



so I think doing examples and tutorials will ease that pain, at least for me. or others who might interest this engine.

imo, of all unity servers plugins available outhere, only one plugin caches my attention when it comes to documentation, for me the best documented server engine plugin for unity and the one that makes more sense and easy to understand i think is the one from uLink website,, very straight to the point and very relevant, also they have their tutorial section with most common usage examples.

so if you go to their website docs you might find it very helpful to at least give you an idea on how or where to start documenting your own for GameMachine , here is a link if you are interested:
http://developer.muchdifferent.com/unitypark/uLink/uLink
hopefully it will help you or give you an idea on where to start for documenting yours .
hope it helps.

 

So I'm putting together something that will be familiar to people to get started, which is a basic network sync of components or other objects to other players (routed through server not P2P) or it can be server authoritative.

It requires using attributes to declare the fields you want persisted for components, or you can put one class level attribute to declare all public/all. It uses protocol buffers so as far as I know it's more space efficient then the Bitstream or any other product that just does simple conversion to bytes. It's also smart and tracks last known values on the field level, and only sends fields that have changed since the last transmission.

Just working on the special types like Vector3, etc.. right now.

I've also started a tutorial series that's one scene per tutorial and starts out really basic and then works up, so you don't have to digest so much all at once.

It's basically a case of I know what needs to be done, there's just only so much time in the day

 

I am not sure if i am following here,

you mean you doing some sort of RPC unity like system for GameMachine, where we can send method calls client to server or vise? you also mentioned that "(routed through server not P2P or it can be server authoritative)" not sure if understood the routed through server, you mean this is non authoritative and P2P will be authoritative?
can you elaborate a little bit here, i am kinda lost of maybe i missunderstood.

this is greatly appreciated. if you need a tester, i can be your guy by testing your tutorials before you release them to public if you like , maybe it can help me speed up the learning process for me
or maybe you might need some help on the art side. to put together a nice scene to show off the tutorials etc.. ill be glad to contribute and help you if you like.



I tested the matchmaking demo, and it seems to be working great.

so, I notice that the matchmaking lets you do so far:
- server shows which players are logged in and their grouping status (if they are in group or not)
- player can start a private or public group.
- choose group name and group size.
- once group started one can invite other players to join the group.
- once the group is formed and if other groups are formed aswell the server will let you find match to start battle or game.


1- Now, i see that when the server finds you a match, it shows that the match started in the server, but can it be like to send you to a private instanced scene or small region so it can be a private event for the matched groups only. maybe the server is already doing that, but the client is not loading the instanced scene yet? if this is the case what callback should i look in the client to make the client load a scene for the private match?

2- also, so far it is a manual group matching only, where players forms the groups by inviting or others choose an existing available group to join,
but can it be some sort of blind matchmaking, where one player chooses a blind match option and server chooses an available player looking for same blind match style and groups them together? like most mobas do?
or where do i look and see to add that functionality?

3- i noticed that it still has the chat bug, chat just do not work out for me, nothing i type is getting sent, the server returns a "red" word, ive tried using a slash before the text, etc.. but it does not seem to work for me yet.


regards.

 

I have doc page for matchmaking

https://github.com/gamemachine/gamemachine/wiki/Matchmaking-&-Teams

Yes it's just not loading in anything at the point where the match is made. That's where the demo ends on the client side.

There is a server side plugin you can configure that lets you pick matches on the server, filter a number of things, etc.


The network sync stuff is really more for simple state synchronization between clients, works a lot like the Ulink state synchronization with some small differences. I should have a working demo on that in a couple of days.