Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

UniVerse MMO Framework

Discussion in 'Works In Progress - Archive' started by JamesPro, Jun 18, 2014.

  1. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    I would like to present UniVerse MMO Framework. UniVerse is a Fast, Reliable UDP based Networking Framework built on top of a Customized version of the Lidgren Networking Library.

    UniVerse features:
    - UniVerse Lobby - A full complete Lobby System complete with a Console based Lobby Server, Game Server SDK and Client SDK with example Unity based GUI.

    - Console based Authentication Server

    - MySQL Persistence layer

    - IronPython Scripting Engine built into the UniVerse Core so it can be accessed from a Server or Client

    - Server SDK which can be used in ANY C# based Application (Unity3D, Console, WinForms, XNA, ect). The Server SDK holds no dependencies on Unity.

    - Unity3D based Client SDK. (More Client SDKs to be added in the future including our own Custom Client using Ogre3D and other opensource Libraries).

    - Custom Messaging System that allows you to register to receive specific message types in different parts of your code

    - Game Logic based around Server and Client side Plugins which register to receive the message types they care about.

    Example HelloWorld Server Plugin:

    Code (CSharp):
    1. class HelloWorldPlugin : IPlugin
    2.     {
    3.         public void Start()
    4.         {
    5.         }
    6.  
    7.         public string getName()
    8.         {
    9.             return "HelloWorldPlugin";
    10.         }
    11.  
    12.         public void onActivate()
    13.         {
    14.             UniVerseCore.EngineCore.RegisterMessage(HelloWorldClient.MSG_HELLO_WORLD, HelloWorldHandler);
    15.         }
    16.  
    17.         public void onDeactivate()
    18.         {
    19.         }
    20.  
    21.         public void HelloWorldHandler(object obj)
    22.         {
    23.             NetIncomingMessage msg = (NetIncomingMessage)obj;
    24.             Debug.LogError(msg.ReadString());
    25.  
    26.             NetOutgoingMessage outMsg = UniVerseUnity.Instance.CreateMessage();
    27.             outMsg.Write("helloworld.helloback");
    28.             outMsg.Write("This is a message from the Server!");
    29.             UniVerseUnity.Instance.getServer().getServer().SendMessage(outMsg, msg.SenderConnection, NetDeliveryMethod.ReliableOrdered);
    30.         }
    31.     }
    Code (CSharp):
    1. class HelloWorldClient
    2.     {
    3.         public static MessageType MSG_HELLO_WORLD = new MessageType("helloworld.hello");
    4.     }
    I have created some basic Demos with typical MMO features and the demo runs great. I needed more work on AI, and player/mob Movement but I had combat looking great and other features. I am currently developing a Lobby based game called "City Life" with UniVerse and I'm doing some Prototypes. I haven't done any large scale testing as of yet to see how many players are supported on a single Server but that will come as I continue development of City Life.

    City Life's Forum Thread: http://forum.unity3d.com/threads/citylife-morpg.251795/

    I plan on packaging the whole system into a MORPG or MMORPG (Depending on my testing) Development Kit that will include ALL of the features of a typical Class based WoW Style MMO. Developers will then be able to script their own Quests, Items, Mobs, ect. using Python.
     
  2. sandboxgod

    sandboxgod

    Joined:
    Sep 27, 2013
    Posts:
    366
    Sounds pretty interesting. Surprised not much responses but guess there is a little bit of competition in this area or most Unity games are probably single player
     
  3. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Last edited: Jun 25, 2014
  4. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    Been following your work and sounds great

    was trying to test but, do not know where to start, no docs :-(

    does the server run inside a unity instance too?
    how does it work? client connect to lobby, then lobby connects users to server , can it be several servers connected to lobby for a zoned mmo type of game, can the servers intercommunicate eachothers for handling players acrossed them?
    or is this some sort of room based game being a unity server instance a single room where the lobby links current rooms connected?

    any chance for a demo project or at least a quick guide, instructions or description on how it works, example of mysql db dump etc.....

    thanks
     
  5. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    Also, i fired up the UniVerseLobby.exe
    but it seems to be a CPU resource hungry little app, eats more than 50% of my cpu cores when running,
    any advise on how to make it idle while waiting for connections?

    unless this idles once a server is connected??

    please advise
     
  6. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    The Lobby Server is optional. You can create a MMO with out using the Lobby Server. I'll have to work on improving performance of the Lobby Server.

    The package comes with some example Plugins both for the Server and the Client. The Authentication Plugin shows current use of MySQL. I believe there are also example on how to set up the Python Scripting Engine. I'll add Documentation as I further Develop UniVerse. The best I can offer right now is to come join our forum and ask any questions you run into there.
     
  7. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
  8. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    ohh yeah,, the new lobby now idles beautifully. 1% cpu load :)
    at least it idles correctly and performance will depend now on current connection load and not the actual app idling high.

    thanks for the quick update.


    regarding the client and server plugins,

    what i meant is a very quick doc about how to put it all together, the current files for client and server are just plugin folder and script folders, no scenes in the files no components attached etc...

    so my questions was a request for a quick startup doc.
    like:
    what script goes where to initialize the server and what script goes after etc... to connect server to lobby etc...

    so far they are just scripts and I am just guessing what goes first and where and how, so a very quick tut or maybe a very basic unity package with demo scene where we see all components added to see the actual flow.

    not sure if i am explaining myself correctly.

    thanks in advanced.
     
  9. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Ok here is an example for use with the Lobby. This uses the scripts provided in the initial release. It requires the use of Python for configuring your Game Server and it requires access to MySQL if your going to use the Authentication Plugin.

    The Data folder that is included you will need to make sure you add to your Compiled Game's data folder ("You Application"_data located in the root directory with your exe).

    http://dev.riseofheroesmmo.com/releases/LobbyExample.zip

    The Config Python file looks like this:

    from Configuration import *

    GameConfig.Config.serverName = "(Official) Prototype Server v1"
    GameConfig.Config.serverHost = "127.0.0.1"
    GameConfig.Config.mapName = "smallCityCenter"
    GameConfig.Config.serverPort = 12345
    GameConfig.Config.maxPlayers = 10
    GameConfig.Config.playerCount = 0

    I think this is pretty self explanatory.
     
  10. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    thanks,
    well, before you showed this lobbyExample project , i was playing with the files and started to attach components to the scene whatever made sense and i had several compiling errors and was confused, so hence the reason i was requesting a quick demo.

    but now with the demo , i think i was close to what i was doing, altho still getting the same compiling errors i had before.

    so here is my compiling error which i cant not seem to overcome or find why in the server project?
    I see the Data folder and config files there in the server but still getting the compile error.

    i loked on the hierarchy project and all public vars are being used, looked at the file where it references the Data folder for the config , but it is all there.

    so what am i missing?

    any advise on what is happening here?

    thanks
     
    Last edited: Jun 25, 2014
  11. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Do you have the Game Configuration Script attached?
     
  12. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    line 27 is this: GameConfiguration.config.ServerHost = GameConfig.Config.serverHost;

    Which tells me you don't have the GameConfiguration Script attached or you do have the Data Folder as part of your project. In order to get it working in your Editor you need to add the Data folder to your root Assets folder as well.
     
  13. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    yup it is attached,

    but:
    it is right infront of my nose, lol, I missed the reference link:
    is this hardcoded into the compiled .dll plugins?

    cant seem to find in scripts.



    but, nnow that you mentioned,

    the data folder is inside my project, do i need to do another copy of the Data folder outside the Asset folder?
     
  14. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    well, i just reconfirm and started from scratch, i am using the server example, the Data folder is inside my Assets folder of my project, also the game configuration file is attached and still getting the same error.
    :
    this reference might be hardcoded in the plugins .dlls files since i do not have a user in my windows setup named: ElectricCrowGaming so this should be your windows user where it might be hardcoded somewhere in the compiled .dlls files

    so i think this is why this error popus up
     
  15. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    No, no other copy of the Data folder is needed until you go to compile it. As long as your running it from with in the Editor you only need the one Data folder in your Assets folder.

    GameConfig.Config.dataPath = DataPath() + "/Data/";

    PythonScript.getInstance().setScriptDirectory(GameConfig.Config.dataPath + "Configs/");
    PythonScript.getInstance().runConfig("/config.py");

    public string DataPath()
    {
    string path = Application.dataPath;
    if (Application.platform == RuntimePlatform.OSXPlayer)
    {
    path += "/../../";
    }
    else if (Application.platform == RuntimePlatform.WindowsPlayer)
    {
    path += "/../";
    }

    return path;
    }

    There's how it's getting the Data path. Nothing is hard coded. It's getting Application's Data Path which should be pointing to your data path.
     
  16. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    so i dont understand why i am getting this path :
    C:/Users/ElectricCrowGaming

    in the error, it sure is not my windows user, something is hardcoded somewhere that references this windows user path
     
  17. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Try deleting all of the Meta files and see if it corrects the paths.
     
  18. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Try this Unity Package and see if it works.
     

    Attached Files:

  19. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    nope, still same error,

    i imported the ExampleGameServer.unitypackage into a new freshly created empty project, opened up the LobbyExample scene,

    in the scene hierarchy selected Lobby game object, and added my server info to the already attached Game Configuration component script ie:
    server name, server host, map, port max users etc.. hit play button from my editor and same nullreference error below:

    just dont understand whats up.

    maybe someone else can try it out in their system, to make sure it is not just my pc setup the issue.
     
  20. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Ok try commenting out the following lines and see if it works:

    GameConfiguration.config.ServerHost = GameConfig.Config.serverHost;
    GameConfiguration.config.ServerName = GameConfig.Config.serverName;
    GameConfiguration.config.MapName = GameConfig.Config.mapName;
    GameConfiguration.config.MaxPlayers = GameConfig.Config.maxPlayers;
    GameConfiguration.config.ServerPort = GameConfig.Config.serverPort;
    GameConfiguration.config.PlayerCount = GameConfig.Config.playerCount;
     
  21. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    thanks, commenting those lines fixed the issue,

    now the server starts ok, but...
    can not connect to it from the client project.

    the client project by default seems to try to connect to lobby standalone console server, right?

    I fired up the lobby standalone server console and did not see any signs of connection from client, using the client default host 127.0.0.1 and default port 12345 configuration.


    unless the lobby is setup for another port? no idea since no way to see.


    now, you mentioned the lobby standalone app is not necessary, but can you state how to connect client to server directly without the lobby?


    also, can servers intercommunicate between them to hand over players for a zoned mmo? or is this a one unity instance server per game setup?
    thanks again.
     
  22. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    The hardcoded port for the Lobby Console Server is 10002

    And the App ID for connecting to the Lobby Server must be UniVerseLobby
     
  23. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    got it, thanks.

    so now we getting somewhere :)

    so far got the following :
    fire up the lobby console and unity server instance using the above hardcoded ports and lobby server name, the lobby now registers the newly connecter server.
    then fire up the client using the above info for port and lobby server name and get successful connection and see the server ready in the gui layout.

    so now the next step, when i try to connect to the available server i get the login screen,

    so new step ahead, where shall i look for this new step of persistance db, is there any mysql dump or which file shoudl i look for to learn about it and get the authentication setup?



    sorry , too many question but i get exited with new server apps to test.

    thanks again for your support.
     
  24. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    lol No problem. It would be easier with Documentation I just haven't had time to work on it yet.

    For everything related to Database is in the AccountPlugin or AccountClient. I believe it's in the Client Script.

    You also have the following in the ServerManager script:
    MySQLManager.Instance.CreateConnection("localhost", "citylife", "root", "");

    Right there you have the DB host, DB name, then username and password.

    You can then get records from the Database with the following:

    MySqlDataReader reader = MySQLManager.Instance.Select("Select * FROM accounts WHERE username='" + user + "'");

    Put in the query you want to make there. I'll prob. look into making this more user friendly so that you don't need to know the actual syntax for the query but for now it works.
     
  25. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    ok,
    thanks, ill look into it.
     
  26. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Also the account table that I currently use with that just has an id which auto-increments, username field, and password field. Should be very easy to setup.
     
  27. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    got it,


    now, how can this setup be able to create an mmo setup,

    i mean, the server registers with the lobby server, client selects the server as some sort of room to join, but once in the room or server, how can this take it to the next level of handling several hundreds of players in the same world/ instance?

    since we know that it is not feasible for a unity instance server to handle more than 100 ccu without beaking/ freezing the server,, in other words how can this be made as an scalable seamless world?

    unless a zoned type with being each server a different zone to join, but again how does player jumps from one zone to another, do we need to logout then join another zone/server.

    still trying to understand your initial point of view to achieve this scalable setup for mmo like game.

    perhaps you can enlighten us on how you planning to achieve this setup.

    thanks again for your time and support
     
  28. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Haven't really gotten that far yet. I have Server Designs running through my head and down on paper but nothing implemented yet. We know it can handle Lobby based MMOs like LoL or MMOFPS like Combat Arms just haven't tested it to see how many players can be supported on a single Server.

    For true openworld MMOs first of all developers are free to setup their own System for going between Servers as for an officially supported way of doing that the design I'll be implementing to test out involves a Relay Server.

    You have your Unity based Game Simulation Servers. Each one handles a section of the world. You then have Relay Servers which connects to the different Simulation Servers. The Clients then connect to the Relay Servers and are associated to whatever Region they are currently in. The Relay Server then relays messages back and forth. If the Simulation Server goes down the Relay Server can transport the player back to their home or some predefined location and a message having some remark about why they were teleported. If you need to allow more players to connect you would just start up a new Relay Server which would be cheaper to run then a completely new Sim. Server. This would also prevent the players from gaining access to the IP addresses and ports of your actual Simulation Servers so it would be harder for them to try and externally hack the Sim. Servers.

    On paper it all looks good but it will be interesting to see how it actually works when it's implemented.
     
  29. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    James,

    what will be the license model for this framework? will it be free to use and distribute with any game as in the current binary state?

    or will you sell source as optional?

    or you just releasing binaries while in beta for non commercial games, then will sell a release at a price when this is completed?

    or the base framework will be open source and you will sell plugins as extras?

    i am intrigued on how you gonna market this product.

    thanks in advanced
     
  30. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    For testing during Alpha at the very least it will be free for development/non-commercial products. After Alpha or Beta there will be a paid commercial License. Not sure how much yet though. The license for Universe will also get you a UniStream Pro (My Asset Streaming Plugin) License which alone I'll be selling for $100.

    A Update on Universe... I'm in the process of integrating a new Messaging System which allows for distributed Messaging. Meaning you will now be able to create different Servers for your different Game Play Features and have them hosted on different Server Hardware. So Games will now scale by adding more Physical Servers.
     
  31. MrMassively

    MrMassively

    Joined:
    Sep 4, 2013
    Posts:
    115
    Awesome,

    now, when you mentioned "Games will now scale by adding more physical servers":

    can you elaborate more on this, like will it be the same lobby style where the servers connect to the lobby then players can select which server to choose from lobby? or you mean that the servers will be able to intercomunicate between eachothers to hand over players for a zoned mmo setup?
     
  32. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    Like I've said in the past The Lobby System has always been and always will be 100% optional. There's nothing forcing developers to use the Lobby. It's just one more added features that's available if you want to use it.

    With the new Distributed Messaging System you will be able to have Plugins/Servers that can intercommunicate with each other. So you could have a Communication Server which is the ONLY Server that players would connect to and then you can have an Account Server, Character Server, Area Simulation Server, ect. which the communication Server passes messages to and receives messages from for the Player.The Messaging System is fast:

    There are some test results for messaging in DotNetMQ:

    Messaging:

    • 10,000 messages in ~25 seconds as persistent (~400 messages/second).
    • 10,000 messages in ~3.5 seconds as non-persistent (~2,850 messages/second).
    Method Calls (in DotNetMQ Services)

    • 10,000 method calls in ~25 seconds as persistent (~400 calls/second).
    • 10,000 method calls in ~8.7 seconds as non-persistent (~1,150 calls/second).
    Test Platform: Intel Core 2 Duo 3,00 GHz CPU. 2 GB RAM PC. Messages/calls are made between two applications running on the same computer.

    Those are just some baseline stats from the testing they have done on it.

    And here are some of the features of DotNetMQ:

    DotNetMQ is an open source Message Broker that has several features:

    • Persistent or non-persistent messaging.
    • Guaranteed delivery of persistent messages even in a system crash.
    • Automatic and manual routing of messages in a custom machine graph.
    • Supports multiple databases (MS SQL Server, MySQL, SQLite, and memory-based storage for now).
    • Supports don’t store, direct send style messaging.
    • Supports Request/Reply style messaging.
    • Easy to use client library to communicate with the DotNetMQ Message Broker.
    • Built-in framework to easily construct RMI services upon message queues.
    • Supports delivering messages to ASP.NET Web Services.
    • GUI-based management and monitoring tool.
    • Easy to install, manage, and use.
    • Written entirely in C# (using .NET Framework 3.5).
     
  33. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
  34. Lostlogic

    Lostlogic

    Joined:
    Sep 6, 2009
    Posts:
    693
    Looks interesting. Would you see any problems running this in an Azure instance for the server? Also, can I tie into my own authentication system?

    Another question, how is threading handled? For example, does it spawn a thread per player? Will it shard across servers if too many people join?
     
  35. JamesPro

    JamesPro

    Joined:
    Mar 5, 2012
    Posts:
    509
    MassiveNet handles the whole Seamless Server to Server handoffs and everything. Basically your world is broken up into Zones, You run each zone on it's own Unity based Headless Server and the Server handles transferring the player between Servers seamlessly as the player moves around.

    Being a Headless Unity Instance the Server can be run on Mac, Windows, or Linux based Servers.