Search Unity

  1. Check out the Unite LA keynote for updates on the Visual Effect Editor, the FPS Sample, ECS, Unity for Film and more! Watch it now!
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  5. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Unity Multiplayer MLAPI - HLAPI Replacement

Discussion in 'Connected Games' started by TwoTen, Jan 5, 2018.

  1. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009


    Hello. I want to tell you about my project called the MLAPI. It's a high level network abstraction library similar to the HLAPI. It's designed to offer more features, greater flexibility and performance.

    It should have all the feature the HLAPI current has and more. Here are some feature highlights
    • Host support (Client hosts the server)
    • Object and player spawning [Wiki page]
    • Connection approval [Wiki page]
    • Message names
    • Replace the integer QOS with names. When you setup the networking you specify names that are associated with a channel. This makes it easier to manage. You can thus specify that a message should be sent on the "damage" channel which handles all damage related logic and is running on the AllCostDelivery channel.
    • ProtocolVersion to allow making different versions not talk to each other.
    • NetworkedBehaviours does not have to be on the root, it's simply just a class that implements the send methods etc.
    • Custom tickrate
    • Synced network time
    • Supports separate Unity projects crosstalking
    • Passthrough messages [Wiki page]
    • Scene Management [Wiki page]
    • Built in Lag compensation [Wiki page]
    • NetworkTransform replacement [Wiki page]
    • Targeted messages [Wiki page]
    • Port of NetworkedAnimator [Wiki page]
    • Networked NavMeshAgent [Wiki page]
    • Networked Object Pooling [Wiki page]
    • Synced Vars [Wiki page]
    • Targeted Synced Vars [Wiki page]
    • Encryption [Wiki page]
    • Super efficient BitWriter & BitReader [Wiki page]
    • Command & Rpc system like HLAPI to allow for quick transfer from HLAPI to MLAPI [Wiki page]
    • Observer system similar to the HLAPI [Wiki page]
    • Custom UDP transport support [Wiki page]

    Github repo
    Binaries
    Wiki
    API Reference

    If you encounter any issues. Please open an issue on Github

    Thanks, TwoTen
     
    Last edited: Apr 27, 2018
    Ellernate, S4UC1SS0N, moco2k and 2 others like this.
  2. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    For those who care, I have created a Wiki on Github. I am starting to get to a point where it's usable. I will be creating example projects and more wiki pages as the project develops and hopefully it will be ready for use fairly soon. Currently it shares many features with the HLAPI. Such as object spawning, object authority. But it removes the many limitations the HLAPI has (and hopefully their bugs). The fancy features such as an easy way to get variables to sync (syncvars) are still not done. Currently you would have to create it similar to a Cmd - RPC setup. But it is planned.

    The wiki can be found here
     
  3. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    I have created another repo for Examples. So far the example is just a unity scene with multiplayer where each player controls a cube and the positions sync across the network. It's simply to show how to write network code with the library and is just very quick and dirty written code.
    And btw, the Host feature is not yet working. So use one server and at least one client for now

    https://github.com/TwoTenPvP/MLAPI-Examples
     
  4. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    466
    Hey TwoTen.

    Glad to see that you have started working on your project.
    But I would like to ask you something. Considering the following code:

    https://github.com/TwoTenPvP/MLAPI-Examples/blob/master/Assets/SyncPosition.cs

    Code (CSharp):
    1.  private void Update()
    2.     {
    3.         if (!isLocalPlayer)
    4.             return;
    5.         if(Time.time - lastSentTime > (1f / PosUpdatesPerSecond))
    6.         {
    7.             using(MemoryStream stream = new MemoryStream())
    8.             {
    9.                 using (BinaryWriter writer = new BinaryWriter(stream))
    10.                 {
    11.                     writer.Write(transform.position.x);
    12.                     writer.Write(transform.position.y);
    13.                     writer.Write(transform.position.z);
    14.                 }
    15.                 SendToServer("PositionUpdate", "PositionUpdates", stream.ToArray());
    16.             }
    17.             lastSentTime = Time.time;
    18.         }
    19.         transform.Translate(new Vector3(Input.GetAxis("Horizontal") * Time.deltaTime, Input.GetAxis("Vertical") * Time.deltaTime, 0));
    20.     }
    The code above won't send "PosUpdatesPerSecond"n packets to the server at the fixed tickrate of "PosUpdatesPerSecond" if the Client's FrameRate would be lower than the "PosUpdatesPerSecond" value.

    Let's assume that your tickrate is 64, which means that the Client sends PositionUpdate 64 times per second. This particular case would work only if the Client's FPS is 64 or higher, if you will try to change it to the lower value on the client by "Application.targetFrameRate = 20" - you will see jittering on the server because the client would send only 20 packets per second instead of 64.

    So what I would suggest, is to somehow process that SendToServer in a new thread, because your main thread would be always busy and limited to the GPU resources.

    But in general, I like the idea of MLAPI, sounds cool. Keep it up ;)
    By the way, what happened to your blog?
     
    Last edited: Jan 7, 2018
  5. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Hello.
    I am currently on vacation in japan and this is a sideproject I started (though I am planning to continue it). As for the example project. You are correct. The example is just to show the basics of how to get started with message sending.

    That out the way, you are corect. You have a good point though. I could add a thread system. So that once the send function is called, we send the byte array off to the thread to be processed. And its only returned to the main thread when it’s ready to be passed to the LLAPI. Same for receive but in reverse.

    Not sure if it’s worth it though. The only gain would be that the binary serialization would happend on a separate thread. The HLAPI works on one thread aswell similar to this. Dont think you would get much of a benefit since game logic has to happend on the main thread and the NetworkTransport can also just be interacted with from that thread



    As for the blog. Moved the articles to twotenpvp.github.io
     
    Last edited: Jan 7, 2018
    wobes likes this.
  6. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    466
    Well, probably you are right. It might not be worth it.
     
    Last edited: Jan 7, 2018
  7. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    https://github.com/TwoTenPvP/MLAPI/...oBehaviours/Prototyping/NetworkedTransform.cs
    Here is a better NetworkTransform I wrote that shows some new features I added to the MLAPI. It currently just syncs transforms and does interpolation based on the sendrate. But it illustrates how to use the MLAPI fairly well.

    Mainly it shows the new Target variant of the send methods. Using the Target version means that only listeners registered on the same NetworkedObject as it's being sent from will be Invoked. Worth noting is that if multiple listeners are registered to the same NetworkedObject they will all be invoked. Even if they are on different NetworkedBehaviours. This is in order to not have to sync the NetworkedBehaviours themselves.

    If anyone has any questions about the MLAPI. Feel free to ask. Currently, I feel that it's at a point where it can be a better alternative to the HLAPI. I just need to consider Variable sync and scene management.


    Also, if you encounter any problems with the library. Please open an Issue on Github.
     
  8. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    I have added a big new feature. MessagePassthrough. It allows messages to be sent from Client to Client via the Server. This is similar to a Command -> RPC Setup but requires a single call. Security features are implemented. You have to manually mark MessageTypes to be used for Passthrough and the Server will also inject the Source ClientId when passing it to the target. I have added a Wiki page for it.
     
  9. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Lot's of new features have been added. Complete feature list can be found on the GitHub readme. It should now pretty much be production ready and I'm happy to hear from anyone using it. Any issues and/or missing features you encouter, please report them on GitHub and they will be looked into.

    The components have replacements now (NetworkedTransform & NetworkedAnimator).
     
  10. MrG

    MrG

    Joined:
    Oct 6, 2012
    Posts:
    224
    Have you considered adding a NetworkedNavMeshAgent component to sync velocity and other NavMeshAgent data from server to clients?
     
    Last edited: Mar 27, 2018
  11. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    I’ll make sure to get that done!
     
  12. MrG

    MrG

    Joined:
    Oct 6, 2012
    Posts:
    224
    My thought was that it would be an alternate to NetworkedTransform, and include transform updates at a much lower frequency, allowing the clients to largely solve and run the agents independently. The big picture would be that the clients would have code that would drive other things off the NavMeshAgent that the server wouldn't have because such would be non-critical to game play or precision across all clients, e.g. the destination is more important than the journey. If properties were sent as reliable only when changed, the data traffic would be a lot less for such entities. One scenario might be to have the server raise or lower the transform update frequency based on distance from client character for such agents. Come to think of it that might be a nice optimization for NetworkedTransform as well when proximity checking is active.
     
  13. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    It's been added.
    https://github.com/TwoTenPvP/MLAPI/...haviours/Prototyping/NetworkedNavMeshAgent.cs

    As for the Distance & Proximity thing, i'll get that sorted. For future feature request. Please use the Issue feature on GitHub. Makes it super neat for me to track. That's where pretty much all feature requests have been put so far and almost all have been implemented.
     
    Last edited: Mar 28, 2018
    MrG likes this.
  14. Artaani

    Artaani

    Joined:
    Aug 5, 2012
    Posts:
    376
    Hello. I can't figure out how to launch it.
    I downloaded MLAPI-Examples-master.zip
    But there is missing component on GameObject.
     
  15. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Right, I think that's due to the import order of the components. Idk why Unity does that tbh.

    NetworkManager has a NetworkingManager component, the player object just has NetworkedObject, NetworkedTransform and NetworkedAnimator that is from the library.
     
    Last edited: Mar 30, 2018
    Artaani likes this.
  16. MrG

    MrG

    Joined:
    Oct 6, 2012
    Posts:
    224
    @TwoTen You might want to update your readme and wiki that you support Websockets / WebGL and document it.
     
  17. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    The documentation, API reference and wiki are documenting the last release. Thus they are a bit outdated. I am currently working on version 1.0 thus there has been no release for a bit. We are getting a brand new serializer in that uses bits instead of bytes to save as much space as possible & lots of new features. A new Editor script that lets you handle your versions and update the MLAPI in the engine.

    Here is a peek
    https://i.imgur.com/RuW71Z4.png
     
  18. MrG

    MrG

    Joined:
    Oct 6, 2012
    Posts:
    224
  19. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
  20. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    I've just released version 1.0. It offers lot's of new features and fixes and is now ready for use (IMO). It has a nice way to manage your MLAPI version to prevent your project from breaking while at the same time letting you easily upgrade. Please do excuse my horrible Editor scripting skills.

     
    Last edited: Apr 17, 2018
    wobes likes this.
  21. TriangularCube

    TriangularCube

    Joined:
    Apr 6, 2016
    Posts:
    15
    Am I missing something? I literally replicated the network config on the example project, but the example is able to connect from a client, but my project won't connect.
     
  22. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    You're not giving me much to work with haha. You need to describe your issue a bit more in detail. If you are getting started, I suggest reading the Wiki. There are articles on how to get started.
     
    Last edited: Apr 18, 2018
  23. TriangularCube

    TriangularCube

    Joined:
    Apr 6, 2016
    Posts:
    15
    That's fair, I'm just a little frustrated is all. Been trying to diagnose this for 3 hours.

    So I loaded up the example project, built it, and had no trouble connecting to a host running in the Editor. So far so good.

    So I started a new project. I imported MLAPI, made a manager GO, added the NetworkingManager component, and copied the NetManagerHud script from the example project. Added a prefab to the Manager's list, and checked the box for player object. Built the project, and proceeded to test. No luck, whichever instance runs host works fine, but the other instance will not connect to the server. Tried adding ConnectionApproval to see if any clients are even connecting, also got nothing.

    Changed channels, message types, server transports, and ports for both projects. Same result. Used Netcat to test if any data is being sent from the client, and it is.

    At this point I'm not even sure what options are even left to try, the Wiki wasn't of great help.
     
  24. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Seems very odd. I mean, if the MLAPI is not even getting it's connection approval things invoked. Then there is two things that could go wrong.
    1. LLAPI issue, this is unlikley.
    2. Configuration missmatch. If certain parts of the NetworkConfigs doesn't match. They won't talk to each other. But it should give you a nice message in console. If you send me the project. I will happily have a look. Sorry to hear your frustration.

    EDIT:
    One thing that could go wrong, is if you have NO Transports. See the networkConfig. Then it will not listen anywhere.
     
  25. TriangularCube

    TriangularCube

    Joined:
    Apr 6, 2016
    Posts:
    15
    No, definitely got some transports.

    I'm starting to think it's my PC, but then I've been working on 3 different PCs now and it's the same.

    I'll upload my project, and hopefully you can figure it out better than I. Thanks for taking a look by the way, I apologize for for sounding like a douche earlier.
     

    Attached Files:

  26. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Issue is the same on my machine. Let me look into it.
     
    TriangularCube likes this.
  27. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Located the issue. Try running your project in development mode and you will get the console spammed. Essentially, the cryptography classes in .NET are fantastic. In Mono, not so much. Pretty much none are implemented. We had to write our own. It's VERY time consuming. That's why you don't see proper key exchanges etc in other C# UDP libs for Unity or the HLAPI. But we did not have the time to write our own BigInt implementation. So we used IntX. We are working on our own. But for now we ship with IntX. This is an IntX issue. As far as I know, IntX is not open source, and I am not looking into the cause of the issue. I rather spend my time on working on our own implementation.

    HOWEVER, I know a solution. Change the projects runtime from .NET 3.5 to the new 4.6 and the issue will no longer persist. Sorry for troubling you. This should have been documented, I just did not know.
     
  28. TriangularCube

    TriangularCube

    Joined:
    Apr 6, 2016
    Posts:
    15
    Thanks so much. I thought I turned encryption off at some point to test, but I cannot remember off the top of my head now.
    (EDIT: Not sure if turning encryption off actually affects this issue, so nevermind!)

    In any case, glad to see it working!
     
    Last edited: Apr 18, 2018
  29. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    It happends even with key exchange turned off. It's part of a static field initializer.
     
    TriangularCube likes this.
  30. TriangularCube

    TriangularCube

    Joined:
    Apr 6, 2016
    Posts:
    15
    Ah I see. Thanks again!
     
  31. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    No problem! I did a bit more digging, and found that if you want to use the .NET 3.5 version, you can as long as you change from .NET 2.0 subset to .NET 2.0.
     
  32. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Here is a video showing the install process. You can install the MLAPI in 15 seconds.
     
  33. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    568
    With HLAPI Pro I had to replace files within the actual editor folder and it affected all projects, is this contained within only the current project you are using and my others will be fine?


    ------------ Edit
    Nevermind, I see, its all good. I am going to give it a whirl. Any additional features that might benefit a RPG in some way would be appreciated. : P I dunno what, but... any. I am definitely interested in the observer system.

    I'd venture to guess it isn'tI 2018.1 ready? I just tried to build in 2017.4 using net 4.6 and got this. ArgumentException: The Assembly LiteNetLib is referenced by MLAPI ('Assets/MLAPI/Lib/MLAPI.dll'). But the dll is not allowed to be included or could not be found.

    Edit again - - Errors are gone after trying 5-6 times but it wont build. The editor log says it did, but it dings at me and makes no executable and throws no error. I tried both 4.6 and 3.5(non-sub and sub). The closest I can get to an error in the log is this.

    Fallback handler could not load library C:/Program Files/Unity/Hub/Editor/2017.4.1f1/Editor/Data/Mono/lib/data-0000000039F462D0.dll

    I tried to go back a version as well and still no build though.
     
    Last edited: Apr 27, 2018
  34. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Oh, that's intresting. I thought I could get away with referencing LiteNetLib but not including the library. I will get that sorted.

    Edit:
    This has been resolved in the v1.1.1 version. I removed LiteNetLib from the DefaultTransports, if you want to use it. Just download the transport from the "SampleTransports" folder.
     
    Last edited: Apr 27, 2018
  35. Twist01

    Twist01

    Joined:
    Jan 21, 2017
    Posts:
    46
    Hello TwoTen I was wondering if I can make the player host the sever with no cost because I currently have a budget of $0.
     
  36. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    Yes, that is supported
     
    Twist01 likes this.
  37. PixelMind

    PixelMind

    Joined:
    Aug 16, 2013
    Posts:
    94
    Hey, Just a quick thank you for this project! This could be recommended for anyone who's struggling to find a replacement for uNet or just wants a nice and clean networking solution.
    I had a couple of very specific needs for my own game so I ended up taking a partial fork and building upon it. Your bit writer / reader system could be a project of its own :)
    Thank you!
     
  38. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    That's what many people do! Create a fork, add their own custom magic if needed (a lot of the time it's not). Then they submit a PR and the stock library becomes better!

    Glad you're enjoying it!
     
    PixelMind likes this.
  39. imgodot

    imgodot

    Joined:
    Nov 29, 2013
    Posts:
    166
    MLAPI intrigues me. I like the cut of its jib.

    I am thinking about using MLAPI for a rewrite of a turn-based game that needs some heavy updating.

    A few questions (just to clarify I am reading the wiki correctly):
    1) MLAPI allows me to synchronize an array of structs across the network, correct?
    2) Like the HLAPI, it does not allow synchronizing an array of classes, correct?
    3) When synchronizing an array of structs, and one property in one struct in the array is modified, does the entire array re-synced, or only the one struct in the array?

    Thanks very much.
    -- Paul
     
  40. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    1. Short answer: yes, Long answer: you need to create a wrapper class if you don’t want to sync primitives
    2. Wrong
    3. You need to provide your own delta. So it does not have to resync evertythibg
     
    imgodot likes this.
  41. imgodot

    imgodot

    Joined:
    Nov 29, 2013
    Posts:
    166
    Thanks for the reply.
    I actually have a list of classes in my current code that gets synced across the network.

    Q: MLAPI does not allow a LIST of classes to be synced, correct?

    Q: Do you have a sample anywhere showing syncing of a LIST or an ARRAY of classes?

    Thanks.
    -- Paul
     
  42. imgodot

    imgodot

    Joined:
    Nov 29, 2013
    Posts:
    166
    DOH!
    I just found the wiki section about NetworkedList, therefore, my only remaining question is whether there is a sample running around using NetworkedList.
     
  43. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,009
    1. Wrong
    2. No sample needed really. Just create a NetworkedList field and you are good to go. As long as it's located in a NetworkedBehaviour it will automatically get synced.