Search Unity

  1. We've closed the job boards. If you're looking for work, or looking to hire check out Unity Connect. You can see more information here.
    Dismiss Notice
  2. We're running great holiday deals on subscriptions, swag and Asset Store packages! Take a peek at this blog for more information!
    Dismiss Notice
  3. Check out our Unite Austin 2017 YouTube playlist to catch up on what you missed. More videos coming soon.
    Dismiss Notice
  4. Unity 2017.2 is now released.
    Dismiss Notice
  5. The Unity Gear Store is here to help you look great at your next meetup, user group or conference. With all new Unity apparel, stickers and more!
    Dismiss Notice
  6. Introducing the Unity Essentials Packs! Find out more.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice
  8. Unity 2017.3 beta is now available for download.
    Dismiss Notice

Assets DarkRift Networking 2: The next level of networking

Discussion in 'Works In Progress' started by Jamster, Jan 20, 2016.

  1. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Possibly, I agree with the consistency aspect...
     
    Lisk likes this.
  2. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    DarkRift 2 Beta has been submitted to the asset store!
    Hooray!
     
    BouncedPhysical likes this.
  3. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    89
    Hi, Jamster. I only used to run Unity instance as my server. I've read that DarkRift can also run inside Unity. But it will have better performance if it runs as standalone. So, if I want to use physics in my game. Is it only possible to run DarkRift inside Unity? Is there any way to run DarkRift as standalone and can still enable physics in my game?

    My past game are just normal multiplayer and they are not real time so Unity instance is good enough. I'm currently developing MMO game for my client. I've read many people in this forum said that running Unity instance as server is not good for MMO. I don't know the truth behind such statement. But it makes you wary if many people said the same thing. I would like to ask your suggestion. Do you think that DarkRift inside Unity is capable of running big MMORPG like Tree of Savior?
     
    Last edited: Jul 17, 2017
    Lisk likes this.
  4. antifuzz

    antifuzz

    Joined:
    Feb 22, 2013
    Posts:
    57
    Hi @Jamster Is it possible to get an advance copy of the DarkRift 2 beta? I'm trialling DarkRift for use on a potential project and if v2 is nearly out it makes sense to focus on that if I can..?
     
  5. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Firstly, I'm going to pull DarkRift from the store again temporarily since I packaged a few things wrong and a couple of testers found a load of new, major bugs. An early release of the free version is currently available on the discord (pinned to one of the channels) so if you want to join the open beta please go there!

    I'll hopefully put future builds up here as well but for now there a (buggy) build available there!

    @jjobby Running things in Unity allows you to use things like physics really easily but there are performance limitations and it gets much trickier with large worlds etc. so it can make it very awkward for servers, particularly MMOs. On the other hand, a standalone server doesn't have world size limits (not that you'd want to have your whole world in one server anyway!) and would be better for performance but doesn't have built in physics etc.. So it's swings and roundabouts really but you can do either in DarkRift.
     
    Lisk likes this.
  6. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    I've just realised I've been posting weekly updates on the Discord but not on here! I'll try and start posting more on here in future so that more people see it!

    Recently I've been working on a new release for DarkRift 2; I've rewritten the log system to allow you to write custom log writers as plugins and import them into DarkRift, this means you can get DarkRift to write out to databases etc. as well if you need and it should offer more flexibility than the previous system.

    On top of that I've been doing a similar thing for DarkRift's persistence layer. Previously, I was using SQLite but for some reason it would only work if DarkRift was forced to use 32 bit mode hence not allowing it to run as well on 64 bit servers (which is most!). I've now also made this a plugin-based system so that you can configure exactly where DarkRift should store it's persistent data and this should make it significantly easier to add server-server communication/proxies later. I've included a basic XML based storage plugin that should work out the box but it's possible to make DarkRift store in databases, another format, etc.

    Of course there was also a DarkRift 1 release that fixes a bug with lagging players and numerous bugfixes on DarkRift 2 :)

    Jamie
     
    Lisk likes this.
  7. Cushmily

    Cushmily

    Joined:
    Mar 25, 2015
    Posts:
    3
    Thanks man, I LOVE DarkRift Networking very much! Great to hear DarkRift 2 is under development.
     
    Jamster likes this.
  8. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    DarkRift2 Beta r2 and r2p1 released!
    DarkRift 2 Beta r2
    • Lots of example code etc. now on GitHub
    • SQLite dependency has been removed and replaced with DataHandler plugins
    • LogWriters are now plugins
    • Lots of bug fixes
    • File path for FileLogWriter now more customisable
    • Configuration for log writers improved
    • (See updates channel for more details)
    DarkRift 2 Beta r2p1
    • Fixed Unity Server and updated editor to latest DarkRift
    • ServerSpawnData now takes lists rather than arrays - Fixed XmlDataHandler bug
    Don't forget this is still beta and there'll be plenty of issues! Please keep reporting things on the issue tracker :)
     

    Attached Files:

    Cushmily likes this.
  9. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    (Apologies for cross-posting this late!)

    2 releases have been made this week for DarkRift 2! The majority of my time, thus, has been spent trying to package and test those releases beforehand so not a huge amount of functionality has been added. I have however bene able to address some bugs and documentation issues since the release and have added a Disconnected event to the client side (an oversight when writing it!). I've also taken some time to move a lot of my private repos to GitHub for things like the Agar tutorial completed code and the block demo plugin code. The link is available in the #links channel and it would be awesome if people wanted to help out with maintaining them!
     
  10. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    This week I've had a lot of time on my hands so a lot's been done! I started looking into the UDP issues but found a number of other bugs in the process so I've had to fix those first; I've added a Disconnected event in the client (not sure why it wasn't there) and fixed bugs where UnityClient was passing events the wrong object as a sender.

    I've also added performance enhancements to the internal plugins by unsubscribing them from events when they're not in use and I've redesigned the sniffer interface to be more clear and include new options:
    Code (csharp):
    1. sniffer add -t=100 -s=89
    2. sniffer remove -a
    3. sniffer clear
    4. sniffer list
    Lastly, a lot of people (mainly those using PlayFab) wanted to be able to pass command line arguments to set parameters in the server so now you can pass the server variables to use when loading the configuration file. For example, to create a server on port 80 you could start it using
    Code (csharp):
    1. DarkRift.Server.Console.exe -differentPort=80
    and define your configuration file to use that variable:
    Code (csharp):
    1. <server ... port="$(differentPort)" ... />
    On top of that you can now pass in the name of the configuration file you want to load:
    Code (csharp):
    1. DarkRift.Server.Console.exe MyConfig.xml -differentPort=80
     
  11. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Hey all!

    This week I've mainly been on holiday so not much has been done! However, this evening I was able to write some build scripts to make my life easier when building the packages.

    I've previously been copying the files across by hand (hence the mess ups on some builds!) but for DarkRift 2 I've added an editor script so I can simply hit a menu item and it'll import the correct build into the project. It's fairly crude at the moment but it's a definite improvement over spending half an hour per package each time as I was for DarkRift 1!

    I also had the opportunity to rewrite a post build script that organised the DarkRift files into the Lib folder as you see it. Instead of being a quick CMD command that frequently broke it now uses MSBuild to do it in a bit more of a stable way (I also really wanted to learn MSBuild!)

    Jamie
     
  12. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    DarkRift 2 Beta r3
    Bugs:
    • Added disconnection event to UnityClient
    • Disconnection events on client now invoked when the client causes the disconnection
    • UnityClient now pases itself as event sender rather than the backing DarkRiftClient
    Improvements:
    • Internal plugins now unsubscribe from events when not in use
    • Redesigned interface for the sniffer plugin
    • Added command line arguments and configuration variables in the XML configuration
    • Added more information to disconnection events
     

    Attached Files:

    Kalle801 and AdmiralThrawn like this.
  13. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Not much has been done the past week but I did get a chance to do some optimisation and greatly simplified one of the layers in DarkRift's internal protocol stack, this should mean one less object allocated for each message received and so shows a bit of a performance increase just using the demos!

    Hopefully next week more will get done!
     
    Cushmily and AdmiralThrawn like this.
  14. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    This week's work has been mainly working out the UDP issues trying to make sure they work across NAT routers. It turns out the way I had implemented UDP isn't very NAT compatible so I'm doing a bit of a rebuild to make it operate more similarly to how Hazel and TCP work, using one socket rather than one per connection.

    I've learnt that my networking knowledge still has a lot of holes to fill! :)
     
  15. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Unfortunately (although fortunately for me!), last Sunday I built myself a gaming PC and have hence done nothing on DarkRift this week!

    I wish I could update you a bit but all I can say is that NieR:Automata is a pretty good game :p
     
    jjobby and zyzyx like this.
  16. algut20

    algut20

    Joined:
    Oct 5, 2016
    Posts:
    1
    Hello Jamster,

    DarkRift 2 has caught my attention. I do have a question. On the websites it says that DarkRift does not use Area of interest, so I am wondering how DarkRift connected users handle unnecessary requests from users that are not in proximity? This would be important for an mmo with a large world.
     
  17. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Hey,

    DarkRift 1 doesn't have any AoI built into it and doesn't come with any packages like so it requires that users write it themselves in their plugins. DarkRift 2 now has an AoI helper which you can use in your plugins to implement this neatly but it's still optional so it won't interfere with games that don't require it.

    For large open world games you would want to split users across multiple servers which still isn't supported in DR2 but will hopefully be coming soon after release! Nevertheless, you can certainly build similar systems with DarkRift and people have done in the past.

    Jamie
     
  18. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    A little more productive this week! The UDP issues have all been fixed and the clean up done, there's possibly a few vulnerabilities in there so it might be a fun exercise for you guys to try and break the server with corrupt connections etc.! Go for it!

    In this update there actually been quite a lot despite it not really showing from the updates. Most majorly, the UDP is now all fixed but it's also worth pointing out the new message command...

    The message command is now mostly there and allows you to send out message from the server as if you were a plugin. This has already been super useful in sorting the UDP issues as I could really easily send a message to check if Unity was receiving it or if the router had not been punched through correctly. To use it you simply write:
    Code (csharp):
    1. message <id> <reliable|unreliable|r|u> (-t=<tag>) (-s=<subject) <data...>
    Or as an example that's easier to follow:
    Code (csharp):
    1. message 2 reliable -t=0 -s=9 00 00 00 00
    There's perhaps a good opportunity here to also add a mocking command to test plugin functionality and perhaps even aid integration testing a little!

    Of course there the usual bug fixes, some of which were rather awkward (thanks Unity!), and the usual optimizations in the message pipeline trying to remove large copy operations and object allocations. Hope you enjoy! Don't forget to check out the new release!

    ----

    DarkRift 2 Beta r4
    • Fixed missing documentation on some types in both DarkRift and the Unity package
    • Fixed bug causing selected plugin types to unselect on play in Unity
    • Added profiling to DarkRift client events in Unity
    • Optimized the message frame layer to reduce the allocations per message
    • Optimized out a memory copy in reliable sends
    • Fixed bug stopping the IP to connect to being changed in Unity
    • Added error details to problems when loading the configuration file
    • Moved XmlConfigurationException to DarkRift.Server
    • Converted IP addresses to IPEndPoint types so the port is accessible
    • Partially added message command (bugs remain, no documentation still)
    • Fixed UDP hole punching issues
    • Added new images (more to come)
    DarkRift 2 Beta r4p1
    • Removed profiling to fix build issues in Unity 5.5+
    DarkRift 2 Beta r4p1
    • Added missing server archive
     

    Attached Files:

    Last edited: Oct 9, 2017
  19. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Just updated Beta r4 to include the missing server archive!
     
  20. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Very late but this week I was able to finish off a few things. Firstly, the message command received a few bug fixes so that it no longer adds a 4 byte length to the start of the data (oops...) and secondly the mock command has also been added. The mock command is essentially exactly the same as the message command except instead of sending the message to the client it injects it into the server as if the client sent it. These 2 combined should make testing and debugging a little easier! There's a few little bits to do like abstracting out the code into one class but it's most certainly complete in functionality now.

    I have also finished off the build pipeline. Previously the build pipeline had a gap in the middle between building the server project/arranging the built assemblies into the structure you see (e.g. with the Lib folder etc.) and importing it into the Unity project on command. Basically I couldn't find a neat way to package up the build into the zip archive automatically so I had to do this manually each time before importing into Unity (else you'd get the wrong server or worse, none at all). Now I'm using the MSBuild Community Tasks library to zip it all up every time it's built :)

    It would be nice to get some automated deployment to the asset store/website going but I'm fairly sure that's impossible :(
     
  21. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    This week (all 4 days since the last update) I've been able to do some optimizing! I've significantly reduced the number of allocations per message that DarkRift 2 has both server side and client side by recycling objects. This is still a work in progress but it should bolster performance and reduce the frequency of garbage collections in your servers. There'll be some breaking changes on release as you'll have to use
    Code (csharp):
    1. DarkRiftWriter.Create();
    etc. rather than just
    Code (csharp):
    1. new DarkRiftWriter()
    but in beta I'm just marking them as Obsolete.

    There may be a few issues with maintaining references to old items if they get recycled but I'll make sure there's plenty of warnings in the docs when they get updated!
     
  22. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Since losing the recycling code temporarily (left it on my old hard drive...) I've been working on some other features for the next release. I've removed the analytics database table that was kept behind the scenes (check the data folder!) since this wasn't suitable for production servers as it would become quite large in file size and wasn't actually being used for anything anyway. This has been partially replaced by a clients table which only tracks the clients that are currently connected so in future you can locate clients across servers. This change also makes the ID allocation unique across all servers sharing that database but will change how IDs are allocated (at least temporarily).

    I've also been doing some work on the bad word filter in the pro version, renaming and making it less intensive on start up (it no longer downloads the latest bad word list each time).

    On top of this I've also remove the dependency on the Unity Standard assets which no longer work in Unity 2017+ and have been working on an updated website for release!

    Hopefully there will be a new release soonish!
     
  23. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    DarkRift 2 Beta r5
    - Added message command
    - Added mock command
    - Removed analytics data persistence
    - Renamed Profanity Filter to Bad Word Filter
    - Added local caching for bad word list
    - Removed dependency on Unity Standard assets
    - Fixed compilation error in Unity 2017+
    - Add access to data manager for pro users
    - Internal build process improvements

    This release includes the Pro version!
     

    Attached Files:

  24. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    This past week not much has happened (and even less will next week!) but I have been able to remove the demo scenes dependency on the Unity standard asset pack since this had compile errors in Unity 2017+, it still uses the same script but it's massively simplified so that it is more self contained.

    I was also able to move the Bad Word filter (previously Profanity filter) into a plugin so that it can have it's own commands and can run code of its own accord. This allows it to now download and cache the bad word list so that it doesn't run each time you create the filter! I'm hoping to do similar for the Room system, AoI and any future helpers but they will be slightly more complicated to do.

    I've also made a release with all the latest updates, checkout r5!
     
  25. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Recycling has begun once again! Rather than try to salvage the code off my old hard drive I've decided to restart the recycling code and I'm pretty much back to where I was but I think I've done it better this time.

    Firstly, why recycle? Every time you stop referencing an object in .NET you create garbage and, once you amass enough garbage, the garbage collector will eventually stop your code and clean up. This is obviously bad in a server because it can introduce lag and other problems when it triggers, also allocating objects is very slow. To get around this DarkRift will recycle objects as much as it can! This, however, is going to introduce a few breaking changes. Most of the changes are simply deprecations and so you'll just get warnings until you change your code (though the code will be removed before release) but there are 1 or 2 breaking changes.

    DarkRiftWriters, DarkRiftReaders and messages all now implement IDisposable so when you are done with them you can return them to the object pool as you would normally with IDisposable
    Code (csharp):
    1. using (DarkRiftWriter writer = DarkRiftWriter.Create())
    2. {
    3.    //Stuff
    4. }
    Also note that you use Create() now so that it can get a recycled instance if there's one available.

    Messages are no longer shared between event handlers because that makes recycling the message really tricky, you can now use e.GetMessage() which will return your function its own copy - put it in a using statement and it will get recycled once you're finished with it. This does however mean that DistributeTo has gone because changes to the message will no longer persist between event handlers.

    Recycling will be the main change for the next version, there's still more to do in future but most of the public facing changes are now done, I need to cache arrays (a nightmare because they're all different sizes!) and I still need to find a way of caching event arg objects (a nightmare because of the dispatching for Unity!)