DarkRift Networking 2

Hey all!

I apologise for being a bit quiet recently, I've been overly busy and had very little time to myself, even less to work on DarkRift That said, things are easing up again and I'll be back developing it more again. The last few weeks have seen a lot of new bugs and features added to the board so they'll be first priority!

On Monday I did get to do a bit on the room system though! I had an idea to solve a problem with the matchmaking and so did a little rewriting. Previously it was trying to assign players to rooms (great if the rooms are already partially full) but I'm now looking to assign players to other players to form groups. It was quite tricky working out the point to instantiate a new room as the players were already being added to them but with this it should be much simpler as it automatically happens when a group of a set number is made. It should also allow predefined groups (e.g. partially full rooms or squads) to be easily added!

Anyway, back to bug fixing this week!

 

Hey Chad,

Thanks for pointing that out, the docs do seem to have some old docs for the unreleased room system!

The discord now has a fair number of users now, just looking at the side panel and there are 83 online (don't know about total registered) with maybe 20 who show up regularly? It's becoming a bit more self sustaining which I really do like, but I make sure to try and fact check a lot of the answers so that no misinformation gets given to people!

Hazel was designed to complement DarkRift but it was decide they should go their separate ways. I tend to focus on DR most of the time now and don't get much time at all to work on Hazel which is a shame. At some point I'd like to release an abstraction for DR that allows you to write your own transport layers for it as plugins, this would make it much easier to do things like WebGL support, reliable UDP through Hazel , or even random stuff like HTTP based communication if you wanted!

Hope that helps, definitely join the discord and see for yourself though!
Jamie

 

and
how the 'maxStrikes' means?Now my ServerConfig the maxStrikes is 3.

 

The max strikes is the number of strikes a player can incur before being kicked from the server. A strike occurs when they send malformed data, do something suspicious, or when your plugins calls the Client.Strike method for example

 

Question about the room system. Will this be like one server multiple rooms where people do have different data transfer like room1 players listen to room1 and not other rooms?
Second is this already possible with DK2, I mean if I go for my own room system solution?

 

The main component of the DR room system is the matchmaker which will be able to match users together on a single server or even across multiple servers (on release with some work, with inter-server communication a lot more easily). The other components are more undecided at the moment as I realised some architectural choices weren't compatible with the upcoming inter-server communication, so it's hard to say exactly how they'll operate I'm afraid.

It's definitely already possible with DarkRift and people have already done so!

 

Unfortunately progress is still slow at the moment. The only noteworthy thing to say in this update is that I've been working with Deusald to get ConnectInBackground working more reliably, it seems partially related to using a separate thread to run the connect within but we've also seen different results with .NET3.5 and .NET4.6 so it may be a mono bug that's causing it!

I'd like to do a patch ASAP with some other bug fixes that were done a couple of weeks ago but my time is very limited over this next week

 

DarkRift 2.1.1 has been submitted to the asset store
This should fix 2 smaller bugs (sniffer not accepting tags larger than 255 and ReadRawInto reading incorrect data), add a generic overload for writing IDarkRiftSerializable into writers, and contains a potential fix for the ConenctInBackground issues. These fixes have been around for a while and so I just want to get them out and into your hands rather than wait any longer!

 

DarkRift doesn't seem to work on UWP. Given the growing market for Windows Mixed Reality, perhaps it is an interesting to make the client UWP compatible. Any chance of this?

 

DarkRift 2.1.1 is now available on the asset store!

In other news this week, someone asked if it would be possible to get DarkRift working with .NET Core and .NET Standard and I had to tell them that no, it won't work with .NET Core/Standard until Unity 2018 is the minimum supported version because Unity has been very slow off to keep up. Well I got thinking and after much searching, a lot of editing csproj files, and even more searching to get it all working DarkRift will soon be available in both .NET Framework (for Unity) and .NET Core/Standard (for the console based servers) variants! These are fully compatible so you can mix and match the client/server components and it seems like no changes will be required to your plugins unless you want to. There's still a bit of work to do on this, some of the build tools aren't fully compatible yet and it would be good to get the documentation updated to reflect the different versions, particularly the tutorial where users might get the wrong one!

More work has also been done on the room system although not a huge amount. I've started an idea that might work to fill some of the gaps left when part of the plans fell through a few weeks ago but it's still early stages so there's not much to report!

.NET Core support will likely be a Pro only feature on release and will also likely be in beta for a few releases until it's definitely working fine.

 

Interesting, I thought it would work on UWP actually. I'll definitely take a look at this and see if why it might not be working thought it would definitely be good if you could submit a bug report to GitHub with any information you have!

Jamie

 

Hi how to get started Darkrift ? I have tried DragDemo, opened in two build but nothing happen.

 

Did you type 'demo' into the console to put it in demonstration mode?

 

Hey all!

This week a lot has been done! Firstly, more progress has been made on the room system. I've been trying lots of different options for the backing queue and threading to make sure that the matchmaking code is as efficient as possible but doesn't lockup your code when it's running. I think the final solution to this is to queue up users being enqueued or dequeued from the matchmaker and have the matchmaker threads merge those queues each tick; there's some subtleties to this though since when you enqueue someone you need to have a list of others currently queued in order to rank them all so it's a little tricky to implement!

I've also been able to tick a lot of the smaller issues off. DarkRiftServer.Start now returns a boolean indicating success and a Listening property is also available now, The rooms section has been removed form the next release's documentation because it shouldn't have been there anyway, and stack traces should now not get lost in the dispatcher (they can be found in the inner exception).

I'm going to be away this weekend so apologies for any slow replies!

 

Hey @Jamster , I'm interested in Using DR2 for https://lastepochgame.com - do you have a Discord that our team could chat with you on?

 

https://discord.gg/EUYY7Z

 

Firstly, apologise for the lack of update last week, is somehow escaped me...

These last two weeks have seen a fair bit of work on DarkRift. I have successfully found a solution to the problem of the dispatcher masking the stacktraces of errors that occur in tasks (hence making it look like there are some very strange errors coming from DR). If you need a patch for urgently this there is one in the Discord #darkrift-2-support channel!

I've also fixed a few more minor bugs such as Client.SendMessage having an inverted return value, some unreleased documentation sneaking in and unmanaged DLLs crashing the UnityServer custom inspector along with updating the feature list on the website.

Lastly I've gone back to the network listener plugins that were heavily requested. I'm hoping to get these in a releasable state for the next feature update where they will either go out in beta (so I can still make some breaking changes if need be) or will be partly implemented in the next feature update (all there but not supporting custom plugins).

Jamie

 

Hey Jamster,help me，The server runtime the server is shutdown：
Unhandled Exception:
System.Net.Sockets.SocketException (0x80004005): The socket is not connected
at System.Net.Sockets.Socket.get_RemoteEndPoint () [0x0003e] in <5071a6e4a4564e19a2eda0f53e42f9bd>:0
at DarkRift.Server.NetworkListener.ConnectionTimeoutHandler (System.Object state) [0x00062] in <39d1105326e9434f8a273baafee6795d>:0
at System.Threading.Timer+Scheduler.TimerCB (System.Object o) [0x00007] in <1fd744d936be4a229b315d57c3b96c68>:0
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00019] in <1fd744d936be4a229b315d57c3b96c68>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00096] in <1fd744d936be4a229b315d57c3b96c68>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <1fd744d936be4a229b315d57c3b96c68>:0

 

My server code

 

Interesting, that looks like a bug in DR to me. Can you post in on the GitHub issue board please?

 

Ok,No preblem.
Because players are reacting badly online now, please help me solve it as soon as possible

 

I'll get that sorted for the next release, I'm fairly sure there's a simple fix that will at least stop the server crashing!

 

thank you，I wait you

 

When the Client crash，The server still connect it，how to disconnect it？

 

DarkRift 2.1.2 has been submitted to the asset store!

Woohoo!

I apologise that it's taken a while to get this release out but this patch has a lot of content! Firstly, this contains all the bugfixes that have been made over the last month so you should now see the correct stack traces when exceptions occur in the dispatcher, disabling CreateOnEnable will no longer throw NullReferenceExceptions on UnityServer, unmanaged DLLs will also no longer crash UnityServer plus much more.

The next release should actually be a feature/minor release rather than just bug fixes and should contain listener plugins, .NET Core support, and if I work very hard possibly even rooms.

Enjoy!

 

I am your backend

 

DarkRift 2.1.2 is now available on the asset store!

 

About Hazel: DarkRift 2 was originally going to be built on top but was changed during the development to a simpler bi-channel TCP/UDP setup to make it easier to maintain.
So, the Hazel development has ceased and have some unresolved bugs, see https://github.com/DarkRiftNetworking/Hazel-Networking/issues

 

Yes and no, I'm not personally doing any work on it, just because I don't have time to, however I'm still happy to merge pull requests etc. if people submit them.

Some of the bugs are certainly not ideal but it's very unlikely I'll fix them any time soon.

 

hi Jamster
How to set timeout in client connet the server？

 

It's not currently possible I'm afraid, I plan on adding it but haven't got around to it yet unfortunately.

 

So in the ios,if can't connet the server, the client is dead. it's biggest issue.
I am already disappointed.

Client ：Unity 5.5 Android ios（some ios dead）
Server：CentOs

 

And some user is 4G,can't connet the server.
some user is wifi,can't connet the server.

 

Hey Gugua,

Can you provide more details about this? You've linked to the Hazel networking issue board rather than the DarkRift issue board, can you link to the exact issue for me and provide more details?

I understand that major bugs can be disappointing and I'm very sorry you feel that way, but I do need more details in order to solve it and you must understand it's only me working on this project!

Again, I really need more details in order to solve whatever this is... There could be any number of things that might cause this.

Jamie

 

Is it possible to make a deterministic lockstep model out of DarkRift?

 

Definitely!

I think there's a few people on the discord who are building them so it might be worth checking in there and asking for some of their advice! They're a lovely and welcoming community

Jamie

 

I can provide the code for my server and client connection if you need it.
Sometime server have this error in windows.

 

Hey gugua,

I think I know what's causing this and I'll try and produce a patch to fix it today. Just to make sure I'm right can you translate the error message "System.Net.Sockets.SocketException: <this text here>" for me please?

Jamie

 

Hey gugua,

Try this patch and let me know if it works for you! Would still be good to get a translation of the error though just to be sure

Jamie

 

[Just realised that I posted this on the wrong thread so now moved over!]

Hey all, this is a very odd timed update because I keep forgetting to do the updates

The past week or so has been focussed on improving the testability of plugins and this has been a nightmare! I've realised how horrible it is to unit test plugins as I've started testing the rooms system more, with the PluginLoadData having no public constructors it meant you sort of had to test in situ and couldn't use any mocking tools whatsoever. Instead this has now been very heavily remade to support dependency injection of almost all internal parts of DarkRift. This means you can test without the rest of DarkRift and can DI things like IClientManager to mock the behaviour of DarkRift.

There is one small change however, LogWriter plugins now take a LogWriterLoadData instead of PluginLoadData as that allows them to have a different set of accessible components . PluginLoadData will still work for the time being though so you should still have backwards compatibility!

I've also been able to do some bug fixing and some of the small features you've asked for ready for the next release. I've knocked a few issues off the board including fixing the Write/Read Bools method and adding arrays of IDarkRiftSerializables. I've also added a couple of questions to some of the bugs I don't have enough information for so make sure to check your issues else I won't be able to help!

Again, sorry I've not been very vocal about updates, I'm losing track of when Thursdays are!

 

hi Jamie:
"System.Net.Sockets.SocketException: <The remote host forcibly closed an existing connection>"
Thank you the patch,i try it.

 

Not a lot of time's past since the last update but a fair bit has been done!

Firstly and most importantly I have *hopefully* patched a bug causing the server to crash when a client very quickly connects and disconnects. This should be released in the next update if I hear back that it works! (See posts above)

The rest of the time has been dedicated to the rooms system. As I want to get this out in beta form with the next update I've been working hard to improve all aspects of it. The matchmaker has seen some interface changes and updates, now when you enqueue a player it returns you a MatchmakerQueueTask which you can use to track the progress of the matchmaker with that player or cancel it.

The matchmaker also now runs on a separate thread whether you put the server in threadsafe mode or not! This means there's a nightmare of queues and other parts to maintain but it should bring a lot of performance improvements to your servers - particularly when you have a lot of people to match.

Lastly I've abstracted groups of entities in the matchmaker to an EntityGroup collection. Since the matchmaker doesn't work directly on clients (it's very incompatible with the designs for inter-server communication if it does!) it can't provide specific functionality for them. EntityGroups help bridge that gap and should allow you to broadcast and receive messages to/from groups of clients while also being able to pass the group to the matchmaker easily or operate on groups of non-clients.

I'm very happy with the improvements to the room system and can't wait to share them with you!

 

Hey Jamster:
1.How to change to tcp or udp ?
2.when in the ios moblie,if use the "UnityClient,Connect" to server and can't connet,the client thread blocking，there is no response to what to do touch the moblie unless the process is closed.

 

Hey gugua,

DarkRift uses TCP and UDP in parallel, so if you want to use just one or the other just use SendMode.Reliable (TCP) or SendMode.Unreliable (UDP). After the next version it will be possible to use just one or the other with a bit of work!

You can use UnityClient.ConnectInBackground to not block the main thread but be aware that it seems to have problems with .NET 3.5, if you use .NET 4.6 in Unity then it should work fine!

Jamie

P.S. did the patch above work for you?

 

Hey Jamster:
I'm preparing the patch server for the next version.

P.S.When the client receives the message, the message may be a bit too large, which will cause the client card to lose the frame slowly. Is this my logical question?

 

If you send with SendMode.Reliable (TCP) you shouldn't have any issues with message size. With SendMode.Unreliable (UDP) you probably don't want to send more than 512 bytes at a time otherwise the chances of the packet being lost get much higher very quickly.

How large a message are you sending and what SendMode are you using?

 

"Unity or no Unity - Run your cloud servers from a Unity instance or run them standalone for a little extra performance."
Does this mean you can host servers without running Unity instances, and I can still use navmeshes and rigidbodies for a server auth game?

 

Not quite. You can run servers without Unity but you won't have access to Navmesh/Physics unless you add some 3rd party libraries in. If you need Unity's Navmesh/Physics/etc. then you'll need to run the server within Unity

 

This week has been continuing work on the room system/matchmaker and the listener plugins ready for the big update!

Firstly, the matchmaker now has a load of documentation! I've tried to included a lot of code examples with this page as well as providing an almost step by step usage guide so that it's very obvious as to how to get up and running. Hopefully I've covered all the major topics but I'd definitely like feedback on this during the beta!

That means that the room system is almost complete, I need to catch up on testing for it but that can partially overlap with the beta as long as all the main parts are covered.

Secondly, I've been trying to finish the network listener plugins. I noticed that I'd focussed so much on getting the server side working that I had forgotten to actually implement anyway of connecting with a custom layer on the client side so that was the rest of my week! Again there's a lot of testing to do on this still, documentation and I need to have a look through to ensure that the API is workable and future proof as well.

A short update but progress nonetheless!

 

A little more done this week!

Firstly, I've spent a fair amount of time rebasing and merging a lot of bugfixes into the master branch (I shouldn't have left it so long...) which means I have less to do when we get to release. These include boolean arrays getting reversed when deserialized and a crash when clients connect and disconnect very quickly. Also, you can new read and write arrays of IDarkRiftSerializables.

Secondly, I've written up a much more comprehensive reference for the configuration files in the advanced documentation. It should be a little better than the brief comments that were in the config file and should actually tell you *all* the attributes you can include and what they do!

Sadly, I'm not sure I'm going to be able to include the listener plugins in the next update. I suspect that the time it's going to take to thoroughly test and merge into master (particularly considering that I know there's going to be a lot of conflicts) isn't going to be reasonable for the next update. Particularly since I still need to work out source code for the Pro version. I'll try and get it in the version after though!

Jamie