DarkRift - Fast and Flexible Cross Platform Networking

I actually don't remember emailing you! Guess I did!

Do both. And anything else you think of

Wrapping everything in a try catch would be good to check the length is correct and that you don't try and read too much (that would throw an exception so could be used maliciously)

I would ReadInt and then validate the int, if it's the wrong type then hopefully are it will fail validation after the read. Malicious users wouldn't get very much from changing the data type, they'll be more interested in the value, so this is really just to stop against bugs in the code.

 

I see. Yeah, that makes sense. Any performance tips on this stuff? If not I'll just do what makes sense logically and profile later.

 

It's a trade off: performance vs security...

A few if statements shouldn't make all that much difference to performance but if you're worried then, as you say, profile it.

 

Hello,
i want to try making an authoritative server and your asset seems perfect. I was impressed by the demo.
I am thinking about map awareness. The server should know the map and its colliders. I want to try to make a moba (like dota, but got a few nice ideas for heroes and different game rules), so physics are only colliding with trees/creepss/players. For this the server must know the map, but i dont know how to implement that. The embedded server could be an option, but i like the console version better. Should be easier to host it on a vps then the embedded server.
Do you know any trick?
Something like creating tiles and a tile could either be walkable or not. (Ground or tree)
Then i would have to implement path finding. For a noob at dark rift and not so super Unity experienced person like me, this seems quite difficult. Any trick or tip on that?
PS: Looking forward too see dark rift 2

As already said before, it would be nice to have some better docs than pdf files. I hate PDF Files, they seem to be a pain to read
Sadly there arent any dark rift tuturials on youtube etc... But it seems great and simple by looking at the demo scripts, even if this ist powerful and customisable.

 

Just some feedback: I wanted to write a server-manager-type-master-server-type-thing and downloaded DarkRift Free version. After an hour of trying to find some tutorials / documentation beyond the very sparse PDF, I gave up. I moved on and used something else.

 

I agree that documentation is pretty non-existent, but the core principle of this plugin seems to be it's extremely light-weight. You simply write a plugin for all of your functionality, and darkrift only takes care of sending the messages. That's a benefit, imho.

 

Really it depends on how you want your game to operate. Tiles are definitely a good idea and wouldn't be too hard, embedded server would be much easier but would have reduced performance. Pathfinding isn't particularly hard to do, there's plenty of options, A* is the go-to (maybe this helps) and shouldn't take too long to do.

Yeah PDFs aren't the best method of documentation, nor are they the most comprehensive. Realistically, that's not going to change for DarkRift 1 as it's simply not a good investment of my time but for DarkRift 2 hopefully there will be a community editable wiki (though I'll probably do the most) for tutorials/howtos/code snippets/etc. and then HTML docs for the references which should be better to browse. I've done you tube tutorials before so that could definitely happen in the future.

I'm sorry DarkRift wasn't for you, hopefully you're still willing to try DarkRift 2 though!

Jamie

 

I've just finished converting my basic "mmo" project from uNet to DarkRift, it took some weeks to complete but now it is working the same or better than it was before. Most notably I have a lot less problems sending bigger chunks of terrain over the network now without getting those dreaded buffering errors from Unity. And of course now I can use the latest .NET version on the server and there is no more mix-ups between server side and client side code, it is much cleaner.

I did realize though, after a while, that uNet is really amazingly effective in doing a lot of plumbing for simpler scenarios. It takes some effort before you accomplish the same things with DarkRift. Most notably routing of messages to the correct player object and behaviour and such things that you get for free with sync vars and RPCs. Also object spawning and proximity checking are nice features to have built in.

Overall when all of that is taken care of I do like the separated client and server, the almost complete control of what is happening and the multi-threading possibilities though.

One thing I have not yet understand completely is the Update method in DarkRift. Documentation says it gets called 100(?) times per second. That sounds fine, but what I have noticed also is that it can get called several times simultaneously in different threads, I've seen up to 4 threads at once. Now I'm wondering what the logic behind this is. Is it when something I do in one update call is not finished before the next tick that it calls using a new thread? To be able to use this method rather than creating my own "game loop" I need to know the exact logic behind it.

I will post a you tube video later showing what I've done so far. Just need to polish it a little bit more first.

Thanks

 

I'm glad to hear it!

UNet works at a much higher level than DarkRift and hence has features like that that wouldn't necessarily fit with DarkRift. It's probably better to think of DarkRift closer to LLAPI rather than the HLAPI

The Update method is probably a bit fast, to be honest I would half expect people to simply use their own System.Threading.Timer/System.Timers.Timer instead as you can then control the update frequency etc. In fact the Update loop occurs on multiple threads purely because it's implemented using System.Threading.Timer which tries to retain the update frequency rather than waiting on your code to complete as you suggested. In your scenario I would create some simple background threads for this instead, that way you can enforce sequential updates and manage timing yourself.

I really look forward to seeing it!

Jamie

 

Hi Jamster,

I'm running into a bug where the server mysteriously loses track of the client.

I have a timer that repeatedly broadcasts a snapshot of the world to all clients in DarkRiftServer.GetAllConnections.

• Things work at first.
  
• Then, a short time later, DarkRiftServer.GetAllConnections suddenly is empty. So there is nobody to send snapshots to.
• This bug only happens when I have a lot of entities (100 for example). When I test with a smaller amount (12 for example), the connection stays online.
• The client thinks it is connected. (In an Update() loop, I check if DarkRiftAPI.isConnected and it is returning true even during the disconnect)
• The server has no logs of the client disconnecting

Edit: Been digging around more deeply the past hour and I discovered that DarkRiftServer.GetAllConnections is actually fine. I think it's a bug with another part of my code.
It's very odd; I have an event that fires off when the timer ticks, and after some time, the event seemingly unsubscribes itself despite the fact that there is no code to unsubscribe.

Edit 2: Fixed the event subscription bug, but still running into the same error. This is really weird! Both the server and client think they are connected and the server is sending packets. But the client isn't receiving them. I wonder if there is a built-in throttle for memory usage or network usage.

 

Yes, I don't really miss the functionality in DarkRift, it's more that I was used to it in UNet almost without realizing it and it was quite a lot of work to replace that functionality in the new code. Maybe some kind of event/message routing could be a nice thing to create a s a plugin or add-on library. Once done it is actually better than with UNet, or at least a lot simpler under the hood and more predictable. I also concluded that I wanted to have different solutions for message distribution on server side compared to client side.

Ok, thanks for info about Update method, it's clear now. I will probably use my own logic then. To be able to control when new threads are created. I have some of those background threads already for save jobs (terrain, items, character skills etc.) I was looking at using Update for handling some other things but as you say it's nicer to start with something sequential light weight stuff that starts heavier things in worker threads.

Anyway, thanks for good answers as always, back to coding

 

Uh huh... Can you confirm what the error is now you've fixed various things? is It back to GetAllConnections returning an empty list or is it something different in that the client just isn't receiving messages anymore?

There's no throttle in DarkRift, so I don't think that's the case.

That's good, I'm glad to hear it!

If you mean a replacement for onDistribute DarkRift 2 has a really nice system in that there is the HashSet of connections and plugins can add or remove them from the set during the DataReceived event. At the end the server distributes the message, with all changes, to all the connections in that set

Yeah, update was just a small addition to make it similarish to Unity but really wanted a bit more love!

 

Can't confirm what's causing the error. Everything seems to be running fine. Messages just stop being received when there are too many.

Maybe it's a hardware limitation?

 

When there are too many connections?

Interesting... I'll take a look!

 

I'm not sure. I think it might actually have to do with message size rather than frequency of messages.

I was looking back through the thread and it seems like DarkRift uses TCP? Maybe it's some TCP processing that is getting in the way.

 

What is the best way to detect when the server goes down, from client side? As far as I can see the Connection.isConnected does not get set to false. Also, if I try to send things to the server all seems to go well, no errors even though the server is not there anymore.

Do I need to send pings or keep alives myself to detect this?

 

There's no size limit on TCP, the only limit you could hit would be inside darkrift but that's 2147483647 bytes (i.e. 2.1GB) which I'd be surprised if you'd hit!

If you've got workInBackground set to false then you will get exceptions, if it's set to true then nothing will happen (the thread eats it) and that's a bug! I'll make it always sets isConnected false on that occasion.

Jamie

 

Jamster - I think it's less about hitting the theoretical limit, and more about the TCP trying to adapt its buffer or something like that...

For stress-testing, I'm sending about 30 kbps consisting of 20 snapshots per second.

 

Thanks Jamster, that explains it I'll wait for that bug fix then because I like when it works in the background.

 

Hmmm... I'm not convinced it's TCP but it may be is that from a single connection? (Depending on how big the problem is, building something I can reproduce it on would be very helpful)

 

Hey Jamster, was just about to post. I've tried really hard to reproduce it, but I haven't been able to do a simple example. I did make changes in an unrelated part of the code and mysteriously the bug has disappeared. I suspect there's some kind of error being hidden away in my code that was breaking things but not throwing an error.

You're right - I don't think it's a TCP issue and may not be a DarkRift issue. I will post back if I am able to reproduce it!

 

Ok thanks It's always worrying when there's an illusive bug hanging around!

 

Indeed! It's so awesome how responsive you are. It gives me a lot of confidence relying on DarkRift in the future.

I have a question regarding serialization: I want to write an extension function to serialize "Points" which are simply a pair of floats.

Is this the right way to do it? (I don't have access to how DarkRift implements ReadSingles() etc. I basically want to know if I'm approaching reading/writing arrays the right way. I'm also curious how it knows when to stop... right now I just supply a "count" parameter)

Code (CSharp):

1.         public static void Write(this BinaryWriter writer, Point[] value)
2.         {
3.             for (int i = 0; i < value.Length; i++)
4.             {
5.                 writer.Write(value[i].x);
6.                 writer.Write(value[i].z);
7.             }
8.         }
9.  
10.         public static Point[] ReadPoints(this BinaryReader reader, int count)
11.         {
12.             Point[] result = new Point[count];
13.             for (int i = 0; i < count; i++)
14.                 result[i] = new Point(reader.ReadSingle(), reader.ReadSingle());
15.             return result;
16.         }

 

You're code looks sound but you might want to consider extending DarkRiftWriter/Reader directly rather than extending BinaryWriter/Reader and that's mainly stylistic, to anyone outside person they might wonder when you ever use a BinaryWriter/Reader in your code (ok also DarkRift2 writers/readers don't inherit from BinaryWriter/Reader so it would also be easier to port I guess).

Internally DarkRift sends arrays as an Int32 denoting length and then the data using the appropriate Write call so I would probably stick with that, any other way and you're probably just going to increase the bandwidth used.

Thanks!

Jamie

 

Makes sense! Any particular reason you use an Int32 instead of, say, a UInt16 for array length?

 

That's the maximum size of an array Means that DarkRift's limit is the same as .NET's

With hindsight most people aren't going to want to transfer 2GB of data but if the extra 2 bytes are too much than you could always re-implement it On the otherhand there's a fair chance people would want to send more than 65KB (eg level streaming maybe?) so UInt16 may not be a practical limit - it all varies on application.

 

Is DarkRift compatible with Unity 5?

 

Gotcha - all makes sense!

Absolutely! It's very light weight and doesn't really depend on anything in Unity, so Unity updates shouldn't really break anything. It's one of the things that appealed to me about DarkRift.

 

Hi Jamster,
I am using your superb library!! Any news about DarkRift 2.0 with UDP, etc??

Thank you for your awesome work1
Regards

 

Yup

 

I promised a video showing my game converted to use DarkRift Server. Running two clients for this demo and you can see the server console below the game windows. It uses a database to store characters and items, the map is stored on the server and streamed to the client on demand. A lot left to code but some basic game mechanics are in place. Graphics can be seen as place holder stuff

Anyway here it is:

 

Nice! I really like those tree mechanics and the world streaming! Keep it up!

 

Looks great FStar!

Jamster - would it be possible to use DarkRift to send a file over the network? I want to send game data at the start of the game from the server to the client. Specifically, a zip file containing about 50kb of xml files.

 

Should be fine to 50KB isn't that much!

 

Thanks, Looking forward to convert it to DarkRift 2

 

Great!

Unfortunately I'm getting inconsistent errors when I send/receive a zip file created via DotNetZip. I think it's actually a bug with DotNetZip. Half the time it works, and half the time it gets a BadReadException.

Probably just going to send files individually, although it means I'll be sending several megabytes rather than 50kb.

Edit: I was able to reproduce a problem. When I serialize bytes via

Code (CSharp):

1. writer.Write(File.ReadAllBytes(path_to_my_file))

DarkRift mysteriously doesn't send the messages after the first one in my loop.

I know this because when I substitute in

Code (CSharp):

1. writer.Write("hello")

the messages get sent.

Edit 2: nevermind, my test was erroneous. It's actually being sent. The problem is still on the receiver. Will edit with an update when I am able to diagnose the problem.

Edit 3: Still trying to fix the problem. It is as if reader.ReadBytes() is non-blocking, so what happens is I try to parse it as Xml, but I get Xml errors depending on how far along ReadBytes made it.

Edit 4: Wasn't able to figure out the bug. I will explore using something other than DarkRift for syncing the files.

 

Interesting, the data should definitely not be corrupting... No doubt you've seen it but is this thread does report a bug with BadReadExceptions...

 

Thanks for the reply - I edited my post above. I'm running into trouble even without the DotNetZip interaction.

 

I'll try and mock something up later and see if I can reproduce it, I'll try without DotNetZip see if I lose anything that way we might be able to find out exactly what's the problem.

 

Sounds good, keep me posted. It isn't specific to DotNetZip - I think it just has to do with large-ish files (>60kb maybe).

 

I just realised I never released the last update...

Guess I should do that at some point...

 

@Lisp I can't reproduce your problem, I'm getting all the data and none mangled as I should do, both directions...

Can you send a minimum working example?

 

Jamster - I was unable to produce a minimum working example, unfortunately.

Fortunately, though, I ended up coming up with a different solution to the problem; just syncing the files by using File.WriteAllBytes() rather than trying to read them in memory as it is read off the stream.

 

@Jamster - I have a feature request / another question

Currently, in response to an ConnectionService.onPlayerConnect event, I have some logic which results in sending a message to all connections in DarkRiftServer.GetAllConnections()

However, the GetAllConnections is not populated yet with the currently connecting player, at the time of that player connecting.

Is there a way to get all connections, including the one that just connected?

 

In 1.4.2 which I sent to the asset store last night there's an onPostPlayerConnect event which is called after they've been added, so yes there's sort of!

 

That's good to know!

Btw, I was able to reproduce the file transfer problem.

On the server, I have

Code (csharp):

1.  
2.   using (DarkRiftWriter writer = new DarkRiftWriter())
3.   {
4.   byte[] map = File.ReadAllBytes("Maps/" + name + ".zip");
5.   byte[] hash = Extensions.ComputeHash(map);
6.   writer.Write(name);
7.   writer.Write(map);
8.   writer.Write(hash);
9.  
10.   DarkRiftServer.GetConnectionServiceByID(player).SendReply(1, 11, writer);
11.   }
12.  

On the client, I have

Code (csharp):

1.  
2. using (DarkRiftReader reader = (DarkRiftReader)data)
3.   {
4.   string name = reader.ReadString();
5.   byte[] file = reader.ReadBytes();
6.   byte[] hash = reader.ReadBytes();
7.   string path = "Maps/" + name + ".zip";
8.  
9.   File.WriteAllBytes(path, file);
10. }
11.  

It works perfectly with a very small zip (say, 1kb). But with my actual zip of 76 kb, I get random file corruptions. The file itself shows up on the client's directory with the correct filesize (76kb), but it cannot be opened. I opened the file in notepad and it is missing a large chunk of data. Also, it appears that it is missing different bits every time I run the test.

 

Ok, I'll have a look into it. It's a bit bizarre the corruptions are random though with the internet errors are quite random...

 

DarkRift 1.4.2 is live on the Asset Store!

 

You have any tutorial where unity and Dark Rift use?. I have problems when exporting to any platform

 

The tutorials are all included, what's your problem with the build?