Search Unity

  1. We've closed the job boards. If you're looking for work, or looking to hire check out Unity Connect. You can see more information here.
    Dismiss Notice
  2. We're running great holiday deals on subscriptions, swag and Asset Store packages! Take a peek at this blog for more information!
    Dismiss Notice
  3. Check out our Unite Austin 2017 YouTube playlist to catch up on what you missed. More videos coming soon.
    Dismiss Notice
  4. Unity 2017.2 is now released.
    Dismiss Notice
  5. The Unity Gear Store is here to help you look great at your next meetup, user group or conference. With all new Unity apparel, stickers and more!
    Dismiss Notice
  6. Introducing the Unity Essentials Packs! Find out more.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice
  8. Unity 2017.3 beta is now available for download.
    Dismiss Notice

DarkRift - Fast and Flexible Cross Platform Networking

Discussion in 'Assets and Asset Store' started by Jamster, Apr 20, 2015.

  1. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    I actually don't remember emailing you! Guess I did!

    Do both. And anything else you think of :)

    Wrapping everything in a try catch would be good to check the length is correct and that you don't try and read too much (that would throw an exception so could be used maliciously)

    I would ReadInt and then validate the int, if it's the wrong type then hopefully are it will fail validation after the read. Malicious users wouldn't get very much from changing the data type, they'll be more interested in the value, so this is really just to stop against bugs in the code.
     
  2. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    I see. Yeah, that makes sense. Any performance tips on this stuff? If not I'll just do what makes sense logically and profile later.
     
  3. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    It's a trade off: performance vs security...

    A few if statements shouldn't make all that much difference to performance but if you're worried then, as you say, profile it.
     
  4. illuminatigaming

    illuminatigaming

    Joined:
    Jan 16, 2016
    Posts:
    17
    Hello,
    i want to try making an authoritative server and your asset seems perfect. I was impressed by the demo.
    I am thinking about map awareness. The server should know the map and its colliders. I want to try to make a moba (like dota, but got a few nice ideas for heroes and different game rules), so physics are only colliding with trees/creepss/players. For this the server must know the map, but i dont know how to implement that. The embedded server could be an option, but i like the console version better. Should be easier to host it on a vps then the embedded server.
    Do you know any trick?
    Something like creating tiles and a tile could either be walkable or not. (Ground or tree)
    Then i would have to implement path finding. For a noob at dark rift and not so super Unity experienced person like me, this seems quite difficult. Any trick or tip on that?
    PS: Looking forward too see dark rift 2 :)

    As already said before, it would be nice to have some better docs than pdf files. I hate PDF Files, they seem to be a pain to read :D
    Sadly there arent any dark rift tuturials on youtube etc... But it seems great and simple by looking at the demo scripts, even if this ist powerful and customisable.
     
  5. Yukichu

    Yukichu

    Joined:
    Apr 2, 2013
    Posts:
    413
    Just some feedback: I wanted to write a server-manager-type-master-server-type-thing and downloaded DarkRift Free version. After an hour of trying to find some tutorials / documentation beyond the very sparse PDF, I gave up. I moved on and used something else.
     
  6. Conquestor2

    Conquestor2

    Joined:
    Aug 17, 2015
    Posts:
    5
    I agree that documentation is pretty non-existent, but the core principle of this plugin seems to be it's extremely light-weight. You simply write a plugin for all of your functionality, and darkrift only takes care of sending the messages. That's a benefit, imho.
     
  7. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Really it depends on how you want your game to operate. Tiles are definitely a good idea and wouldn't be too hard, embedded server would be much easier but would have reduced performance. Pathfinding isn't particularly hard to do, there's plenty of options, A* is the go-to (maybe this helps) and shouldn't take too long to do.

    Yeah PDFs aren't the best method of documentation, nor are they the most comprehensive. Realistically, that's not going to change for DarkRift 1 as it's simply not a good investment of my time but for DarkRift 2 hopefully there will be a community editable wiki (though I'll probably do the most) for tutorials/howtos/code snippets/etc. and then HTML docs for the references which should be better to browse. I've done you tube tutorials before so that could definitely happen in the future.

    I'm sorry DarkRift wasn't for you, hopefully you're still willing to try DarkRift 2 though!

    Jamie
     
  8. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    I've just finished converting my basic "mmo" project from uNet to DarkRift, it took some weeks to complete but now it is working the same or better than it was before. Most notably I have a lot less problems sending bigger chunks of terrain over the network now without getting those dreaded buffering errors from Unity. And of course now I can use the latest .NET version on the server and there is no more mix-ups between server side and client side code, it is much cleaner.

    I did realize though, after a while, that uNet is really amazingly effective in doing a lot of plumbing for simpler scenarios. It takes some effort before you accomplish the same things with DarkRift. Most notably routing of messages to the correct player object and behaviour and such things that you get for free with sync vars and RPCs. Also object spawning and proximity checking are nice features to have built in.

    Overall when all of that is taken care of I do like the separated client and server, the almost complete control of what is happening and the multi-threading possibilities though.

    One thing I have not yet understand completely is the Update method in DarkRift. Documentation says it gets called 100(?) times per second. That sounds fine, but what I have noticed also is that it can get called several times simultaneously in different threads, I've seen up to 4 threads at once. Now I'm wondering what the logic behind this is. Is it when something I do in one update call is not finished before the next tick that it calls using a new thread? To be able to use this method rather than creating my own "game loop" I need to know the exact logic behind it.

    I will post a you tube video later showing what I've done so far. Just need to polish it a little bit more first.

    Thanks
     
    Jamster likes this.
  9. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    I'm glad to hear it!

    UNet works at a much higher level than DarkRift and hence has features like that that wouldn't necessarily fit with DarkRift. It's probably better to think of DarkRift closer to LLAPI rather than the HLAPI :)

    The Update method is probably a bit fast, to be honest I would half expect people to simply use their own System.Threading.Timer/System.Timers.Timer instead as you can then control the update frequency etc. In fact the Update loop occurs on multiple threads purely because it's implemented using System.Threading.Timer which tries to retain the update frequency rather than waiting on your code to complete as you suggested. In your scenario I would create some simple background threads for this instead, that way you can enforce sequential updates and manage timing yourself.

    I really look forward to seeing it!

    Jamie
     
  10. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Hi Jamster,

    I'm running into a bug where the server mysteriously loses track of the client.

    I have a timer that repeatedly broadcasts a snapshot of the world to all clients in DarkRiftServer.GetAllConnections.

    • Things work at first.
    • Then, a short time later, DarkRiftServer.GetAllConnections suddenly is empty. So there is nobody to send snapshots to.
    • This bug only happens when I have a lot of entities (100 for example). When I test with a smaller amount (12 for example), the connection stays online.
    • The client thinks it is connected. (In an Update() loop, I check if DarkRiftAPI.isConnected and it is returning true even during the disconnect)
    • The server has no logs of the client disconnecting
    Edit: Been digging around more deeply the past hour and I discovered that DarkRiftServer.GetAllConnections is actually fine. I think it's a bug with another part of my code.
    It's very odd; I have an event that fires off when the timer ticks, and after some time, the event seemingly unsubscribes itself despite the fact that there is no code to unsubscribe.

    Edit 2: Fixed the event subscription bug, but still running into the same error. This is really weird! Both the server and client think they are connected and the server is sending packets. But the client isn't receiving them. I wonder if there is a built-in throttle for memory usage or network usage.
     
    Last edited: Apr 22, 2016
  11. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    Yes, I don't really miss the functionality in DarkRift, it's more that I was used to it in UNet almost without realizing it and it was quite a lot of work to replace that functionality in the new code. Maybe some kind of event/message routing could be a nice thing to create a s a plugin or add-on library. Once done it is actually better than with UNet, or at least a lot simpler under the hood and more predictable. I also concluded that I wanted to have different solutions for message distribution on server side compared to client side.

    Ok, thanks for info about Update method, it's clear now. I will probably use my own logic then. To be able to control when new threads are created. I have some of those background threads already for save jobs (terrain, items, character skills etc.) I was looking at using Update for handling some other things but as you say it's nicer to start with something sequential light weight stuff that starts heavier things in worker threads.

    Anyway, thanks for good answers as always, back to coding :)
     
  12. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Uh huh... Can you confirm what the error is now you've fixed various things? is It back to GetAllConnections returning an empty list or is it something different in that the client just isn't receiving messages anymore?

    There's no throttle in DarkRift, so I don't think that's the case.

    That's good, I'm glad to hear it!

    If you mean a replacement for onDistribute DarkRift 2 has a really nice system in that there is the HashSet of connections and plugins can add or remove them from the set during the DataReceived event. At the end the server distributes the message, with all changes, to all the connections in that set :)

    Yeah, update was just a small addition to make it similarish to Unity but really wanted a bit more love!
     
  13. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Can't confirm what's causing the error. Everything seems to be running fine. Messages just stop being received when there are too many.

    Maybe it's a hardware limitation?
     
  14. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    When there are too many connections?

    Interesting... I'll take a look!
     
  15. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    I'm not sure. I think it might actually have to do with message size rather than frequency of messages.

    I was looking back through the thread and it seems like DarkRift uses TCP? Maybe it's some TCP processing that is getting in the way.
     
  16. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    What is the best way to detect when the server goes down, from client side? As far as I can see the Connection.isConnected does not get set to false. Also, if I try to send things to the server all seems to go well, no errors even though the server is not there anymore.

    Do I need to send pings or keep alives myself to detect this?
     
  17. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    There's no size limit on TCP, the only limit you could hit would be inside darkrift but that's 2147483647 bytes (i.e. 2.1GB) which I'd be surprised if you'd hit!

    If you've got workInBackground set to false then you will get exceptions, if it's set to true then nothing will happen (the thread eats it) and that's a bug! I'll make it always sets isConnected false on that occasion.

    Jamie
     
  18. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Jamster - I think it's less about hitting the theoretical limit, and more about the TCP trying to adapt its buffer or something like that...

    For stress-testing, I'm sending about 30 kbps consisting of 20 snapshots per second.
     
  19. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    Thanks Jamster, that explains it :) I'll wait for that bug fix then because I like when it works in the background.
     
    Jamster likes this.
  20. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Hmmm... I'm not convinced it's TCP but it may be o_O is that from a single connection? (Depending on how big the problem is, building something I can reproduce it on would be very helpful)
     
  21. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Hey Jamster, was just about to post. I've tried really hard to reproduce it, but I haven't been able to do a simple example. I did make changes in an unrelated part of the code and mysteriously the bug has disappeared. I suspect there's some kind of error being hidden away in my code that was breaking things but not throwing an error.

    You're right - I don't think it's a TCP issue and may not be a DarkRift issue. I will post back if I am able to reproduce it!
     
  22. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Ok thanks :) It's always worrying when there's an illusive bug hanging around!
     
  23. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Indeed! It's so awesome how responsive you are. It gives me a lot of confidence relying on DarkRift in the future.

    I have a question regarding serialization: I want to write an extension function to serialize "Points" which are simply a pair of floats.

    Is this the right way to do it? (I don't have access to how DarkRift implements ReadSingles() etc. I basically want to know if I'm approaching reading/writing arrays the right way. I'm also curious how it knows when to stop... right now I just supply a "count" parameter)

    Code (CSharp):
    1.         public static void Write(this BinaryWriter writer, Point[] value)
    2.         {
    3.             for (int i = 0; i < value.Length; i++)
    4.             {
    5.                 writer.Write(value[i].x);
    6.                 writer.Write(value[i].z);
    7.             }
    8.         }
    9.  
    10.         public static Point[] ReadPoints(this BinaryReader reader, int count)
    11.         {
    12.             Point[] result = new Point[count];
    13.             for (int i = 0; i < count; i++)
    14.                 result[i] = new Point(reader.ReadSingle(), reader.ReadSingle());
    15.             return result;
    16.         }
     
  24. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    You're code looks sound but you might want to consider extending DarkRiftWriter/Reader directly rather than extending BinaryWriter/Reader and that's mainly stylistic, to anyone outside person they might wonder when you ever use a BinaryWriter/Reader in your code (ok also DarkRift2 writers/readers don't inherit from BinaryWriter/Reader so it would also be easier to port I guess).

    Internally DarkRift sends arrays as an Int32 denoting length and then the data using the appropriate Write call so I would probably stick with that, any other way and you're probably just going to increase the bandwidth used.

    Thanks!

    Jamie
     
  25. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Makes sense! Any particular reason you use an Int32 instead of, say, a UInt16 for array length?
     
  26. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    That's the maximum size of an array :) Means that DarkRift's limit is the same as .NET's :)

    With hindsight most people aren't going to want to transfer 2GB of data but if the extra 2 bytes are too much than you could always re-implement it :) On the otherhand there's a fair chance people would want to send more than 65KB (eg level streaming maybe?) so UInt16 may not be a practical limit - it all varies on application.
     
  27. Subtleoliver

    Subtleoliver

    Joined:
    Feb 10, 2016
    Posts:
    3
    Is DarkRift compatible with Unity 5?
     
  28. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Gotcha - all makes sense!

    Absolutely! It's very light weight and doesn't really depend on anything in Unity, so Unity updates shouldn't really break anything. It's one of the things that appealed to me about DarkRift.
     
    Subtleoliver likes this.
  29. Vytek

    Vytek

    Joined:
    Apr 29, 2016
    Posts:
    35
    Hi Jamster,
    I am using your superb library!! Any news about DarkRift 2.0 with UDP, etc??

    Thank you for your awesome work1
    Regards
     
    Jamster likes this.
  30. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
  31. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    I promised a video showing my game converted to use DarkRift Server. Running two clients for this demo and you can see the server console below the game windows. It uses a database to store characters and items, the map is stored on the server and streamed to the client on demand. A lot left to code but some basic game mechanics are in place. Graphics can be seen as place holder stuff :)

    Anyway here it is:
     
    RavenOfCode, Lisk and Jamster like this.
  32. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Nice! I really like those tree mechanics and the world streaming! Keep it up! :)
     
  33. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Looks great FStar!

    Jamster - would it be possible to use DarkRift to send a file over the network? I want to send game data at the start of the game from the server to the client. Specifically, a zip file containing about 50kb of xml files.
     
  34. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Should be fine to :) 50KB isn't that much!
     
  35. FStar

    FStar

    Joined:
    Sep 11, 2015
    Posts:
    50
    Thanks, Looking forward to convert it to DarkRift 2 :)
     
    Jamster likes this.
  36. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Great!

    Unfortunately I'm getting inconsistent errors when I send/receive a zip file created via DotNetZip. I think it's actually a bug with DotNetZip. Half the time it works, and half the time it gets a BadReadException.

    Probably just going to send files individually, although it means I'll be sending several megabytes rather than 50kb.

    Edit: I was able to reproduce a problem. When I serialize bytes via

    Code (CSharp):
    1. writer.Write(File.ReadAllBytes(path_to_my_file))
    DarkRift mysteriously doesn't send the messages after the first one in my loop.

    I know this because when I substitute in

    Code (CSharp):
    1. writer.Write("hello")
    the messages get sent.


    Edit 2: nevermind, my test was erroneous. It's actually being sent. The problem is still on the receiver. Will edit with an update when I am able to diagnose the problem.

    Edit 3: Still trying to fix the problem. It is as if reader.ReadBytes() is non-blocking, so what happens is I try to parse it as Xml, but I get Xml errors depending on how far along ReadBytes made it.

    Edit 4: Wasn't able to figure out the bug. I will explore using something other than DarkRift for syncing the files.
     
    Last edited: May 3, 2016
  37. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Interesting, the data should definitely not be corrupting... No doubt you've seen it but is this thread does report a bug with BadReadExceptions...
     
  38. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Thanks for the reply - I edited my post above. I'm running into trouble even without the DotNetZip interaction.
     
  39. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    I'll try and mock something up later and see if I can reproduce it, I'll try without DotNetZip see if I lose anything that way we might be able to find out exactly what's the problem.
     
  40. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Sounds good, keep me posted. It isn't specific to DotNetZip - I think it just has to do with large-ish files (>60kb maybe).
     
  41. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    I just realised I never released the last update...

    Guess I should do that at some point...
     
  42. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    @Lisp I can't reproduce your problem, I'm getting all the data and none mangled as I should do, both directions...

    Can you send a minimum working example? :oops:
     
  43. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    Jamster - I was unable to produce a minimum working example, unfortunately.

    Fortunately, though, I ended up coming up with a different solution to the problem; just syncing the files by using File.WriteAllBytes() rather than trying to read them in memory as it is read off the stream.
     
  44. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    @Jamster - I have a feature request / another question

    Currently, in response to an ConnectionService.onPlayerConnect event, I have some logic which results in sending a message to all connections in DarkRiftServer.GetAllConnections()

    However, the GetAllConnections is not populated yet with the currently connecting player, at the time of that player connecting.

    Is there a way to get all connections, including the one that just connected?
     
  45. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    In 1.4.2 which I sent to the asset store last night there's an onPostPlayerConnect event which is called after they've been added, so yes there's sort of!
     
  46. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    90
    That's good to know!

    Btw, I was able to reproduce the file transfer problem.

    On the server, I have

    Code (csharp):
    1.  
    2.   using (DarkRiftWriter writer = new DarkRiftWriter())
    3.   {
    4.   byte[] map = File.ReadAllBytes("Maps/" + name + ".zip");
    5.   byte[] hash = Extensions.ComputeHash(map);
    6.   writer.Write(name);
    7.   writer.Write(map);
    8.   writer.Write(hash);
    9.  
    10.   DarkRiftServer.GetConnectionServiceByID(player).SendReply(1, 11, writer);
    11.   }
    12.  
    On the client, I have

    Code (csharp):
    1.  
    2. using (DarkRiftReader reader = (DarkRiftReader)data)
    3.   {
    4.   string name = reader.ReadString();
    5.   byte[] file = reader.ReadBytes();
    6.   byte[] hash = reader.ReadBytes();
    7.   string path = "Maps/" + name + ".zip";
    8.  
    9.   File.WriteAllBytes(path, file);
    10. }
    11.  
    It works perfectly with a very small zip (say, 1kb). But with my actual zip of 76 kb, I get random file corruptions. The file itself shows up on the client's directory with the correct filesize (76kb), but it cannot be opened. I opened the file in notepad and it is missing a large chunk of data. Also, it appears that it is missing different bits every time I run the test.
     
  47. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    Ok, I'll have a look into it. It's a bit bizarre the corruptions are random though with the internet errors are quite random...
     
  48. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    DarkRift 1.4.2 is live on the Asset Store!
     
  49. Gavii

    Gavii

    Joined:
    Apr 27, 2015
    Posts:
    1
    You have any tutorial where unity and Dark Rift use?. I have problems when exporting to any platform
     
  50. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    830
    The tutorials are all included, what's your problem with the build?