Disrupt Networking

Hello everyone,

I have been solo developing a networking solution for about six years, I hope this solution disrupts the current market selections we have today. As developers we are the ones under the gun and although some solutions are easy to use they come at a cost, others bring promise then let us down. As a community we have searched high and low for a decent product that can fill this void without breaking our pockets and/or time constraints. I have had an idea to make things easy, networking should not be a pain staking process but instead be fun. Thus Distrupt is what I have embarked on and here are the results.

• Source will be made available through a Proprietary license.
• Send data with networked instantiate (classes, structs, variables, (Unity3d's Vector3 and Quaternion's)
• Send as many variables through Sync params as needed.
• Views consist of instantiator, host or shared.
• Reliable / Sequenced type networking
• Each host can handle (100+ players).
• Relay Server handles room-based games, Stun, Turn and ICE calls.
• UPNP can be setup if it fails drops back to Relay Server.
• LAN based discovery.
• Relay Server is also setup to handle saving and loading of data.
• Packet headers are 2 bytes.
• Packet RD's remote deliveries are 8 bytes + Payload.
• 100 players each syncing 20 Game Object orientations constantly
• Easy to use Events.
• Easy learning curve, you should know how to use the whole system in about one hour.
• Once purchased its free for life (Per seat guys please be honest).
• Runs on potatoes BenchmarkNet test results a second (12% cpu, 50mb ram, 240mb transfer) 1k benchtest. https://github.com/nxrighthere/BenchmarkNet/wiki/Benchmark-Results
• RD attributes are a bit like magic but do not use Reflection during runtime. (1 million calls to RD takes 240ms) using store actions based off of Reflection during editor mode.
• Currently Disrupt uses boxing/unboxing for Sync params, I am searching for answers to find the fastest possible way to do this. For now 2k+ objects all moving in the scene is not bad at all. If I find away to avoid boxing/unboxing we can double that.
• Standalone RelayServer with instructions to setup on Digital Ocean or AWS.
• A Discord channel to stay in touch with me the developer or the community using Disrupt.

Cons?

• Must be compiled against 4.x
• Must allow unsafe code (why?)

"I dont use a BinaryWriter instead I have a custom solution to speed up the packing process."

Code (CSharp):

1.         public unsafe void GetBytes(Packet packet, short value)
2.         {
3.             fixed (byte* b = &packet.PayLoad[packet.Length])
4.                 *((short*)b) = *&value;
5.             packet.Length += 2;
6.         }

In depth look below.

Simple Messaging System (37 lines of code)

Code (CSharp):

1. using RavelTek.Disrupt;
2. using UnityEngine;
3. using UnityEngine.UI;
4.  
5. public class Messenger : NetHelper
6. {
7.     public Transform MessageHolder;
8.     public Text Message;
9.     private Peer[] peers = new Peer[4];
10.     private int count;
11.  
12.     public void Start()
13.     {
14.         Disrupt.Client.OnConnected += Client_OnConnected;
15.     }
16.     private void Client_OnConnected(Peer peer)
17.     {
18.         if(!View.IsServer) return;
19.         peers[count] = peer;
20.         Sync("NetMessage", peer, "Welcome to the server");
21.         count++;
22.     }
23.     public void SendMessage(InputField field)
24.     {
25.         //Many ways to sync a message
26.         Sync("NetMessageReceived")
27.         .Add("Some Message")
28.         .Send(SendTo.Others)
29.         .Send(SendTo.All)
30.         .Send(SendTo.Server)
31.         .Send(SomePeer or Some Peers[])
32.     }
33.     [RD]
34.     public void NetMessageReceived(string message)
35.     {
36.         var inMessage = Instantiate(Message, MessageHolder);
37.         inMessage.text = message;
38.     }
39. }

The RD stands for Remote Delivery (Thought it had a nice ring for an attribute).

Serializable Values

Code (CSharp):

1.  
2. namespace RavelTek.Disrupt
3. {
4.     [System.Serializable]
5.     public enum ValueType
6.     {
7.         Boolean,
8.         BooleanA,
9.         SByte,
10.         SByteA,
11.         Byte,
12.         ByteA,
13.         Char,
14.         CharA,
15.         UInt16,
16.         UInt16A,
17.         Int16,
18.         Int16A,
19.         UInt32,
20.         UInt32A,
21.         Int32,
22.         Int32A,
23.         Single,
24.         SingleA,
25.         UInt64,
26.         UInt64A,
27.         Int64,
28.         Int64A,
29.         Decimal,
30.         DecimalA,
31.         Double,
32.         DoubleA,
33.         String,
34.         StringA,
35.         Object,
36.         Vector3,
37.         Vector3A,
38.         Quaternion,
39.         QuaternionA,
40.     }
41. }

Vector3A and QuaternionA are just enums that can handle Vector3 arrays and Quaternion arrays. Quaternions are sent as 4 bytes each variable ("xyzw") being 1 bytes each, Vector3's are sent as 12 bytes ("xyz") each currently 4 bytes as half math didn't work as promised. There is a OrientationView component that only syncs the variables in positions or rotations that have changed. So if position X changes in the position you will only send one variable being 4 bytes Quaternion would have been 2 bytes.

Many types of Networking Instantiates

Code (CSharp):

1.         #region Default Instantiates
2.         public static GameObject Instantiate(GameObject item)
3.         {
4.             return ProcessInstantiate(item.transform.position, item.transform.rotation, item);
5.         }
6.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation)
7.         {
8.             return ProcessInstantiate(position, rotation, item);
9.         }
10.         #endregion
11.         #region Specialized Instantiates
12.         public static GameObject Instantiate(GameObject item, params object[] data)
13.         {
14.             return ProcessInstantiate(item.transform.position, item.transform.rotation, item, null, data);
15.         }
16.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation, params object[] data)
17.         {
18.             return ProcessInstantiate(position, rotation, item, null, data);
19.         }
20.         public static T Instantiate<T>(T item, params object[] data) where T : Component
21.         {
22.             var outItem = ProcessInstantiate(item.transform.position, item.transform.rotation, item.gameObject, null, data);
23.             return outItem.GetComponent(typeof(T)) as T;
24.         }
25.         public static T Instantiate<T>(T item, Vector3 position, Quaternion rotation, params object[] data) where T : Component
26.         {
27.             var outItem = ProcessInstantiate(position, rotation, item.gameObject, null, data);
28.             return outItem.GetComponent(typeof(T)) as T;
29.         }
30.         #endregion
31.         #region Default Single Send Instantiates
32.         public static GameObject Instantiate(GameObject item, Peer peer)
33.         {
34.             return ProcessInstantiate(item.transform.position, item.transform.rotation, item, peer, null);
35.         }
36.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation, Peer peer)
37.         {
38.             return ProcessInstantiate(position, rotation, item, peer, null);
39.         }
40.         #endregion
41.         #region Default Multi Send Instantiates
42.         public static GameObject Instantiate(GameObject item, Peer[] peers)
43.         {
44.             return ProcessMultiInstantiate(item.transform.position, item.transform.rotation, item, peers, null);
45.         }
46.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation, Peer[] peers)
47.         {
48.             return ProcessMultiInstantiate(position, rotation, item, peers, null);
49.         }
50.         #endregion
51.         #region Specialized Single Send Instantiates
52.         public static GameObject Instantiate(GameObject item, Peer peer, params object[] data)
53.         {
54.             return ProcessInstantiate(item.transform.position, item.transform.rotation, item, peer, data);
55.         }
56.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation, Peer peer, params object[] data)
57.         {
58.             return ProcessInstantiate(position, rotation, item, peer, data);
59.         }
60.         #endregion
61.         #region Specialized MultiSelect Send Instantiates
62.         public static GameObject Instantiate(GameObject item, Peer[] peers, params object[] data)
63.         {
64.             return ProcessMultiInstantiate(item.transform.position, item.transform.rotation, item, peers, data);
65.         }
66.         public static GameObject Instantiate(GameObject item, Vector3 position, Quaternion rotation, Peer[] peers, params object[] data)
67.         {
68.             return ProcessMultiInstantiate(position, rotation, item, peers, data);
69.         }
70.         #endregion

Events can be extended through partial class using Client

Code (CSharp):

1.         public delegate void ConnectionRequest(Packet packet);
2.         public delegate void Connected(Peer peer);
3.         public delegate void Disconnected(EndPoint endPoint);
4.         public delegate void Message(Packet packet);
5.         public delegate void NatSuccess(Packet packet);
6.         public delegate void Discovery(NatInfo natInfo);
7.         public delegate void HostList(NatInfo[] hosts);
8.         public delegate void HostSuccess();
9.         public event ConnectionRequest OnConnectionRequest;
10.         public event Connected OnConnected;
11.         public event Disconnected OnDisconnected;
12.         public event Message OnIncomingMessage;
13.         public event Discovery OnDiscovery;
14.         public event HostList OnHostList;
15.         public event HostSuccess OnHostSuccess;

Those are only some of the highlights if there is anything else the community would like to know I am transparent with questions.Price aiming for $20 per seat this wont last for long I will not go any higher then $60 so get yours as soon as it comes out. -Levon

 

nice to see new network asset, haven't seen good one since fholm's days
in any case even if it can hold 100 players it will be impressive..

but it sounds like science fiction to be frank..
1k players with 8% cpu usage? how many packets per second do you send ?
does it have delta compression ?
what kind of networking library do you use ? (ENet, NanoSockets, etc..)
its only standalone ? (so not authoritative unless you have custom physics)
source code available ?
available platforms ?

also any videos and demos would be great

 

GCat, Your questions in order
how many packets per second do you send ?

• Packets sent are based per frame, in my case I was doing 100 packets per frame at 60 frames a second it was about 6k packets a second from the server. All movement was tied into one update and then I looped the 2k Object positions and Rotations using a Orientation View that was driven in the networking loop. I have a list of Orientation Views in my networking loop and based off the ObjectsPerSync a for iteration is called and does that amount then the next frame increases the count etc, etc.(if you have a 4 player game you should not be sending that many in a frame)..

does it have delta compression ?

• Yes, I take most values and do a few equations to reduce the bytes sent. Its an option because you wont get precise results for example.
• Quaternion's are 1 / -1 as a float thats 4 bytes so with this example I can reduce the bandwidth cost by multiplying the value by 100 lets say you have .99999 gives you 99.9 now we send that value over the wire as a short (2 bytes) being 99 once it arrives it will be 99 * .01 giving you .99 we miss a few values here but to the eye its not noticeable.
• Little tricks here and there to reduce bandwidth are implemented, I got you guys.

what kind of networking library do you use ?

• I used six years of my life to roll my own solution, Sequence packets were easy the Reliable layer was not I do not use an ACK / NACK type system. One layer for transport is all that's needed improving the performance on any application. There is no GC in my solution as packets are pooled/reused and essentials are placed on the stack. This is an image of Distrupt the reason im fast is because all of the RD's are stored before the game is even compiled. Also I forgot I dont use a window slider for the reliable layer so there is that.. Happy figuring that one out to all networking developers
• Please note I am trying to figure out a way to make viewing the method info store as an option, I might be releasing it as hidden so there isn't much clutter, but for those who would like to see the coolness I am working it out. All the Values seen here are the RD attributes gathered during Editor session then when you press play we essentially have a direct reference to all the methods.

its only standalone ? (so not authoritative unless you have custom physics)

• No, A lot of time was spent in this area looking at all the other solutions on the AssetStore, I chose a different path for physics. The players will set their Ridged Bodies off the host / server will control all of the physics and or movement.Put your application into headless mode for the server/host not sure what its called now but I remember 4.6 if you know about Fholm more then sure you were around then.
• If you make a build like one person as a host and one as a player then put your game into headless mode essentially you have a Server Auth type game just host that on the cloud.

source code available ?

• This one is tough, I am in a toss up honestly I would love to release the source.. Right now I am releasing the plugin source code. My back-end for Disrupt is RavelNet and that is what I am wondering about "With proper licensing" Yes.

available platforms ?

• Any Platform will work with this solution, all calls are Mono friendly.

Videos are on the way, hope I was able to answer all your question properly (Fholm is who im trying to beat plus ENet with ease of use that RakNet days back in 4.x) Fohlm's socket was in C++ though, I did everything in C# . -Levon

 

I want to make a note in the features, to be fair you can have 2k players moving all at once but thats all they will do, so realistically 2000 / 100 players gives a very very fair game with each player having 20 Orientation views. I will update my features to read correctly, you can possibly squeeze in more that if you know exactly what your doing. If you just slapping updates everywhere in your game and not optimizing it sure thats not Disturbs fault, if your careful and manage the ms you could hit way more then 100 players. Depends on the developers skillset, if you have been developing for a few years and know what GC is and how ms effects the frame rate your looking at more connections.

 

2k Objects syncing from server to clients all clients look the same.. Sorry guys could not figure out how to post a gif.. This was tested from California (USA) to India I had one server hosted on a Droplet and one client on my PC, The person in India recorded the results it matched my client so I am not sure exactly how you guys feel about it but I feel great. Here is the last one each round fired from the tank is still moving even though it cannot be seen on the screen.. This has to be downloaded unfortunately to view. @ the end of that video 1.6k objects were syncing rotations and positions off the scene view. I will get more tests up on Youtube soon but these were some impressions that I captured with ShareX.

Let me know if you think I was being obtuse about the results I was projecting through my post from the above information. I do not think I was but who knows? Guess what im getting at is none of the objects in those gifs would be in a real world application. 2K all at once? Even if we did a Battle Royal type game the chance for everyone to be all in one area at one time is not that great, so you can optimize for off scene calls -Levon

 

you should consider granting source access (in github) for ones with invoices
can't think of a reason why not.. just write some none sharing license

specially this day and age when networking assets are abandoned
it will boost confidence in this asset and let people do customization for their own needs

 

Wow, I thought it was an image.. you really should upload to YouTube
its an amazing system if it works like in the GIF,
Bolt's Achilles heel is syncing many states.. let alone 2k

looking forward for this asset to be on the store!

 

Thank you I am currently documenting everything and making videos shortly I was very sick over the weekend.

 

I am making the source available through a Proprietary License.

 

that's great news! very excited for this system

I wanted to ask if its possible to send custom byte[] into your low level network library (RavelNet)

btw an idea for future updates is to use C# Job System and Burst Compiler within Disrupt/RavelNet
should also consider replacing locks if you have any with lockless queue ringbuffer

although disrupt seems very fast from the start and that's a good sign
I hope you will take different approach, first maximize performance then add new features

 

I wanted to ask if its possible to send custom byte[] into your low level network library (RavelNet)

• You may send almost anything you can think of as long as its serializable

btw an idea for future updates is to use C# Job System and Burst Compiler within Disrupt/RavelNet

• I use unsafe code to convert to bytes, not sure how the Unity.Mathmatics will help in that area.

I hope you will take different approach, first maximize performance then add new features

• I have maximize as much as possible, there could be room for improvement for now I am releasing the package so everyone can take a look then possibly let me know where or what could be optimized.

should also consider replacing locks if you have any with lockless queue ringbuffer

• I do use a few locks on the Networking thread for the Queue but they are strictly in and out
• lock(objectLock) var packet = SendQueue.Dequeue(); <-- negligible

Also Unity3d will not be blocked by Distrupt as it is a Consumer Producer pattern. -Levon

 

Finally got around to making the banner for the Asset Store check it out in the first post. -Levon

 

you don't use Mathf anywhere else in your code ? what do you use to compress the data ?

 

No Mathf in the code, as for compression take a look at the unsafe in first post (search for) "I dont use a BinaryWriter instead I have a custom solution to speed up the packing process." .. When you do Sync("Method", SendTo.Blah, (byte) 1, new byte[2]{1, 2}) my backend does this.

Code (CSharp):

1. var packet = client.CreatePacket(); ("Creates a packet from a pool)
2. packet.Protocol = Protocol.Reliable;
3. packet.Flag = Flags.Dat;
4. packet.SendType = YourSendType;
5. var writer = new Writer(); (I dont create a new Writer every time but for example im doing it here)
6. writer.PushByte(params[0]);
7. writer.PushByteA(params[1]);
8. lock(sendQueue) SendQueue.Push(packet); (lock is negligible)

its a very fast process 1 million of these calls takes about 2ms -Levon

 

I want to reply to all the likes and let you guys know I am fixing a few bugs that have popped up during testing. I am at about 90% bug free based off of many tests I and beta testers are throwing at it. I promise without a shadow of a doubt upon release improvements and maintenance will be in a timely manner. This is for those who know how hard networking is and seeing my results. "It has taken 6 years with a family of five day in day out to make what is coming shortly." So without further adieu my gratitude for those who are paying attention to the thread. (No particular order just a recognition and my thanks to you as I do watch the forum every day. I also want to do something special for those who comment I am adding your names with links to your profile in blue. If you have assets or contributed to Unity3d if they click your name developers will see your references.-Levon)

• Mikael-H
• GCat - Developement testing and code optimizations (Thank you)
• Neviah
• LostPanda
• MrArcher
• ElroyUnity
• Tiny-Tree
• Giovanni (Lead Developer on EonWar, development fixes and testing)
• Mario (Owner of Eonwar)
• Sabaresh (Owner of FlipSetter)
• Dario Ortega (Development testing)
• xVergilx
• Player7
• jmjd
  
• Also to those who are looking but are hesitant to reply or like I still want to thank you. "I know seeing is believing and Disrupt Networking is very hard to believe in. As a developer I don't rely on assets from the store as there are very few assets that say "what is promised do as promised". I am skeptical just as you are, I will do my best to show you through comments and reviews that this is the real deal hopefully one day I can make a believer out of you."

I also would like to mention, the videos you might have seen with cubes falling or moving could let you down. With Disrupt Networking I have distributed beta API's for testing in projects that have been growing for many years. To developers amazement, the ease of use and performance over anything in the Asset Store has surpassed many expectation's. "Do not take my word for this, I don't expect you to." I am gathering the developers of such said projects to comment here on the forum.

Thank you everyone for your time and support, from the bottom of my heart. -Levon

 

Update, I am tight in development getting past a few bugs. Honestly I can say its almost ready for release there is a short hiccup with mixture of server auth and peer to peer. I know this is going to happen with a lot of developers so I am doing my best to make sure if and when it does happen its not an issue.-Levon

 

There's no sources right now, I can only assume Sync looks similar to this:

Code (CSharp):

1. public void Sync(string methodName, ... destination, params object[] objects);

The only way of unwrapping that would be to create specific overloads for the method to avoid boxing value types + creating object array.

UNet used an automatic weaver for that kind of things by using attributes.
But since networking is mostly per project, I think its best to make separate base methods for basic value types up to 3 parameters. This can be done via code autogen swizzling.

Rest of overloads should be possible to re-implement on the customers' end.
But anyhow params must go, as its pretty nasty.

Also, looking at OP, it seems like params arrays also used for the Instantiates. That might hinder the performance as well. Personally I'd rather use custom messages for instantiations. Then again, it might be useful for quick prototyping.

Other than that, looks promising, even though I don't make network games anymore, I might check this one out.

 

I had done something similar to this before instead I created a DomCompilerUnit and just looked through all the Attributes and created a new class that referenced all of the methods. I might go back to that example but yes your right params object[] currently is being used, also thank you for your time and interest. -Levon

 

Interesting, though I think getting your networking solution supported by some of the other networking related assets is what will sell it for some.. like dissonance-voice-chat , simple-network-sync etc maybe provide code early to some of them to see if they will implement support?

 

Most of the plugins your mentioning are already implemented as components in Disrupt, I am going to release more as time passes but for now the main solution is what I am trying to pack down. Maybe later I can investigate assets that have been tried and tested with very low bandwidth consumption and optimized techniques to plug into Disrupt.-Levon

 

I am currently looking at the params objects that xVergilx and myself were going over, I have decided to keep it as a params object but, I will be making a thread for the networking. Basically what this will do is take in your parameters from the Sync method and queue them to the networking thread so Disrupt takes the heavy lifting leaving the engines thread running without issues. So Disrupt is no longer a Task running off of Unity3d, the downside of this is the engine and network have a thread each so your application will be running two threads. The upside of this is major performance gains, the engine will no longer have to wait for networking logic to complete and a bit more time for development testing. I know waiting sucks, I am really excited for the solution and want the best possible release for everyone. "I promise you will enjoy it not as much as I have developing it" one to many headaches . -Levon

 

any new updates ?

 

There is a bug with id assignment with prefabs, wouldnt have been an issue if Unity3d didnt use nested prefabs. I have also optimized the orientation view to process faster. Turns out you should never compair to null its faster to check is null. Other then that documentation and tutorials then release..

Here is an example of what I mean by is null or == null

Code (CSharp):

1. if(object == null) <--- very slow
2. if(object is null) <-- way faster less overhead

 

If 'object' in your example is a UnityEngine.Object, then using "is Null" may cause undesirable effects as "==" is overridden in Unity to check if the object has been destroyed on the c++ side of the engine. There's probably many places where this is discussed, but a quick search brought up this thread about it: https://forum.unity.com/threads/optimizing-null-check-null-vs-bool-out-functions.482118/

 

The object isn't a unity object its actually a class, I should have made the example better . Good catch though and thank you for the provided link and information, very cool. - Levon

 

Latest video of performance improvements & stress testing, seems at around 3k objects all moving is the limit; at 4k im getting some break up. https://drive.google.com/file/d/1SSnM9DDLTlg50QXYIQlUs8RjE3cFgugd/view and yes im running Ubuntu 18.04 for those who noticed the desktop. -Levon

 

To everyone watching, I had GCat take a thorough look at the source code. I will let him explain his findings, I think before the release some house cleaning is in order and that includes direct references to classes made. In turn this will create something called a Dom so I have a bit more work cut out for me. Just hang tight this solution is not like the rest on the AssetStore mark my words. -Levon

 

yep, I've tested the source code and I can say for sure
its better in terms of performance then any networking solution out there!
also its convenient with an automatic state syncing and direct events to functions.

Levon uses unsafe code, smart threading management and high performance codes
where needed for maximum packet throughput

he is being humble when he says it can hold 100 players or sync 1k moving entities
it can hold much much more, in my tests I've also synced thousands of moving cubes like in his example
each cube is syncing Vector3 position and Quaternion rotation of itself at a rate of 25Hz.

that operation took 3.6ms! most of unity's ms went on the cube rendering side..
so in headless mode you can sync insane amount of states..

try syncing even 200 moving cubes (each cube is really 3 states for pos and 4 for rot)
in Bolt, Forge, Mirror or what ever other networking solution out there (publicly available)
unity would crash on the spot, so this is an amazing network engineering feat made by Levon.

the problem with many networking solutions they iterate over every connection and every entity for each connection
then iterate over every state for each entity then sort them out into a packet.
it works well for small number of players but scales very poorly after 64 players..
disrupt compiles everything ahead of time into one giant dictionary and does its magic linearly.

the foundations are solid, but disrupt is still missing few key points like
client prediction, delta compression, entity freezing..
any competent dev can make these parts himself, but it would be nice for easier working stack.

Great work Levon, I will definitely use this system and i think many people will too!

 

Thank you for being so humble, and for the insight hope many others will feel the way you do. -Levon

 

Any news?

 

any news?! what happened to this project ? I kept reading some amazing features then its dead for 3 years

 

reason for the reworks are the design patterns I was introduced to when working at this new company. They actually improved readability of the code and helped structure what I had much better. Quick shout out to Kyle, for helping me with these patterns and please feel free to learn about them here The Catalog of Design Patterns (refactoring.guru) these patterns improved the performance and readability of what I had over 50%.