Showcase Mirror - Open Source Networking for Unity

Consider using a JSON blob and sending that back and forth. It may be more data over the network but you can serialise a lot of data into JSON text string and have it unpacked on the server, etc.

I use JSON for scoreboard updates where the server sends all clients the updated scoreboard every time there's an update.

 

I had an issue where I thought the community version (improvements branch) of UNET was causing packet loss over a Wireless client PC connecting to a LAN computer running Unity Editor as the Host as everything would freeze over the network intermittently and then it would "pop" to the correct locations on the client when it unfroze. The editor displayed everything fine during the client "freeze", but the client would have a delay.

This doesn't happen on wired LAN computers it seems. I then ran iperf3 for Windows and checked the bandwidth between the wifi client and the host - sure enough, the wifi connection throughput on the client was very patchy (even if the router was just across the desk from it!). I can't seem to get this issue to fire unless the host instance does something very intensive that is spamming the CPU, and iperf3 on a LAN pretty much saturates the cable.

Just keep that in mind when using UNET and testing your game network code in your local network. For a ideal setup, I'd recommend using a switch and wired connections, but I guess you need to test it over wifi as well to cover all bases so that'll suffice.

BTW... Has anyone used the network simulator option in the NetworkManager?

 

Good morning everyone.

Working on the 'community' part of HLAPI Community Edition now.
Feel free to join our new Discord: https://discord.gg/N9QVxbM
We can discuss features, goals, etc. there!

Edit: also added explanations about branches, so people don't get disappointed about rejected pull requests.

 

Trying to pass png as bytes from client to server, however I am getting this error:

Code (CSharp):

1. IndexOutOfRangeException: NetworkReader:ReadByte out of range:NetBuf sz:0 pos:0
2. UnityEngine.Networking.NetBuffer.ReadByte ()
3. UnityEngine.Networking.NetworkReader.ReadUInt16 ()
4. UnityEngine.Networking.NetworkReader.ReadBytesAndSize ()
5. SendData.Deserialize (UnityEngine.Networking.NetworkReader reader) (at Assets/ClientMenuServer.cs:15)
6. UnityEngine.Networking.NetworkMessage.ReadMessage[SendData] ()
7. ServerScript.ReceiveSelfie (UnityEngine.Networking.NetworkMessage myMessage) (at Assets/ServerScript.cs:46)
8. UnityEngine.Networking.NetworkConnection.HandleReader (UnityEngine.Networking.NetworkReader reader, Int32 receivedSize, Int32 channelId)
9. UnityEngine.Networking.NetworkConnection.HandleBytes (System.Byte[] buffer, Int32 receivedSize, Int32 channelId)
10. UnityEngine.Networking.NetworkConnection.TransportReceive (System.Byte[] bytes, Int32 numBytes, Int32 channelId)
11. UnityEngine.Networking.NetworkServer.OnData (UnityEngine.Networking.NetworkConnection conn, Int32 receivedSize, Int32 channelId)
12. UnityEngine.Networking.NetworkServer+ServerSimpleWrapper.OnData (UnityEngine.Networking.NetworkConnection conn, Int32 receivedSize, Int32 channelId)
13. UnityEngine.Networking.NetworkServerSimple.HandleData (Int32 connectionId, Int32 channelId, Int32 receivedSize, Byte error)
14. UnityEngine.Networking.NetworkServerSimple.Update ()
15. UnityEngine.Networking.NetworkServer.InternalUpdate ()
16. UnityEngine.Networking.NetworkServer.Update ()
17. UnityEngine.Networking.NetworkIdentity.UNetStaticUpdate ()

 

1. Use improvements version. Fixes this error appearing randomly and shows better debug message.
2. Please don't ask here and in discord. Only ask at one place and we don't have to reply twice.

 

Hi,

Can anyone recommend a good dedicated server package that is tested and working well with UNET community ed?

Thanks!

 

Namecheap works

 

Update: uploaded latest 2017.4_improvements DLLs. A lot of small improvements, and two major big ones:

• NetworkCRC removed
  • NetworkCRC sent all NetworkBehaviour script names over the network, only to compare their channelIds with those on the server. This used a lot of bandwidth for virtually no benefit. Why would anyone build the server, then modify a script's channel for no reason, then build the client? A real CRC for the whole project might have been useful, but only comparing channelIds is not worth the bandwidth and code complexity at all.
• ChannelBuffer/ChannelPacket removed
  • NetworkConnection had a list of ChannelBuffers, each of them had a ChannelPacket. This was implemented to provide reliable messages, message fragmentation and send delay buffering before NetworkTransport implemented those features. NetworkTransport does implement those features now, so it's enough if we send all messages directly to NetworkTransport.
    • => Significantly reduces memory consumption and GC for each packet
    • => Reduces computations needed to buffer and fragment packets
    • => Reduces latency because HLAPI and LLAPI don't both have a SendDelay anymore, only LLAPI
  
  The HLAPI code is getting better and less complex every day, and performance, gc, bandwidth and stability are also improving constantly!

 

Hello,

Is the "SyncVars getting desynced on player join" bug fixed either in the official Unity 2018.1 or in the HLAPI Community Edition fixes branch?
I've been searching for this, also checked the repository, but haven't found anything. Thank you.

Also, you could consider using the Bitbucket built-in Issue Tracker.

 

Hello everybody, I'm thinking about switching to this version of the HLAPI, but I wonder if it allows for "reconnection" (I.E: someone closes the game by accident while playing and wants to come back to the same match). In the standard HLAPI this is not possible because "hot-join" is prevented.

 

Try fixes, then try the improvements version and let us know if it fixes it or not.
I never heard of this bug. If it still exists then please send us a project where we can reproduce it to fix it.

 

already answered you in discord. please don't ask questions in two places

 

Update: uploaded latest 2017.4 fixes and 2017.4 improvements DLLs. Found (and fixed) an interesting bug today where HLAPI sent a message with the wrong MsgType opcode to the client, result in NetworkReader trying to read invalid data and causing errors.

With the built in HLAPI it's impossible to overwrite NetworkManager.OnServerAddPlayer to spawn a player object that's already in the scene while running in host mode. Looks like the code was implemented without ever being tested.

This is what caused the bug, notice how the lower half sends the wrong MsgType.



Friendly reminder that we are fixing Unity's bugs for free. Please support us on Patreon if you benefit from our work.

 

News: we moved HLAPI CE to Github: https://github.com/vis2k/HLAPI-Community-Edition/

 

I was just looking for a solution to my problem and I came across this thread by chance... If you tell me that it is possible to solve it with your UNET Community Edition, I will simply give you a +rep!

 

Feel free to try the improvements DLLs and let us know if it worked or not..

 

Huge Update: latest improvements version cuts bandwidth at least in half, more likely a 2-10 fold reduction in most projects. UNET's serialization code was really poorly designed, serializing all the components for all the channels all the time. What the hell Unity?

Latest improvements DLLs can be found in first post.

Changelog:

• NetworkIdentity.UNetUpdate serialization Bandwidth overkill fixed

• Problem: NetworkIdentity.UNetUpdate serialized all components for each channel if any component for the channel was dirty. This caused at least twice the unnecessary bandwidth, usually even more:
  • Most UNET projects use at least two channels: Unreliable and Reliable. So all of the components would be serialized once for Unreliable and then once again for Reliable and then sent over the Network, doubling the necessary bandwidth just about all the time. If a project uses 5 channels, then it would be 5x the unnecessary bandwidth. This is completely insane.
    
  • If one component was dirty then all components would be synced and sent over the network again. For example, if the player moves and the component gets dirty, then not just the movement, but also all other components like Health, NetworkTime, etc. would be serialized. If the player has 10 components and only one of them is dirty, then all 10 would be serialized. This again causes unnecessary bandwidth depending on the amount of your components, so it can easily cause 10x the required bandwidth. This too is completely insane.
  • Additionally, the SendInterval setting was almost entirely ignored because OnSerialize would be called if another component's sendInterval was elapsed.

• Solution: a dirtyComponents bit mask is included in each serialization. Only the dirty components for a given channel are serialized.

​

I will start another community stress test for my uSurvival asset soon to get some detailed numbers on the improvements here.

... in other words: UNET on steroids.

 

So I have not check here in a few months,

did the possibility of these fixes being added to unity3d main branch go anywhere?

 

If you look at their repository, they have never merged any pull request. They just don't take code from other people at all.

 

Unity tried to use my postprocess sceneid fix, but they didn't use the exact same code and failed to do it properly, the bug is still there.

 

Been researching some bandwidth optimizations, turns out that UNET already does a pretty good job on that. Added some notes to the first post, for those who care:

UNET Byte Optimizations
The UNET Weaver automatically uses WritePackedUInt for:

• SyncVars
• SyncListStruct's structs
• Messages that inherit from MessageBase
• Command/Rpc parameters

So for example, if you have a 'health' SyncVar that uses int, you won't gain anything by converting it to 'short' since UNET uses the WritePacked version anyway. This is really useful since dealing with bytes and shorts in C# isn't very pleasant.

The only thing left for the user to optimize are custom OnSerialize/OnDeserialize functions.

 

Just like to say that you have been doing amazing work. Unet was unusable for my application and since upgrading to community I am very close to getting a workable networked game! Still, it is a little difficult being a novice but getting there!

Regarding the above Byte Optimizations. I'm still not 100% happy with the built in network transform module. Its def better but I get such amazing syncing results using a custom script based on Vector3 and Quaternion SyncVars. Problem is I get the feeling this method is heavy lifting and I am pumping too much data down the network. Does byte optimization do anything when sending Vector3's and such??

 

I am 0% happy with the built in NetworkTransform. It's not highest priority, but we will fix this at some point!

 

Hahha good! At least I'm not crazy!!

 

Is any support for 2018.2 planned or is this project mostly sticking to the 2017.4 LTS cycle?

 

Try the 2018.1 download in the first post. Need to update it to the latest fixes again, but it should work.

UNET team didn't upload 2018.2 HLAPI source yet: https://bitbucket.org/Unity-Technologies/networking/branches/ . Feel free to ask them.

 

I wouldnt make NetworkServer in to static. The singleton pattern allows for using a mock NetworkServer with testing.

 

Take a look at the old NetworkServer + NetworkServerSimple code, it doesn't make testing very enjoyable

 

Ah, that makes sense. I'll give your suggestion a shot.

 

This puts things in perspective.

fixes branch:

Code (csharp):

1. -------------------------------------------------------------------------------
2. Language                     files          blank        comment           code
3. -------------------------------------------------------------------------------
4. C#                              41           2599            731          14579
5. MSBuild script                   1              0              5             91
6. -------------------------------------------------------------------------------
7. SUM:                            42           2599            736          14670
8. -------------------------------------------------------------------------------

improvements branch:

Code (csharp):

1. -------------------------------------------------------------------------------
2. Language                     files          blank        comment           code
3. -------------------------------------------------------------------------------
4. C#                              31           1742            640           9693
5. MSBuild script                   1              0              5             81
6. -------------------------------------------------------------------------------
7. SUM:                            32           1742            645           9774

That only includes the Runtime part of HLAPI CE.

This means we have removed 34% of the code. I am sure a lot of bugs were lurking under the surface in there.

 

Another nice improvement. Let's say you have this code:

Code (CSharp):

1. public class SampleBehavior : NetworkBehaviour
2. {
3.  
4.     [SyncVar]
5.     public int intData;
6.     [SyncVar]
7.     public string strData;
8.  
9.     public SyncListInt intsynclist = new SyncListInt();
10. }

HLAPI sees that code and generates a OnSerialize method for your class. This is what HLAPI regurgitates:

Code (CSharp):

1. public override bool OnSerialize(NetworkWriter writer, bool forceAll)
2. {
3.     if (forceAll)
4.     {
5.         writer.WritePackedUInt32((uint)this.intData);
6.         writer.Write(this.strData);
7.         SyncListInt.WriteInstance(writer, this.intsynclist);
8.         return true;
9.     }
10.     bool flag = false;
11.     if ((base.get_syncVarDirtyBits() & 1uL) != 0uL)
12.     {
13.         if (!flag)
14.         {
15.             writer.WritePackedUInt64(base.get_syncVarDirtyBits());
16.             flag = true;
17.         }
18.         writer.WritePackedUInt32((uint)this.intData);
19.     }
20.     if ((base.get_syncVarDirtyBits() & 2uL) != 0uL)
21.     {
22.         if (!flag)
23.         {
24.             writer.WritePackedUInt64(base.get_syncVarDirtyBits());
25.             flag = true;
26.         }
27.         writer.Write(this.strData);
28.     }
29.     if ((base.get_syncVarDirtyBits() & 4uL) != 0uL)
30.     {
31.         if (!flag)
32.         {
33.             writer.WritePackedUInt64(base.get_syncVarDirtyBits());
34.             flag = true;
35.         }
36.         SyncListInt.WriteInstance(writer, this.intsynclist);
37.     }
38.     if (!flag)
39.     {
40.         writer.WritePackedUInt64(base.get_syncVarDirtyBits());
41.     }
42.     return flag;
43. }

And this is what HLAPI CE improvements produces now:

Code (CSharp):

1. public override bool OnSerialize(NetworkWriter writer, bool forceAll)
2. {
3.     bool result = false;
4.     if (forceAll)
5.     {
6.         writer.WritePackedUInt32((uint)this.intData);
7.         writer.Write(this.strData);
8.         SyncListInt.WriteInstance(writer, this.intsynclist);
9.         return true;
10.     }
11.     writer.WritePackedUInt64(base.get_syncVarDirtyBits());
12.     if ((base.get_syncVarDirtyBits() & 1uL) != 0uL)
13.     {
14.         writer.WritePackedUInt32((uint)this.intData);
15.         result = true;
16.     }
17.     if ((base.get_syncVarDirtyBits() & 2uL) != 0uL)
18.     {
19.         writer.Write(this.strData);
20.         result = true;
21.     }
22.     if ((base.get_syncVarDirtyBits() & 4uL) != 0uL)
23.     {
24.         SyncListInt.WriteInstance(writer, this.intsynclist);
25.         result = true;
26.     }
27.     return result;
28. }

Unity's logic in a nutshell:

 

NetworkServerSimple to me looks like an attempt to fix the design mistakes of NetworkServer. but was never finished.

 

Hence why we removed it. It's also insanely complex.

 

I apologize for my English, translate it using google translate
After the last two updates, my project began to swear at the following:

Code (CSharp):

1.         NetworkServer.RegisterHandler(MsgType.AddPlayer, OnAddPlayerMessage);
2.         NetworkServer.RegisterHandler(MsgType.Disconnect, OnDisconnect);
3.         NetworkServer.RegisterHandler(MsgType.Connect, OnConnected);

error CS1502: The best overloaded method match for `UnityEngine.Networking.NetworkServer.RegisterHandler(short, UnityEngine.Networking.NetworkMessageDelegate)' has some invalid arguments
error CS1503: Argument `#1' cannot convert `UnityEngine.Networking.MsgType' expression to type `short'

Code (CSharp):

1.              public static short UID = MsgType.Highest + 1;
2.             public static short Custom = MsgType.Highest + 2;
3.             public static short Crypt = MsgType.Highest + 3;

error CS0266: Cannot implicitly convert type `UnityEngine.Networking.MsgType' to `short'. An explicit conversion exists (are you missing a cast?)

and

Code (CSharp):

1.         ClientScene.readyConnection.SendWriter(m_LocalTransformWriter, GetNetworkChannel());

error CS1061: Type `UnityEngine.Networking.NetworkConnection' does not contain a definition for `SendWriter' and no extension method `SendWriter' of type `UnityEngine.Networking.NetworkConnection' could be found

How can I fix this?

 

Use NetworkServer.RegisterHandler((short)MsgType.AddPlayer) etc. Sorry about breaking your code, but MsgType is an enum now, as it should be. This is a lot better and .ToString() is built in now, instead of having a MsgTypeToString function with those weird arrays.

 

Well, I made the changes and it worked out
But what to do with this error?

Code (CSharp):

1. ClientScene.readyConnection.SendWriter(m_LocalTransformWriter, GetNetworkChannel());

error CS1061: Type `UnityEngine.Networking.NetworkConnection' does not contain a definition for `SendWriter' and no extension method `SendWriter' of type `UnityEngine.Networking.NetworkConnection' could be found

I took an example from this topic https://forum.unity.com/threads/synchronizing-an-entire-skeleton.355636/

 

The easiest solution is to use the HLAPI_CE fixes version from the first post. This is 100% compatible with original HLAPI.

In HLAPI CE we removed the functions that were redundant. Previously you had SendBytes, SendWriter, SendMessage. People should always send messages though, nothing good can ever happen when you sand random bytes to a client that expects a message. this is why we removed all send functions except SendMessage. It would be best practices for external scripts to use those functions too.

Note: it's not actually called SendMessage. It's 'Send' or 'SendByChannel' or 'SendToReady', whatever you need.

I should explain this in a longer post or on the first page. There's a lot to say on this topic and it makes perfect sense if you understand the reasons behind it.

 

Just after some quick clarification. I'm on 2018.1. Is the download link on the first page being updated with all the latest fixes I see here coming through every week or is 2017.4 the only version getting regular updates?

 

2018 wasn't stable enough for our own projects so the improvements branch will focus on 2017.4 LTS until 2018.4 LTS is released and stable.

Fixes are for 2018 too. Will update to latest one again soon.

Someone said that 2017.4 improvements DLLs work on 2018 too though. Might be worth a try.

 

Thanks for that. I think I will just wait for a 2018 update! I'm on the latest 2018 release and haven't noticed anything too unstable.

 

I have implemented sync to owner if anyone wants to test it out.

To use it, you just add a target attribute to [SyncVar], for example:

Code (CSharp):

1. public class Player : NetworkBehaviour {
2.  
3.     // regular sync var,  works as before
4.     [SyncVar]
5.     public int health;
6.  
7.     // only synchronize with owner
8.     [SyncVar(target = SyncTarget.Owner)]
9.     public string experience;
10.  
11.     [SyncVar(target = SyncTarget.Owner)]
12.     public SyncListInt myInventory;
13. }

With this code, the inventory and experience will not be synchronized to anybody but the client that has the player. This can save a huge amount of bandwidth in some projects.

 

https://blogs.unity3d.com/es/2018/08/02/evolving-multiplayer-games-beyond-unet/

Unet deprecated. No more unet in 2018.4

 

News:

We posted the rumors before, but now Unity made it official. The blog post mentioned early 2019 for a somewhat stable V1.0. Considering that current LLAPI+HLAPI aren't really stable after 3 years, I am pretty sure we can wait for 2020 until we can really use it in production.

Until then we will keep working on HLAPI CE. and I will release a LLAPI CE soon.

We ran into a lot of issues when stress testing uMMORPG. Built in HLAPI+LLAPI were able to run about 50-70 players before all kinds of errors started. I decided to make my own LLAPI and in our recent experiment, it ran really damn good:





101 players with 24% CPU usage, 0 packet loss, 0 networking issues, very low RAM usage.
My LLAPI uses less than 300 lines of code. There is no magic and it can be modified/improved/maintained very easily.

We had to stop the test because we didn't have enough people to add more clients. We will shoot for 200 CCU next.

So there is hope after all..

 

I know that standard UNET does not support generics in classes derived from NetworkBehaviour, the thing simply refuses to compile. Is there any support for this in UNET HLAPI Community Edition?

I have refactored a system I wrote to not need them anymore but the whole system is fustratingly fragile, it strait up can stop compiling randomly if I reference a NetworkBehaviour from a class with generic types in its definition ( public class Foo <T> { )

 

Not planned at the moment, sorry.
Didn't even know that someone had those problems.
What would it be useful for?

 

I'd really like to check out the LLAPI CE to see if it better fits my needs. Congratulations on your achievements with it.

 

Thanks. Gonna be available very soon!

 

You hit your maximum cpu calculation (25%), wich is not good for me. That's mean your LLAPI take the maximum availables resources.

Care about that.

 

What do you mean exactly/ where do you see 25%?

 

"101 players with 24% CPU usage, 0 packet loss, 0 networking issues, very low RAM usage."

Ok near 25% -,-