Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

BenchmarkNet (Stress test for ENet, UNet, LiteNetLib, Lidgren, MiniUDP, Hazel, Photon and others)

Discussion in 'Connected Games' started by nxrighthere, Jan 13, 2018.

Thread Status:
Not open for further replies.
  1. dzmitry-lahoda

    dzmitry-lahoda

    Joined:
    Oct 9, 2018
    Posts:
    4
    Did somebody was able to run benchmark or part of clients in such environments?
    1. Android + .NET Standard 2.0 + Mono
    2. Android + .NET Standard 2.0 + IL2CPP
    2. iOS + .NET Standard 2.0 + IL2CPP

    To be sure System.Net.Socket + IL2CPP + .NET Standard 2.0 implementation of Unity works well on mobile devices.
     
  2. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    @dzmitry-lahoda I can perform such tests for you on LG Nexus 5, if you want.

    And thank you for the contribution, I've sent you an email.
     
  3. Vincenzo

    Vincenzo

    Joined:
    Feb 29, 2012
    Posts:
    46
    Hey, that sounds very intresting, I tried doing this myself once on my game data I send to clients, but found out it was actually adding more bytes than removing (from the header i suppose) I even tried experimenting with the dictionary function for LZ4, by logging data im sending to clients, then training the LZ algoritm on that to create a dictonary, it diden't give any benefits either.

    Maybe I'm already at maximum compression, and cleverness with data, since i'm working on bit level on every piece of data and so on, but anyway, perhaps allowing enet lz4 implementation work with supplied dictionary might be a great idea, what do you think?
     
  4. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    It depends on compression level, but in general, this happens when the buffer doesn't contain enough repetitive data, and the algorithm is not able to do anything with it. The plain text for example.

    Hard to say actually, trial and error until we found what works best for our data, this is how we are doing it. :)

    I've tried several solutions, and in terms of speed and compression ratio, LZ4 works best in my tests. The closest competitor is Density.
     
  5. dzmitry-lahoda

    dzmitry-lahoda

    Joined:
    Oct 9, 2018
    Posts:
    4
    @nxrighthere, my end goal to find if there is any of open source library which works well during intermittent worsening of network conditions (packets larger than MTU during abrupt world change while playing during bus ride) in constrained device (iOS or Anrdoid IL2CPP) with .NET Standard 2.0 Unity target (forward looking compilation target). And that knowledge somehow shared with community (right now, I guess, somebody knowы what open source libraries work well in that setting, but I did not found yet).

    If you or anybody else will test that and share(at least part of setup - like Android IL2CPP .NET Standard 2.0 with tests from BenchmarkNet with no network issue) - would be very cool. Right now I am trying to finger point best open source by reading issues and test only one against Photon. That may not work.

    I bothered because specific build target end environment may not work for networking code, like https://github.com/aspnet/SignalR/issues/3122
     
    Last edited: Nov 5, 2018
  6. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    This is where the concept has already failed. Read this thread, you can find there useful stuff, regarding this. And here's some advice from the creator of ENet.

    LiteNetLib support .NET Standard 2.0 if I remember correctly, so you can try it.
     
  7. logixworx

    logixworx

    Joined:
    Jun 29, 2017
    Posts:
    72
  8. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    Trying to fix two bugs that I found a few days ago, and then pushing the code to the repository. Documentation and API reference is not finished yet as well, so it will take time.
     
    Prodigga, mons00n and logixworx like this.
  9. Prodigga

    Prodigga

    Joined:
    Apr 13, 2011
    Posts:
    790
    Is the repository supposed to be completely empty except for the readme and the license?
     
  10. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    @Prodigga If you are talking about a repository that related to the ScionNet, then yes, I didn't upload the source code yet. I'm trying to fix a bug which reveals when a high parallelism degree used to consume batches of network messages in multiple threads.
     
  11. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    You can do this when your data is optimized even on generic 4-core CPU.
     
    Last edited: Nov 14, 2018
  12. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    705
    500 players. Just amazing.
    Quick question. What's the degree of parallelism on the demo been used?
     
  13. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    First option. In this demo, synchronization functions/state updates called per entity, and every frame batches of network messages consumed and processed in the main thread.

    updatestate.PNG
     
    wobes likes this.
  14. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    705
    Much appreciate that.
    I'm shocked that single producer and single consumer give results like that.
     
  15. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    You can run a thread on a certain core using ProcessorAffinity. But I don't recommend to do that for many reasons.
     
  16. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    705
    So just creating a Thread would be enough?

    Code (CSharp):
    1.   networkThread = new Thread(UpdateNetworkThread)
    2.             {
    3.                 Name = "Network thread"
    4.             };
    5.  
    6.             networkThread.Start();
    7.  
     
  17. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    Yes, and just follow this general recipe.
     
    wobes likes this.
  18. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    705
    Thank you so much.
     
  19. buFFalo94

    buFFalo94

    Joined:
    Sep 14, 2015
    Posts:
    203
    Hi @nxrighthere first of all you did a such amazing job on ScionNet. I'm just curious you wrote this on scionnet github repository
    can you please explain why scionnet is limited to these platforms ? if i remember correctly scionnet use a modified version of Enet which is compatible with Android and IOS
     
  20. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    I don't know anything about ARM processors/operating systems, this is not my area, and I'm not interested/motivated to work in this direction. Also, I don't have appropriate devices/resources. However, everything should work in constrained environments, but how well it will work there is a question.
     
  21. buFFalo94

    buFFalo94

    Joined:
    Sep 14, 2015
    Posts:
    203
    Okay i see
     
  22. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    151
    Does ScionNet handle message/packet batching? For example, if there are requests to send many message/packets to same target in one frame such as different sync var or sync position of other entities. Does your framework send the sync packets individually or automatically batching and send one big packet?
     
    Last edited: Nov 17, 2018
  23. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    If you enqueue within a single frame N messages, networking thread will consume them all at once and then push the whole batch for processing to the transport. ENet will combine as many packets as possible, and send them to the designated peers.
     
  24. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    151
    That's nice! So, ENet will handle the packets combining. Is this feature already builtin to your current ENet? I mean if I use another HLAPI but I use ENet as a transport level. Can I still get this benefit? Is the process automatic or do I have to setup any configuration?
     
  25. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    Sure, you will benefit from it automatically, don't need to change any options.
     
    jjobby likes this.
  26. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    So, the bug that I was trying to catch around a week is sitting in the Unity's Mono (why I'm not surprised?). In total, I found 4 bugs in Unity while working on ScionNet. For 3 of those guys I've made workarounds trading performance, but the last one is impossible to get around, and it's freezing the editor/standalone build. .NET Framework/.NET Core is unaffected of course. How to create something more or less serious when the environment hangs from using only a delegate, array, and for-loop? Tired of this.
     
  27. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    705
    Don't leave the scene though.
     
  28. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    151
    I see that you removed Unity as a support platform from ScionNet git page. That's unfortunate. The network testing in your video looks very promising.
     
  29. Zuntatos

    Zuntatos

    Joined:
    Nov 18, 2012
    Posts:
    523
    Is this with the .net 3.5 mono or the .net 4.6 one? Can you also share some of the specifics of the bugs?
     
  30. lazalong

    lazalong

    Joined:
    Oct 13, 2009
    Posts:
    139
    nxrighthere, can you produce a stripped down project that reproduce the issue?
     
  31. Vincenzo

    Vincenzo

    Joined:
    Feb 29, 2012
    Posts:
    46
    Did you report those bugs to the bugtracker with a simple reproduce project? I would suppose unity is intrested in fixing them.
     
  32. buFFalo94

    buFFalo94

    Joined:
    Sep 14, 2015
    Posts:
    203
    i'll suggest you to share with guys here on the forum maybe someone will find a workaround
     
  33. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    Sorry for the delayed response, I'm chilling and experimenting with smmalloc (I recently brought it to the .NET world) using it with Span<T>, multi-threading, and stuff.

    Still improving the codebase of the ScionNet daily. Also, I'm porting examples to Xenko since it's lack of advanced networking system as well.

    Mono: two bugs related to interoperability, only the .NET 4.x is affected. Another bug related to memory barriers and the last one hangs the runtime, both framework versions are affected.

    IL2CPP works without any problems.

    Not yet, but I'm going to submit them soon.

    The only potential workaround is to use interfaces instead of delegates, but this will involve a complexity that I'm trying to avoid across the code...
     
    Last edited: Feb 21, 2019
    ScottPeal and wobes like this.
  34. docseuss

    docseuss

    Joined:
    Apr 5, 2016
    Posts:
    38
    I know this is an older thread but any chance you have or will be benchmarking the UNET replacement Mirror Networking Solution?
     
  35. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,149
    No, Mirror is not a UDP transport.
     
    nxrighthere and Vincenzo like this.
  36. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    I already said many times across this forum section that comparing TCP and UDP doesn't make any sense.

    TCP is a stream-oriented protocol which works like writing to a file, a dumb stream of data that suffer head-of-line blocking, doesn't provide message boundary preservation, lack of essential transmission mechanics that we are using in games, and so on.

    UDP is a message-oriented protocol that is usually used for userspace protocol implementations on top of it that solves problems that were said above, but the primary goal is multiplexing to achieve better latencies and avoid head-of-line blocking, this is crucial for multiplayer games.

    On top of this, TCP operates with socket descriptors while traditional UDP works with socket addresses (it can work with many socket descriptors as well, but this is a big topic...).

    This differences makes this things absolutely incomparable, it's apples and oranges.

    Worth noting that the overall performance of your networking layer on top of any low-level UDP library is highly depends on how well it's integrated into your application. In Unity your only way to achieve absolutely best performance is ECS, Jobs system, and Burst.
     
  37. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    546
    Valve's new networking APIs has been released officially with the open-source end-to-end protocol which is available on GitHub. The wrapper was updated to the first stable release.

    For those who were waiting for some basic smoke tests, performing the asynchronous simulation within a single process is not possible, unfortunately, because the library has per process recursive timed mutex to preserve thread-safety across functionality.
     
    Last edited: Mar 16, 2019
Thread Status:
Not open for further replies.