Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NAT Traversal - Automatic port forwarding, punch-through, and more!

Discussion in 'Assets and Asset Store' started by thegreatzebadiah, Apr 5, 2016.

  1. KaZe_

    KaZe_

    Joined:
    May 13, 2015
    Posts:
    12
    Thanks for the answers @thegreatzebadiah :)
    Just few more questions and remarks:

    Alright, from now on I'll keep that in mind. That was the only way here to quickly check 2 different networks :)

    Actually we are trying to avoid using the relay servers at all, is there a way to have our own relay server in case?

    Ok. But, can't I just forward an specific port in the beginning the game for both sides and do the connection using this port? is it not recommended?

    Regards,
     
  2. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    Theoretically yes, but I haven't heard of anyone doing it yet and I haven't tried myself.

    That is correct for a direct connection, the NetworkManager does the same thing. The problem is port forwarding doesn't work on all routers or may be blocked or turned off. So if the direct connection fails that's when you would try punching through. It's a separate connection over separate ports. I just realized though that the other ports you're seeing mapped are probably the ports used to connect to the Facilitator which are automatically mapped by the NATHelper.
     
    KaZe_ likes this.
  3. KaZe_

    KaZe_

    Joined:
    May 13, 2015
    Posts:
    12
    Yeah... I was looking for it yesterday and haven't find anything related to.

    So I may use the port forwarding and if it doesn't work I still have a second change by punching a hole. Alright, got it :)

    I also noticed that when I forward ports, lower ports sometimes doesn't work so much (e.g: 7777) and higher ports seems to work better (e.g 50000). Is there any specific reason for that or are these, as you previously said, ports mapped to connect to the facilitator?
     
  4. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    That's one I've never heard before. I know some very low port numbers are reserved / very commonly used but anything in that range should be fine.
     
    KaZe_ likes this.
  5. KaZe_

    KaZe_

    Joined:
    May 13, 2015
    Posts:
    12
    Testing that yesterday I noticed and I decided to change to a higher range.
    Thanks again for all the answers! It helped a lot :)
     
    thegreatzebadiah likes this.
  6. Doghelmer

    Doghelmer

    Joined:
    Aug 30, 2014
    Posts:
    120
    @KaZe_ I'm not quite following this whole discussion, but if I'm understanding correctly, would setting the NetworkManager's Network Port to 50000 be a better choice than the standard 7777?
     
  7. yotingo

    yotingo

    Joined:
    Jul 10, 2013
    Posts:
    44
    I need some help. My players are able to connect successfully according to the debug logs but player gameobjects are not being spawned into the scene for clients.

    Setup:
    1. Everything worked great before adding NAT Traversal
    2. Added NATTraversal.NetworkBehavior to my custom Network script
    3. Changed to StartHostAll() and StartClientAll()
    4. Added NAT Helper to NetworkManager gameobject

    Problem:
    1. Create game. Everything works as expected player prefab is loaded into online scene
    2. Attempt to join game. Debug.Log says that a direct connection was successfully made, online scene is loaded. However, the player prefab isn't loading and the editor hierarchy does not show the host or his player object.

    I believe the problem is with my NetworkMessage subclass but I can't figure out how to make it conform with NATTraversal.

    Here is my custom network script where I believe the problem should be happening:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.Networking;
    5.  
    6. public class NetworkCustom : NATTraversal.NetworkManager
    7. {
    8.     public int chosenCharacter = 0;
    9.  
    10.     //subclass for sending network messages
    11.     public class NetworkMessage : MessageBase
    12.     {
    13.         public int chosenClass;
    14.     }
    15.  
    16.     public override void OnServerAddPlayer(NetworkConnection conn, short playerControllerId, NetworkReader extraMessageReader)
    17.     {
    18.         NetworkMessage message = extraMessageReader.ReadMessage<NetworkMessage>();
    19.         int selectedClass = message.chosenClass;
    20.         Debug.Log("server add with message " + selectedClass);
    21.  
    22.         if (selectedClass == 0)
    23.         {
    24.             GameObject player = Instantiate(Resources.Load("Characters/NakedPlayer", typeof(GameObject))) as GameObject;
    25.             NetworkServer.AddPlayerForConnection(conn, player, playerControllerId);
    26.         }
    27.  
    28.         if (selectedClass == 1)
    29.         {
    30.             GameObject player = Instantiate(Resources.Load("Characters/B", typeof(GameObject))) as GameObject;
    31.             NetworkServer.AddPlayerForConnection(conn, player, playerControllerId);
    32.         }
    33.     }
    34.  
    35.     public override void OnClientConnect(NetworkConnection conn)
    36.     {
    37.         NetworkMessage test = new NetworkMessage();
    38.         test.chosenClass = chosenCharacter;
    39.  
    40.         ClientScene.AddPlayer(conn, 0, test);
    41.     }
    42.  
    43.     public override void OnClientSceneChanged(NetworkConnection conn)
    44.     {
    45.         //base.OnClientSceneChanged(conn);
    46.     }
    47.  
    48.     public void btn1()
    49.     {
    50.         chosenCharacter = 0;
    51.     }
    52.  
    53.     public void btn2()
    54.     {
    55.         chosenCharacter = 1;
    56.     }
    57.  
    58. }
     
  8. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @yotingo

    You definitely want that call to base.OnClientSceneChanged(). The same is true for most methods that you override in the NetworkManager.
     
  9. KaZe_

    KaZe_

    Joined:
    May 13, 2015
    Posts:
    12
    As I said earlier, in my tests here using lower ports aren't so good. Maybe this is particular to the network I'm using here. As also said earlier, some lower range ports are reserved for some purposes in the OS, getting a higher one will prevent you from choosing a port that might be already been used.
     
    Doghelmer likes this.
  10. FlyingHighUp

    FlyingHighUp

    Joined:
    Apr 23, 2012
    Posts:
    16
    Hey there :),

    Any update on the Android/iOS support?
    I'm getting cold feet as my wait nears a year.

    @clintonb
    Have you implemented something by chance?

    By reading the forum posts, I'm under the impression it's as simple as compiling .dll/.so/.a files for the target platform. Is this how others are getting around it?

    Thanks!
     
  11. KaZe_

    KaZe_

    Joined:
    May 13, 2015
    Posts:
    12
    @thegreatzebadiah Another question :)

    I've managed to make the forwarding works but after, someone was testing it and couldn't connect. After a while I realize that I've put the forwarding when the game opens, but since the game has a single and multiplayer mode after a while the port just closes and it can't connect anymore. Do you have any estimation of how long the port forwarding keeps the port open? Also, retrying forwarding the same port after returns some errors sometimes, Can't I forward the same port twice?
     
  12. yotingo

    yotingo

    Joined:
    Jul 10, 2013
    Posts:
    44
    Thanks for the reply @thegreatzebadiah

    I wasn't calling base.OnClientSceneChanged() because it throws the following error:
    "A connection has already been set as ready. There can only be one."

    Do you know how to avoid the error and still use NATTraversal?

    (P.S. you're right that calling base.OnClientSceneChanged() did make everything work perfectly, aside from the error)
     
  13. boxels

    boxels

    Joined:
    Oct 5, 2016
    Posts:
    7
    Getting same thing in our console, having dealt with this now for a few months, trying to find what on earth is causing it... it is so random, but we may have narrowed it down...

    We can play with many players if the game app is started, and everyone connects, but as soon as some players disconnect, and reconnect, the bugs start to occur randomly, where nobody can connect to the HOST at some point.

    My observations (hours and days of testing on 9 machines: 6 iMacs, 2 windows PCs (win 10, win 7), and an Ubuntu Linux Machine) trying to narrow this down :

    Initially, It looks like when a client disconnects from HOST, they never truly disconnect (see screenshot Capture(1).png). Although we disconnect from a hosted game, the console shows they are already connected when trying to re-connect again to the same HOSTed game.

    Also the error "Maximum hosts cannot exceed..." appears, when we’ve only really ever had one true game being hosted the entire test run, but multiple clients connecting and disconnecting all the while. The game app isn’t quit in these tests, suggesting something is going on with what Doghelmer is describing. We've been trying to narrow this down for a couple months, hoping to get closer to the issue. We were to launch 2 months ago, only to have this occur randomly but consistently random for 2 straight months.

    Capture (1).PNG Capture4.PNG
     
  14. Padres

    Padres

    Joined:
    Dec 28, 2012
    Posts:
    1
    Hi,

    when testing, some Host machines report this error message:

    NAT Helper: ID_NAT_TARGET_UNRESPONSIVE, when Client wants to connect, resulting in connection failure.

    Interesting part is, that when machine that hosted a game and returned this error joined as a Client to other Host, and then re-hosted the game, the Client was able to connect.

    Does anybody have a clue what is the cause of this error message?
     
    boxels likes this.
  15. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    I've got no plans for this any time soon. Punchthrough is not effective for mobile networks anyway, though it would still work for wifi.

    You can try compiling the dll for android or ios but I haven't heard of anyone getting it to work yet.

    Sorry I can't be more helpful. I would like to add mobile support but I just don't have the time to add new features at the moment.
     
  16. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    The port should be kept open indefinitely. Are you sure the port is being closed? Maybe there is some other reason the connection stops working.
     
  17. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    I don't seem to get this when testing in the Example scene. Usually it means you've already readied the connection somehow, possibly by calling ClientScene.AddPlayer(). It's not generally a big deal though and is probably safe to ignore since all it means is that the connection is already ready already.
     
  18. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @boxels I'm going to put out a new update tonight that at least fixes the "Maximum hosts cannot exceed" error. I'm not sure if it will fix your other issues or not. Please feel free to contact me via the support email on the asset store page for more direct help (looks like you've already done so previously but just a reminder I sometimes respond there a bit quicker).

    I would definitely recommend testing using the Example scene to narrow down the issue. Your output looks really funky to me though, there seems to be a couple things that are happening twice almost like you have two networkmanagers or nathelpers in the scene or you're calling StartClientAll() twice in a row really quick or something.
     
    Last edited: Mar 1, 2017
    boxels likes this.
  19. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    That is one of a handful of errors that can be returned when punchthrough fails. In this case it could also mean that the host that was being punched through to was hanging for some reason (executing some lengthy process) and was not able to respond fast enough.

    The reason it works after the original host connects to the client and then re-hosts is probably because that's kind of how punchthrough works. Once an outbound connection has been initiated there's no problem receiving incoming traffic from that connection. So by the original host connecting the original client (new host) they are saying it is ok to accept traffic from that client. Then when they re-host the original client is of course able to connect.
     
  20. Meneu

    Meneu

    Joined:
    Apr 21, 2015
    Posts:
    9
    hi~
    I m trying to use this plugin to handle the p2p connection.
    I only need a Nat-helper for performing NAT punch-through and automatic port forwarding.
    but i cant get any idea for how to use Nat-helper :(
    like:
    crate a c# TCP socket in port 12345 locally -> 192.168.0.any:12345
    2 natHelper.mapPort(12345, 0, Tcp, callback)
    i result a port map success with info like
    private Ip 192.168.0.any, private Port: 12345 public ip: 255.255.255.255 public port : 12345

    I assume that others can make connection like a public host with a forwarded port.
    eg: my Real IP:12345
    am I miss something ??

    and can i use upnp feature through plugin?
     
    Last edited: Feb 28, 2017
  21. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @Meneu Everything you described there is port forwarding, which is accomplished either using UPnP or PMP depending on the router. It looks like you've got the general idea right though.

    If you'd like to also use punch-through you'll need to call something like:

    natHelper.StartCoroutine(natHelper.startListeningForPunchthrough(onHolePunchedServer)) on the host

    and

    natHelper.StartCoroutine(natHelper.punchThroughToServer(hostGUID, onHolePunchedClient)) on the client

    When the server gets the onHolePunchedServer callback it will be passed the port that you need to listen on. When the client receives the onHolePunchedClient callback it will be passed two ports. You need to use the first port as the port the client is connecting from and the second port as the port the client is connecting to on the server.

    The NATHelperTester included with the plugin is a pretty good example.
     
  22. Meneu

    Meneu

    Joined:
    Apr 21, 2015
    Posts:
    9
    @thegreatzebadiah
    yeah, the NATHelperTester helps a lot.
    so, can i assume that NATHelper's mapPort will do the UPnP or PMP magic for me ?_?

    and yes I m in the call back :eek:nHolePunchedServer
    and I'm try to forward the new port to the local port where I listening on
    so that if one or more client making a punchThrough, i don't need to setup more server and just map the punched port to a existing server port. is that possible ?
     
  23. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    yes

    I really have no idea if this will work or not. The way I usually handle it is to listen on a new port for every client that punches through. It is an interesting idea though, I never thought try mapping the punch through port to an already listening server. I'm not sure if it makes sense to do or not but it would really simplify some things if that works so let me know!

    [edit]
    Though I did just realize that strategy is going to run into problems on routers that don't support port forwarding or have it disabled. And that's exactly when you need punchthrough to work, so on second thought I don't think that's a road you want to go down. For each call to onHolePunchedServer you'll need to open a new socket listening on the port that is passed in. No port forwarding should be necessary at this point since the hole is already punched.
     
  24. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    I just submitted the latest update. It should hit the store in a couple of days or so.
     
  25. Meneu

    Meneu

    Joined:
    Apr 21, 2015
    Posts:
    9
  26. BmDeveloperz

    BmDeveloperz

    Joined:
    Jul 1, 2013
    Posts:
    62
    This asset is amazing but i really getting hard time on making lobby system work. It would be really helpful if you could add an example scene of how it works or detailed documentation also would help much. I know it would take a couple of hours to do this but it will save much more of your customers.
     
  27. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @Olcay the lobby should work just like unity's lobby so any documentation / tutorials you find for that should apply. If there's a specific issue that seems plugin related that you're running into I can probably help you out. I think the easiest way to get it up and running is to add the NATLobbyManager component and the NetworkManagerHUD to a game object. Make sure you've assigned the lobby scene and game scene as well as the lobby player prefab and game player prefab. The lobby player prefab should have the NATLobbyPlayer component. The gui layout will probably be crap but you should be able to start a match, see players in the list, etc.

    From there you'll probably want to move away from the NetworkManagerHUD by hooking up your own code that calls StartHostAll() and StartClientAll() similar to how it's done in the ExampleNetworkManager. Then you'll probably want to replace the OnGUI methods in NATLobbyManager and NATLobbyPlayer with your own custom way of displaying, readying, and adding players.
     
  28. BmDeveloperz

    BmDeveloperz

    Joined:
    Jul 1, 2013
    Posts:
    62
    Thanks for the explanation it helped alot.
     
    thegreatzebadiah likes this.
  29. Doghelmer

    Doghelmer

    Joined:
    Aug 30, 2014
    Posts:
    120
    Edit: Ignore this post.

    A word of warning if you value your sanity: Don't haphazardly stick a "void Update()" in your NetworkManager, as it will of course overwrite the lower level version and basically destroy NAT Traversal :)
     
    Last edited: Mar 2, 2017
    thegreatzebadiah likes this.
  30. Deleted User

    Deleted User

    Guest

    @thegreatzebadiah Hello, we solved "Maximum hosts cannot exceed" bug for our game. It was disconnecting issue and when we started to use "networkmanager.StopClient()" instead of "networkmanager.StopHost()" this bug is disappeared. And, I have one issue, I using "networkmanager.matches.currentSize" to display players in current match, but for some reason it function always return "1", it's happens because it's working only with relay servers or not? Thanks for the help!
     
    boxels likes this.
  31. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    The maximum hosts bug is fixed in the latest update that is on its way to the store now (and maybe even in the one that's already up there, I'm not sure).

    match.currentSize will be the number of people connecting via the relays so it's probably not a great way to get the actual number of players. NetworkServer.connections.Count should work for you on the host at least.
     
  32. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    NAT Traversal v1.5 Released
    • Added a work around for a unet bug with reconnecting to matches.
    • Fixed an issue with host id going out of range.
    • Added a timeout for fetching external IP.
    • Added a .trim() to the local ipv6 to maybe prevent an issue on macs.
    Look at that, barely any bugs!
     
  33. Deleted User

    Deleted User

    Guest

    Thanks! Yeah, I know that I can use NetworkServer.connection when player already on server, but I can't get how to do this when you not connected to server (for matchmaking/servers list). Trying to find something in google but still nothing, it seems Unity doesn't has solution for this case and need to do own custom system (like make own web database and sending all needed info like players count from servers to this database).
     
    Last edited by a moderator: Mar 3, 2017
    boxels likes this.
  34. Doghelmer

    Doghelmer

    Joined:
    Aug 30, 2014
    Posts:
    120
    So, my game is launching on Steam next week, and I still have one remaining issue that seems to be popping up on occasion. Not sure if this is related to NAT Traversal at all, because it's so hard to reproduce with any consistency, but I figured it's worth a shot to ask here.

    Sometimes, players do not join the game properly following a scene change. No idea why, and I haven't been able to reproduce within Unity itself, but the flow looks like this, if it's any help at all (and I may be leaving something or other out):
    - OnClientSceneChanged is called on the client.
    - OnServerAddPlayer is called on the server.
    - The client's avatar is created, and NetworkServer.AddPlayerForConnection is called when the server is ready
    - OnStartLocalPlayer is called on the client, signaling them to begin the game.

    I've tried to keep all of the code in-between these things as simple as possible, but somewhere along the line, something doesn't go as it's supposed to. I recognize how broad this is and I'd like to try and get more detail when I have the chance, but @thegreatzebadiah if you or anyone else have any ideas on potential causes for this sort of thing, I'd greatly appreciate the input.
     
    Last edited: Mar 3, 2017
  35. boxels

    boxels

    Joined:
    Oct 5, 2016
    Posts:
    7
    Thanks! I think we are getting closer to figuring this out thanks in great part to our amazing dedicated developer and your update and helpful feedback on this thread! I'll reach out as soon as we get results from testing with the latest update to your plugin... keep up the good work.

    I am curious as to one thing (maybe a naive question, as I'm still new to this all). I've been working with Construct2 (for quick prototype testing when I can't seem to get past a Unity C# learning curve, I take a break and prototype with Construct2) and they use NAT Traversal, and it works perfectly every time... connects through signaling server, then peer to peer directly.
    https://www.scirra.com/store/game-making-tools/multiplayer-signalling-server-161
    https://www.scirra.com/manual/174/multiplayer

    It seems your plugin is aiming for the same thing, or is doing the same thing? I don't recall who in this thread mentioned this type of connection, only that it has been amazing so far working seamlessly with Construct2 prototyping I've been playing with (granted we have google fiber, which helps being a host to any peer-to-peer games).
    http://nature.boxels.com/MULTIPLAYER2/
     
    Last edited: Mar 3, 2017
  36. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @boxels yeah the signalling server looks exactly like a facilitator. They also make mention on that page that you may wish to consider:
    • also hosting a TURN server (separate server software required) to improve connectivity for peers where STUN cannot achieve connectivity, e.g. symmetric NAT
    which is what the relay servers are. So it sounds like their punch-through isn't 100% effective either or there would be no need for the relay. It's really just the nature of NAT. Punch-through can't be achieved between two routers with symmetric NAT. The only (unfortunate) solution is to use relays if you want 100% connect rate.
     
    boxels likes this.
  37. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    Thank you for the 1.5 update.

    It's been a while, so I'm posting the connectivity issues we're seeing from some users, ordered by number of cases:

    Host out of bounds:

    Code (csharp):
    1. host id out of bound id {0} no host exists UnityEngine.Networking.NetworkTransport:RemoveHost(Int32) UnityEngine.Networking.NetworkClient:Shutdown() UnityEngine.Networking.NetworkClient:ShutdownAll()  
    When calling this code (we do this to ensure everything is clean before creating/joining a game):

    Code (csharp):
    1.  
    2. if (NetworkServer.active)
    3.         {
    4.             NetworkServer.Shutdown();
    5.  
    6.             yield return new WaitForSeconds(0.25f);
    7.         }
    8.  
    9. NetworkServer.Reset();
    10.  
    11.         if (NetworkClient.active)
    12.         {
    13.             NetworkClient.ShutdownAll();
    14.             yield return new WaitForSeconds(0.25f);
    15.         }
    16.  
    I guess this will be fixed with the 1.5 update.

    Frequency: 5-10% of all connection attempts.

    Failed to join UNET Match

    Code (csharp):
    1. NATTraversal: Failed to join UNET Match. We'll still try and connect but it doesn't look good: failed: Failed enumerating host node id appId=1917202. Match is likely unavailable, please select another to join.
    We're not 100% sure yet if the match id passed to StartClientAll had a valid value when this error is shown, but it most likely is (we treat NetworkID.Invalid and 0 values separately).

    This is probably the issue I consider the most important of the ones we have.

    Frequency: 5-10% of all connection attempts

    Timeouts

    When connecting to a game, plenty of times there seems to be no timeout event. We have this code:

    Code (csharp):
    1.  
    2. time = Time.realtimeSinceStartup;
    3.  
    4.  // connected is set on NetworkManager.OnClientConnect()
    5. // error is set on NetworkManager.OnClientDisconnect() and NetworkManager.OnClientError()
    6.         while (!connected && !error)
    7.         {
    8.             float timeLimit = (nm.connectionConfig.MaxConnectionAttempt + 1)* nm.connectionConfig.ConnectTimeout * 0.001f + 4 ;
    9.             if ((Time.realtimeSinceStartup - time) >= timeLimit)
    10. {
    11. Debug.LogError("Haven't managed to connect in a while ( " + timeLimit + " ), time out!");
    12.                 error = true;
    13.                
    14. }
    15.  
    16.             yield return null;
    17.         }
    18.  
    Supposedly the condition inside the loop should never trigger (because before that time passes, the connection should have been made, or an error should have been passed to the NetworkManager, but it does around 66% of the time.

    Maybe I'm just not using the proper values for the wait time, so if you know of more precise ones it would be helpful.

    (By the way, our current values are 3 for MaxConnectionAttempt and 5000 millisecs for ConnectTimeout).

    Frequency: 2-5% of all connection attempts.

    Haven't released an update of the game with this version yet (will do this week), so some of the following issues may be taken care of.

    Also wanted to add that we're very happy with the plugin and to thank you for your continued support :)
     
    boxels likes this.
  38. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    PS: Just updated to 1.5 . I get a timeout when fetching the external IPv6 with default values, and looks like it also sets the IPv4 externalIP to null:

    Code (csharp):
    1. NATTraversal: Timed out fetching ip from http://ipv6.icanhazip.com
    2. NATTraversal: Timed out fetching ip from http://ipv6.icanhazip.com
    3. NATTraversal: Timed out fetching ip from http://ipv6.icanhazip.com
    Increasing timeout value seems to fix the issue, however the time passed before the timeout message definitely wasn't over the default value of 10 seconds. Maybe the initial time isn't being set and the time is counted from the start of the application?
     
    boxels likes this.
  39. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    Hi, just wanted to share some statistics on connections for our game. The number of connections should be big enough (thousands of them) to be accurate.

    Host game:

    * Successful with NAT and Relay enabled: 97,7%
    * Successful without NAT (could not get a guid in time or something else failed): 1,47%
    * Successful without Relay (could not get a match ID in time): 0,66%
    * Successful with neither NAT or relay: 0,07%

    Join game:
    * Successful through NAT: 26,52%
    * Failed with Timeout: 19,79%
    * Cancelled by user (probably a Timeout): 18,38%
    * Succesful through Relay: 17,47%
    * Successful through Direct connection: 14,82%
    * Error (exception or unexpected state): 1,99%
    * Successful but could not determine if it's relay, NAT or Direct (should not really happen!): 0,94%

    The number of timeouts is worringly high! We have currently set MaxConnectionAttempt to 3, and ConnectionTimeout to 8, I'm not sure but it should be enough time to establish a connection. However, something came to my mind: To detect an error, we override NetworkManager.OnClientError() and treat the connection as failed when an error is received. It could be that one of the 3 clients (Direct, NAT and Relay) is failing while the other two haven't failed yet, and we're stopping the process too soon because of that (specially because connecting through Relay takes longer).

    PS: This is probably related to the issue reported the posts above where the client could not join the match ( "Match is likely unavailable" ). I've done tests and that message is the one used when a match with that ID could not be found. I'm 100% sure the match Id assigned to the ga,e is being properly passed to the clients, so I suppose the problem is related to the matchmaking server deciding the match should be dropped, or the host actually quitting meanwhile the connection is being established (but sounds too unlikely to happen so often).

    I'm not sure if there's a way to ensure the match is still alive in the relay servers...
     
    Last edited: Mar 13, 2017
    thegreatzebadiah likes this.
  40. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    And more issues (I'm on a roll today!) :

    Sometimes I get the following warning after having called StartClientAll():

    NATHelper: Unexpected raknet message received: ID_NAT_TARGET_NOT_CONNECTED

    This is probably because the host connection to the punchthrough server was lost (the host got the following message):

    Unexpected message received: ID_CONNECTION_LOST

    When that happens, the OnMultiClientDisconnect() event for the NAT punchthrough is not called, so the OnClientDisconnect() event isn't either, which would lead to an indefinite wait time unless we manually stop the process.

    I found that the following code in our NetworkManager fixes the issue:

    Code (csharp):
    1.  
    2. public override void OnHolePunchedClient(int natListenPort, int natConnectPort, bool success)
    3.     {
    4.         Debug.Log("OnHolePunchedClient Listen:" + natListenPort + " Connect:" + natConnectPort + " Success:" + success);
    5.         if (!success)
    6.         {
    7.             // This forces a re-check of all pending connections, so if there are no pending connections, OnClientDisconnect() will be called
    8.             if (punchthroughClient != null)
    9.             {
    10.                 punchthroughClient.Shutdown();
    11.                 punchthroughClient = null;
    12.             }
    13.             OnMultiClientDisconnect(null);
    14.         }
    15.         base.OnHolePunchedClient(natListenPort, natConnectPort, success);
    16.     }
    17.  
    However, the main problem still remains: The host lost connection to the punchthrough server, so from that point on nobody will be able to join via NAT. To fix that, I added the following check on the NetworkManager.Update() method:

    Code (csharp):
    1.  
    2. if ((Time.realtimeSinceStartup - lastNatPunchthroughCheck) > PunchthroughCheckInterval)
    3.             {
    4.                 lastNatPunchthroughCheck = Time.realtimeSinceStartup;
    5.                 if (NetworkServer.active && natHelper != null)
    6.                 {
    7.                     if ((natHelper.rakPeer != null) && natHelper.rakPeer.IsActive() && natHelper.rakPeer.NumberOfConnections() == 0)
    8.                     {
    9.                         // Connection to Facilitator was lost, retry manually
    10.                         Debug.Log("Restarting NAT Punchthrough");
    11.  
    12.                         this.natHelper.DisconnectFromFacilitator();
    13.  
    14.                         this.natHelper.guid = 0;
    15.  
    16.                         // Custom event to notify that the guid has changed . This is also used on OnDoneConnectingToFacilitator()
    17.                         if (OnPunchthroughGUIDChanged != null)
    18.                             OnPunchthroughGUIDChanged(0);
    19.                        
    20.                         this.StartCoroutine(this.natHelper.startListeningForPunchthrough(new Action<int, ulong>(this.OnHolePunchedServer)));
    21.  
    22.                     }
    23.                 }
    24.             }
    25.  
    26.  
    Not pretty, but gets the job done.
     
    Last edited: Mar 13, 2017
  41. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    New issue:

    When no matchmaking GUID is available (because the server could not get one for whatever reason), clients connecting through relay or direct connection won't properly detect a disconnection when the host leaves.

    The OnMultiClientDisconnect event is called, but not the OnClientDisconnect. I guess the punchthrough connection is not discarded if the GUID is 0 (although a warning is shown).

    This is easily fixed by adding the following line before calling StartClientAll():
    Code (csharp):
    1.  
    2. if (guid == 0)
    3.       nm.connectPunchthrough = false;
    4.  
     
    Last edited: Mar 14, 2017
  42. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    Hi all. I just got done putting in a ton of work on the plugin to fix some issues that I don't think anyone has actually run in to, but I think they are pretty fundamental so I thought I'd give you all a heads up.

    During the connection process, while a relay connection is being replaced, it was possible for messages to be dropped if they were sent at just the wrong time. It would be pretty unlikely for a random message to just happen to be sent at the wrong time, but it turns out it can actually happen pretty often with things like spawn messages that are sent as soon as a client connects. And of course dropping a message over a reliable channel should be impossible so it had to be fixed.

    I've addressed the issue now so that it is impossible for a message to ever get lost. During the process though I realized another issue that may or may not matter, but it's worth being aware of:

    When replacing a connection I can guarantee that no messages are lost but I can not guarantee their order.

    This is because each connection has it's own message queue. So, for example, if a client is sending a constant stream of messages over a relay connection, and then it gets replaced by a direct connection, the server may receive a few newer messages on the direct connection before older messages on the relay. For the most part I don't think this should cause any actual problems for anyone unless you're sending a constant stream of network messages right after connecting that absolutely must be received in the right order. If anyone does run into this though I am open to suggestions for a solution.

    @WaaghMan I see you. Brain is tired. I'll respond briefly now and try and get into some more detail when brain is more functional.

    I believe I just fixed this recently it probably just didn't make it into the last update. I just tested by setting the facilitator timeout to 0 so guid wouldn't be fetched and it seems to be working now at least.

    I should be able to address this more cleanly in the plugin with some auto-retry. Good find.

    Really awesome. I'll be back to digest this more.

    I see successful through NAT / Relay / Direct adds up to 26.52+17.47+14.82=58.81%

    So less than 60% of your users are successfully connecting!? Yeah, the timeouts and cancelled connections are definitely worrying. Assuming all those are potentially successful connections (which is probably a bit generous) the success rate would be up to 97% which is way more reasonable.
     
    Last edited: Mar 15, 2017
  43. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    Do you think it is likely that the timeouts could be related to the ID_CONNECTION_LOST error? That would be pretty cool since you've already worked around it and it seems simple enough to fix.
     
  44. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    I really hope this is the real problem causing the timeouts. The solution may be as simple as checking if (directClient != null || punchthroughClient != null || relayClient != null || natHelper.isPunchingThrough) before handling the error. If any client is still active or punchthrough is still being attempted the error should probably be ignored. Let me know if that works and I can probably bake it into the plugin so that OnClientError() is only called when appropriate.

    Though I will say...timeouts are handled by OnClientDisconnect not OnClientError so maaaybe none of that matters. Worth looking into though.

    P.S. OnClientError includes an error code that may be informative.
     
    Last edited: Mar 14, 2017
  45. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    Oh now this one is just sloppy. Sorry about that...fix will be in next update (or shoot me an email if you want it sooner).
     
  46. WaaghMan

    WaaghMan

    Joined:
    Jan 27, 2014
    Posts:
    245
    Thanks for your answers, I'll wait for the new version and continue using our patches meanwhile.

    Well, these numbers are from connection attempts, not unique users, so I guess an user having issues will retry a lot more than one that manages to connect successfuly on the first attempt, and thus the stats will be skewed. At least we don't get that many complaints on connection errors in the game forums, and if anything people tells it's become very stable.

    Not sure, I'm not posting enough data to our analytics to be able to tell, but I don't really think so. Most people is able to connect the facilitator and get a GUID, so I don't think connection to it will be lost that frequently. And even if that was the case, what I'm finding more surprising is the low success rate even for relay connections.
    I'm afraid it won't be that easy. After posting, I found out that connection errors are usually reported via Disconnect events, not Errors, which are properly handled by the plugin.

    I'm currently trying to detect Relay server disconnections, so we can reconnect or at least disable it for users joining the game. Wish me luck :)
     
  47. HappyBadgerStudio

    HappyBadgerStudio

    Joined:
    May 6, 2014
    Posts:
    19
    Hey everyone!

    First of all, loving the plugin, it is fantastic and a huge time saver.

    We are getting pretty close to having our game PC side pretty much good to go with the lobby system, just hitting a few snags. I can assume we are having these issues just because I'm not too sure on what I'm doing wrong (there is so much to take in!)

    I will just start with the current issue I've been trying to resolve, and it is just a disconnect from the host. (Client disconnect is working fine).

    We are using the NATLobbyManager (extending it into our own class) and Steam for the match making. Everything is working great for the most part, but when the host disconnects the clients never receive the message. I assume this is because the connection is dropped and clients are never told? What is the best way for a host to disconnect and alert the clients of it doing so?

    This is how I'm currently doing it.

    Code (CSharp):
    1.  
    2. public void Disconnect() {
    3.         if(NetworkServer.active) {
    4.             // event for letting other classes know the host is leaving
    5.             if(HostDisconnected != null) {
    6.                 HostDisconnected();
    7.             }
    8.             // This disconnects the host fine, but the client never receives the message
    9.             NetworkServer.SetAllClientsNotReady();
    10.             ((NATTraversal.NetworkManager)NATTraversal.NetworkManager.singleton).StopHost();
    11.  
    12.             // I was using the one below, but it throws a few errors
    13.             // it seems to reconnect after being called? perhaps due to host migration manager?
    14.             // NullReferenceException: Object reference not set to an instance of an object
    15.             // NATTraversal.ExternalNetworkConnection.Initialize(System.String networkAddress, Int32 netHostId, Int32 netConnId, UnityEngine.Networking.HostTopology hostTopology)
    16.             // HandleData Unknown connectionId:1
    17.             // NATTraversal.NetworkManager.singleton.StopHost();
    18.         } else {
    19.             // event for letting other classes know this client is leaving
    20.             if(ClientDisconnected != null) {
    21.                 ClientDisconnected();
    22.             }
    23.             // works best for clients
    24.             ((NATTraversal.NetworkManager)NATTraversal.NetworkManager.singleton).StopClient();
    25.             NetworkLog.singleton.LogMessage("Client Disconnected.");
    26.         }
    27.     }
    28.  
    Any guidance would be greatly appreciated :)

    If you need me to elaborate on anything please let me know, thank you for your time :D
     
  48. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    @HappyBadgerStudio It's hard to say for sure what could be going wrong. You're calling StopHost() correctly there.

    What version of Unity are you on? I just tested in 5.5 with a basic NATLobbyManager + NetworkManagerHUD example and disconnecting seemed to work as expected.

    Make sure that if you override any methods from NATTraversal.NetworkManager you call the base method. In most cases it's necessary to keep things from breaking in weird ways.
     
    HappyBadgerStudio likes this.
  49. HappyBadgerStudio

    HappyBadgerStudio

    Joined:
    May 6, 2014
    Posts:
    19
    Thanks for the reply!

    I am running version 5.5.2f1 (64 bit) in Standalone mode.

    I am overriding pretty every OnLobby (OnLobbyStartHost, etc) callback, but I am mostly just logging it out and calling the base method.

    The only other (non lobby) methods I am overriding are these.. but I am calling base on them. (mainly for event registering and stuff)
    OnServerConnect(NetworkConnection conn)
    OnServerDisconnect(NetworkConnection conn)
    OnClientConnect(NetworkConnection conn)
    OnClientDisconnect(NetworkConnection conn)

    Reading your comment, I made a test scene doing exactly that what you did, and I am getting the same results as you, it seems to be working fine.

    After more and more testing, I think I have it down to it being an issue with time, if I wait for ~15 seconds before trying to make the host disconnect, the client will destroy all instances of the lobby player in the scene.
    If I do it too shortly after the client joining the room, then it does not seem to receive the message and does not destroy anything!

    Is there any event that is getting called internally that maybe I can register to? It seems like OnLobby callbacks, and the OnClient callbacks (listed above) do not trigger on the client machine at all regardless of wait time. (or maybe they are an I am just missing it?)

    Thank you again for your time, I think I was just missing something. :)
     
  50. thegreatzebadiah

    thegreatzebadiah

    Joined:
    Nov 22, 2012
    Posts:
    836
    You may be suffering from a bug that I believe is fixed in the latest version that I just submitted to the store. If you shoot me an email at the support email I can get you a copy so you don't have to wait for it to show up on the store.

    Try overriding OnMultiClientDisconnect()