Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

MatchMaker client timeout

Discussion in 'Multiplayer' started by Pelican_7, Jul 22, 2015.

  1. Pelican_7

    Pelican_7

    Joined:
    Nov 25, 2014
    Posts:
    190
    Hi all,

    I am having an issue when using MatchMaker whereby my client gets kicked off the server after approximately 1m30s every time due to a UNet Client Disconnect Error: Timeout. The host player continues to run. I do not have this issue when not using the matchmaker and joining the game using an IP address.

    Does this sound like something that could be misconfigured on my part? Has anyone else had a similar issue?

    Thanks in advance,
    Andy
     
    glitchers likes this.
  2. glitchers

    glitchers

    Joined:
    Apr 29, 2014
    Posts:
    64
    We are having the same issue with the timeout, 90 seconds after joining the lobby all the clients timeout and disconnect. All I can find in the settings is how many max players per game.
     
  3. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    It's likely given the timing you're being kicked because of the bandwidth limiter.

    When you use the Matchmaker/Relay it tracks used bandwidth per match, which equates to 4kb per second times the number of players in the match. There's no window involved, it's calculated from the match start. In fact, when you start you're given a pool of 2 minutes worth of free bandwidth to allow for spiky connections etc.

    From the behavior you're describing it sounds like you're a bit over the 4kb limit and end up disconnecting because of it. Currently we don't provide an error or tools to see this is the cause for the disconnect; that's something we're working on right now.

    For now i'd suggesting cutting back on on network traffic and see if you still encounter the disconnect after a time period, or if the disconnect time increases as bandwidth used decreases.
     
    Azurne likes this.
  4. Pelican_7

    Pelican_7

    Joined:
    Nov 25, 2014
    Posts:
    190
    Hi Jeremy,

    Thanks a lot for your help. That seems to be exactly the issue. I have greatly reduced the network traffic and do not get this disconnect until much later (~8 mins). I now know to work on optimising what we send over the network.

    Are there plans to provide a way to be able to see and debug the network bandwidth that the Matchmaker is tracking, and how close to the limiter you are? Or to expose the data so we can build a tool to visualise it? It would be really useful for us. :)

    Thanks,
    Andy
     
  5. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    The visualization for bandwidth use is exactly what we're working on now, though i don't have an ETA for when it will go live. We're planning on a web snap in where you'll be able to see in depth information about this and other performance metrics related to your games.
    For now some math would probably help out. Bandwidth is charged against each client it's delivered to, with 4k/sec and a 2 minute buffer. So that means after 8 minutes you've sent roughly 2400kb to the client and triggered a disconnection.
    To avoid that you should have sent under 1920kb in that time, so you need to cut your bandwidth use by 20%-25% or more.
     
    Azurne, zhangjin_ and glitchers like this.
  6. Pelican_7

    Pelican_7

    Joined:
    Nov 25, 2014
    Posts:
    190
    Great, that's very useful. Thanks for your help Jeremy.
     
    JeremyUnity likes this.
  7. sebeisdrache

    sebeisdrache

    Joined:
    Jan 16, 2015
    Posts:
    34
    Waht If I need a highter Bandwidth for my game?
     
  8. sglindme

    sglindme

    Joined:
    Jun 30, 2015
    Posts:
    18
    They seem to keep ignoring this question sebeisdrache. It's weird cause uLink doesn't seem to put such a tiny limit.
     
  9. Furious-Witch

    Furious-Witch

    Joined:
    Mar 1, 2015
    Posts:
    11
    Hi All,
    So I have the same issue. My client disconnects after about 15 seconds. I read your comments about the bandwidth so I open the profiler to try to get some info. It gives me the numbers of packets and messages in and out but how to use this to calculate the size of the data being sent to the client? Or is there another way to calculate it?

    Edit: I don't think this is a bandwidth issue. The log file doesn't say anything about timing out the last thing in the log about it is indicating success :

    JSON Response: [[UnityEngine.Networking.Match.JoinMatchResponse]-success:True-extendedInfo:]-address:52.28.24.148,port:9999,networkId:0x00100000000435AD,nodeId:0x6D76,usingRelay:True
    UnityEngine.Networking.Match.<ProcessMatchResponse>c__Iterator0`1:MoveNext() (at C:\buildslave\unity\build\Runtime\Networking\Managed\MatchMakingClient.cs:290)

    And also I am only syncing one Vector3 variable, nothing more. How could that eat the bandwidth?
     
    Last edited: Jan 20, 2016
  10. Lofar42

    Lofar42

    Joined:
    Oct 11, 2014
    Posts:
    12
    Hey,

    I'm having the same issue - A disconnect after around 15 to 90 seconds.

    What kind of options may be available to us for this should we want to go over the limit?

    @JeremyUnity , @seanr

    That said, Furious Witch seems to be syncing one single vector and getting this issue so I'm wondering if something else may be involved
     
  11. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    @Furious Witch - That log indicated the matchmaker succeeded in allowing you to join (the success:True part). The client timeout after 15 seconds in the match means something happened talking to Relay server.
    You could check the profiler for some stats related to network traffic like packets out. Though it doesn't give an exact bandwidth amount you can see if you're sending something more frequently than expected.

    @LaisyLofar - We don't have any options to raise the CCU or bandwidth limit while we're in beta, and that's one of the reasons we're in beta. That said, we're working on exactly that right now.
     
  12. Furious-Witch

    Furious-Witch

    Joined:
    Mar 1, 2015
    Posts:
    11
    @JeremyUnity Thanks for replying. I used the profiler and it helped me some. I understood the number of packets sent + received affect how early the disconnection happens. So now the client stays connected for about 1 to 1.5 min. But I don't know how to decrease the data sent any more than it is. I am almost certain I have not reached the bandwidth cap of 4 Kb/s per client. This is what I did so far to decrease data sent:

    - Decreased packet size default to 1000 (I don't if this should help)
    - Decreased Network.sendRate to 5 (I read it should help but it didn't or may be very little)
    - I now sync the Vector3 through a function that is called 4 times per second instead of the Fixed Update function. This was the most helpful change but the problem is the player movement looks lagging and stuttering and still the game disconnects although not as early. Even when I call the function 2 times per seconds it disconnects.
    - Depending on the previous point packets number decreased from 50 to 6
    - I also noticed that pausing the game sometimes make the connection drop instantly after resume or a little while after resuming.

    So what do you think this is?

    @LaisyLofar The above points certainly did not solve my problem but at least they made it less severe (not by much :) )
    I hope may be they could help solve your problem a little. At least until we figure something out.
     
  13. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    @Furious Witch I'm glad that improved things. The fact that it lengthened the time you stay connected means this is almost certainly errant traffic being sent, the question really is what and why.

    Let's do some quick math. assuming you have a 2 node match with 1 server and 1 client and the client joins roughly as soon as the server started the room to exceed the bandwidth you'd have to use the following amount of traffic:

    15 second disconnect: 1080k total (72k/second)
    1 minute disconnect: 1440k total (24k/second)
    1.5 minute disconnect: 1680k total (18.66k/second)

    So that means just by reducing the sync rate of the vector 3 you cut your traffic by between 70% and 79%! Good find.

    My guess is something else is still being sent that is contributing to the excess bandwidth. In general anything network related doesn't belong in fixed update so that's a good thing to change, but it's possible there's still something left somewhere synchronizing too often. I'd probably try not synchronizing the vector3 at all as a test, and if it still disconnects that's not the problem (alone that is).

    Something else that's possible is retransmits contributing to the issue if the network loss is really high and you're sending over a reliable channel, just something to check.
     
    Furious-Witch likes this.
  14. Furious-Witch

    Furious-Witch

    Joined:
    Mar 1, 2015
    Posts:
    11
    @JeremyUnity I commented the Vector3 Syncing part and the game didn't disconnect. At the beginning of client connection there was the same little burst of activity in the profiler and afterwards the profiler was totally empty.

    As for the channel, thanks very much for pointing that out. Although it didn't solve the issue totally, but after changing both channels to unreliable (the top channel was already unreliable) now things are much more stable and I can sync the Vector3 variable five times per second without losing the connection (anything more and the connection is lost). The motion is still jerky but at least its more stable. The problem is the game is supposed to have many clients and this is only one :)

    So the question is since the channel is probably the cause of the problem probably because other traffic is being sent through that doesn't appear in the profiler so is there a channel type that will be more convenient than the unreliable type? I already read about the different types but it appeared to me that unreliable is the best for players movements. But with this result, I really hope something else is better.

    Thanks again very much for the help :)
     
    Last edited: Jan 21, 2016
  15. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    @Furious Witch You'll need to interpolate positions on the client, it's unavoidable generally. Some games synchronize 20 hz, some as low as 6-10. Even at 20, if the client is running at 30 or 60 fps you'll notice a smoothness issue without interpolation.

    I'm still a little curious about the overall bandwidth use here. A vector 3 payload should look roughly like this on the wire for size (Assuming no gathering happens to reduce the traffic before it's sent):

    Worst case (1 message per packet) if the packet is unreliable:
    8 bytes UDP header + 20 bytes IPv4 header + 10 bytes UNet headers + 12 bytes Vector3 = 50 bytes each vector update

    Worst case (1 message per packet) if the packet is fragmented reliable:
    8 bytes UDP header + 20 bytes IPv4 header + 22 bytes UNet headers + 12 bytes Vector3 = 62 bytes each vector update

    And that means the per second cost is:

    Unreliable:
    50 bytes * 20hz = 1000 bytes per second
    50 bytes * 5hz = 250 bytes per second

    Reliable (excluding retransmits):
    62 bytes * 20hz = 1240 bytes per second
    62 bytes * 5hz = 310 bytes per second

    So something here doesn't quite add up still. Even when synchronizing at 20hz you should be well within the bandwidth limit in your test.

    In any event the bandwidth limit does scale with number of clients connected, so as you add clients each will be able to digest 4k down from the server.

    As for reliable vs unreliable, yes reliable is not always optimal for movement because even if you drop packets you cant really rewind time to insert missing positions if you would have already used them, it's best to just handle the most recent movement packet as it comes in and wait for the next newer one to arrive.

    If you're always sending out the packets and not checking for their dirty state i'd recommend sending unreliable. If you DO check for position differences however you'll need to use reliable. This is because if you don't send packets out because nothing has changed, and the most recent movement packet you sent was dropped, the object's position will be incorrect on the remote client since it never received that last position. In effect if the code is only sending when changes occur and the network layer is not resending dropped packets you run the risk of object being at the wrong position permanently (or until another change occurs and a packet is successfully received), if that makes sense.
     
    bboysil likes this.
  16. Furious-Witch

    Furious-Witch

    Joined:
    Mar 1, 2015
    Posts:
    11
    @JeremyUnity I know interpolation is essential and its at the top of my priority list of things to learn and do. Probably my next step after insuring stable connection with a little reasonable sendrate :)

    So as for the bandwidth (thanks a lot for the detailed and patient explanation), it looks like I'm barely using a slice of my portion of bandwidth. I have no idea what is behind the issue so I'm sharing my syncing code. Its very minimal but if there is anything I'm doing wrong I'll be thankful if you find it. So here is my code :

    public class Player_SyncPosition extends NetworkBehaviour
    {
    @SyncVar
    private var syncPos : Vector3;
    var myTransform : Transform;
    var lerpRate : float = 15;

    function Start()
    {
    InvokeRepeating("TestTimeOut", 2, 0.2);
    }

    function TestTimeOut()
    {
    TransmitPos();
    LerpPosition();
    }

    function LerpPosition()
    {
    if(!isLocalPlayer)
    {
    myTransform.position = Vector3.Lerp(myTransform.position, syncPos, lerpRate*Time.deltaTime);
    }
    }

    @Command
    function CmdPosToServer(pos : Vector3)
    {
    syncPos = pos;
    }

    @ClientCallback
    function TransmitPos()
    {
    if(isLocalPlayer)
    {
    CmdPosToServer(myTransform.position);
    }
    }
    }

    Both my channels type are Unreliable and I already tried commenting the syncing part before to make sure it is the only thing being sent which it is. If there is no problem in the code may be I can send the whole project if it is not much trouble.

    Thanks again I really appreciate your help.
     
    Last edited: Jan 22, 2016
  17. JeremyUnity

    JeremyUnity

    Joined:
    Mar 4, 2014
    Posts:
    147
    @Furious Witch I don't see anything obviously wrong but needing to use InvokeRepeating() shouldn't be necessary. The docs we have at on CommandAttribute specifically show updating a position (granted 2d, but the idea is the same) via a command attribute. You should only need to add a syncvar to this example to make what you're going for here. The important thing there is to see how Update and FixedUpdate are used differently.
     
  18. Lofar42

    Lofar42

    Joined:
    Oct 11, 2014
    Posts:
    12
    @JeremyUnity Thanks for the reply. Turns out I was being totally over the top with my data synchronization. I've managed to get it down by a massive amount and am no longer having any bandwidth issues. A blessing in disguise really, this whole thing - So thanks.
     
    JeremyUnity likes this.
  19. Furious-Witch

    Furious-Witch

    Joined:
    Mar 1, 2015
    Posts:
    11
    @JeremyUnity Sorry for the late reply. I did like the example in the docs but since its using the update function things got bad like they were before. Only stays for about 20-30 seconds before disconnection. So I uploaded the project it contains both my version of syncing (Player_SyncPosition) and the version from the docs (Player_SyncPosition2). I also made a new prefab for the docs example (Player2). The project is already on the docs example when you run it.

    https://www.mediafire.com/?s2caapy7rfro574

    The only thing I added is now I generate cubes randomly and sync their position only at the beginning and when collision detected. Already tested with my version that they don't affect the connection much since they are synced on occasion.

    Another important point is when I tried to connect four players using my version of syncing 5 times per second to test if things will get worse. The connection dropped on all clients in about one minute. I know this shouldn't happen since every client should get extra 4k/s bandwidth and since it worked for two it should work for the rest but again things are strange.

    I hope the project will show to you what it doesn't to me and thank you very much for the help, greatly appreciated. :)
     
  20. scottlaforge

    scottlaforge

    Joined:
    Mar 12, 2013
    Posts:
    42
    Hi @JeremyUnity! Any updates on providing more bandwidth? Since much in this post, Unity now has LiveMode. If accepted to LiveMode, can we increase that bandwidth, and no longer encounter these timeout issues? We have a training simulation going out to a client in just over a month and are looking to solve this issue, especially for extensive testing. Thanks in advance for any help!
     
  21. scottlaforge

    scottlaforge

    Joined:
    Mar 12, 2013
    Posts:
    42
    In case it's useful, the magic number in the profiler is between 4300 and 4400 when the timeout seems to occur, every single time.
    upload_2016-12-29_16-20-5.png
     
  22. scottlaforge

    scottlaforge

    Joined:
    Mar 12, 2013
    Posts:
    42
    In case anyone is interested, it turned out our issue was VOIP services eating up all of our data. After removing it, the application has been running solid for over 50 minutes. Has anyone else experienced issues with UNet VOIP services?
     
  23. pavan123

    pavan123

    Joined:
    Nov 5, 2016
    Posts:
    4
    Hi all since couple of days i am facing one issue that is client connection keeps breakdown from server and no longer available beyond 1min and throws an error
    ServerDisconnected due to error: Timeout
    UnityEngine.Networking.NetworkManager:OnServerDisconnect(NetworkConnection)could anyone tell me why i am getting this error and how to figureout
     
  24. akil-ajax

    akil-ajax

    Joined:
    Feb 3, 2016
    Posts:
    1
    Yup same here. Cant able to connect on matchmaking server. Timeout errors.
     
  25. Anisoropos

    Anisoropos

    Joined:
    Jul 30, 2012
    Posts:
    102
  26. pophl

    pophl

    Joined:
    Mar 3, 2017
    Posts:
    14
    Please can you confirm that the limit is still 4kb per second times the number of players in the match and that there is 2 minutes worth of free bandwidth.

    Is this in the Dev environment and the Live environment or does that have other rules?
     
  27. pophl

    pophl

    Joined:
    Mar 3, 2017
    Posts:
    14
    Also, is it still not possible to identify this as the cause of a "Timeout" error, and is there no query function to determine remaining bandwidth (if for example we want to scaleback of network data when approaching the kick limit)
     
    Anisoropos likes this.
  28. Anisoropos

    Anisoropos

    Joined:
    Jul 30, 2012
    Posts:
    102
    That would be great actually, and I don't see why Unity wouldn't want us to have that info.

    For now you could have your own counter - I mean, you know the byte cost of each basic data structure and you could calculate accordingly complex ones. Not pretty, but if you really need it it shouldn't take more than a few hours to put together.
     
  29. amynox

    amynox

    Joined:
    Oct 23, 2016
    Posts:
    177
    @JeremyUnity
    I'm starting facing the same problem just after updating to Unity 5.6 !

    I tryied to tweak the connectionConfig but still having ServerDisconneted (i guess droped by the MatchMaker) !

    Here is my tweak since updating to Unity 5.6 :

    Code (CSharp):
    1.         myNetworkManager.customConfig = true;
    2.         myNetworkManager.connectionConfig.NetworkDropThreshold = 45;
    3.         myNetworkManager.connectionConfig.OverflowDropThreshold = 45;
    4.        
    5.         myNetworkManager.connectionConfig.AckDelay = 200;
    6.         myNetworkManager.connectionConfig.AcksType = ConnectionAcksType.Acks96;
    7.         myNetworkManager.connectionConfig.MaxSentMessageQueueSize = 256;
    Any help will be much appreciated
     
  30. abgamers

    abgamers

    Joined:
    Dec 22, 2009
    Posts:
    97
    Hi,
    Im trying to build a VOIP using Unet and im facing the same problem. it will be great if you could share some of the knowledge with us. it will be of great help.

    Thanks :)
     
  31. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    You have probably not "gone live" yet. That means, that you have limited amount of CCU and a limited amount of bandwidth. By using VOIP over the relay. You are killing your bandwith limit. Try to move the VOIP off the relay.
     
  32. abgamers

    abgamers

    Joined:
    Dec 22, 2009
    Posts:
    97
    In unity multiple dashboard I have made it to live mode, but my app is still in development.
     
  33. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Has this fixed the issue? If not, you prolly have to move your VOIP off the relay.
     
  34. abgamers

    abgamers

    Joined:
    Dec 22, 2009
    Posts:
    97
    This was never the issue as the app was always in the live mode
     
  35. Deleted User

    Deleted User

    Guest

    Hey guys, I'm having the same issue as everyone keeps mentioning. The client times out about 15 seconds into the game every time. I have a bunch of UI panels as children of the NetworkLobbyManager, Which obviously doesn't get destroyed on load. Would these UI panels have any impact on the Network Bandwidth?
     
  36. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Are they sending data?