Question GameServer/Client : connection has been disconnected by timeout

Hello,

We have some timeout happening kinda randomly while playing 1v1.
We are using Unity2020.2.2f1 and the Master Server Kit package (https://assetstore.unity.com/packages/tools/network/master-server-kit-71604#description), which uses Unity Multiplayer HLAPI 1.0.8, along with Kubernetes/Agones to spawn game servers in the cloud.

The logs (a bit obfuscated to remove our IPs) are as follow :

Log Game Server:

Code (CSharp):

1. INFO,[05/17/2022 09:46:35][INFO] OnGameServerRegistrationRequested Userid : 2
2. INFO,"[05/17/2022 09:46:36][INFO] [MSK.BaseServer.OnPlayerInfoReceived] Player not found, creating"
3. INFO,[05/17/2022 09:46:36][INFO] AddPlayer 2 - username: Client1
4. INFO,[05/17/2022 09:46:36][INFO] [MSK.BaseServer.AddPlayer] adding player: Client1
5.  
6. INFO,"Log: connection {1} has been disconnected by timeout; address {::ffff:ZoneServerAddress:58802} time {266004}, last rec time {259969} rtt {55} timeout {6000}"
7. INFO,[05/17/2022 09:46:41][WARN] OnServerDisconnect initiated by . Last error : Timeout.
8. INFO,"Empty player list given to NetworkServer.Destroy(), nothing to do."
9. INFO,ServerDisconnected due to error: Timeout
10. INFO,[05/17/2022 09:46:41][INFO] RemovePlayer by conn id: 1
11. INFO,[05/17/2022 09:46:41][WARN] [MSK.BaseServer.RemovePlayer] player not found in list for id: 1
12.  
13. INFO,[05/17/2022 09:46:51][INFO] Received AwaitingPlayerJoin msg - connectionId 6
14. INFO,"[05/17/2022 09:46:51][INFO] [MSK.BaseServer.OnPlayerInfoReceived] Player not found, creating"
15. INFO,[05/17/2022 09:46:51][INFO] AddPlayer 1 - username: Client2
16. INFO,[05/17/2022 09:46:51][INFO] [MSK.BaseServer.AddPlayer] adding player: Client2
17.  
18. INFO,[05/17/2022 09:46:52][INFO] Game started ! (from CheckForGameStarted coroutine)
19.  
20. INFO,[ServerSide - OnChosenManaPool] Msg sent from player NetworkInstanceId 3
21. INFO,"[ClearManaPool] User chose:0 - ManaPool 0: 0, 0, 2, 0, 0, 0 / ManaPool 1: 0, 0, 1, 0, 0, 0"
22.  
23. INFO,"Log: connection {1} has been disconnected by timeout; address {::ffff:Client1Address:59289} time {298680}, last rec time {292659} rtt {53} timeout {6000}"
24. INFO,Warning OnServerDisconnect initiated by . Last error : Timeout.
25. INFO,[05/17/2022 09:47:14][WARN] OnServerDisconnect initiated by . Last error : Timeout.
26. INFO,ServerDisconnected due to error: Timeout
27. INFO,[05/17/2022 09:47:14][INFO] RemovePlayer by conn id: 1
28. INFO,[05/17/2022 09:47:14][INFO] [MSK.BaseServer.RemovePlayer] player remove: Client1
29.  
30. INFO,[05/17/2022 09:47:14][INFO] Opponent disconnected.
31.  
32. INFO,"Log: connection {2} has been disconnected by timeout; address {::ffff:Client2Address:52099} time {298860}, last rec time {292794} rtt {31} timeout {6000}"
33. INFO,Warning OnServerDisconnect initiated by . Last error : Timeout.
34. INFO,[05/17/2022 09:47:14][WARN] OnServerDisconnect initiated by . Last error : Timeout.
35. INFO,ServerDisconnected due to error: Timeout
36. INFO,[05/17/2022 09:47:14][INFO] RemovePlayer by conn id: 2
37. INFO,[05/17/2022 09:47:14][INFO] [MSK.BaseServer.RemovePlayer] player remove: Client2
38.  
39. INFO,Verbose Closing empty game server
40. INFO,[05/17/2022 09:47:14][INFO] Closing empty game server

Log Client1:

Code (CSharp):

1. Warning: Attempt to send to not connected connection {1}
2. Error: Failed to send internal buffer channel:0 bytesToSend:4
3. Error: Send Error: WrongConnection channel:0 bytesToSend:4
4. Error: ClientDisconnected due to error: Timeout

Log Client2:

Code (CSharp):

1. Error: ClientDisconnected due to error: Timeout

From what I've gathered, the ZoneServer first spawns the GameServer, then gets disconnected because it has nothing left to do with it once it is launched as a standalone.
Then, the player who requested the GameServer gets connected (Client1).
Then the other player joins the GameServer (Client2).
Then, the game is started.
Then, Client1 does an action which sends a message to the GameServer and at that point, the timeout happens which triggers a serie of timeout on both clients and server side, which crashes the entire game.

We are kinda clueless as to where we could dig to get more info / how we could fix this, but if you have any idea, feel free to share.

Thanks.

 

I have the same issue. Did you resolve it?

 

I ended up switching to Mirage to get rid of UNet's black box (knowing what your library's code does is great) AND we updated our GKE version (there was a weird issue with Agones Fleet autoscalers that affected hostPort on the version we used, see : https://github.com/containernetworking/plugins/pull/705, but it got fixed in 1.22.9-gke.1500+ and 1.23.6-gke.1700+).

Also, if you are having problems related to Agones, I greatly encourage you to go on their slack (link available on their website), the devs over there were really helpful!