out of sync: Standalone player over internet only

I have some hard to debug problems and while it might seem a bit abstract without specific code or the scene, I hope someone has an idea what might cause it.

The scene contains some networked objects. I am not spawning anything manually and let the network manager handle everything. On the host's end, everything is always how it should be, on the clients end however, I get inconsistent results:

Client uses the editor or the standalone player, local machine: Works fine
Client uses the editor, internet connection: Works fine
Client uses the standalone player, internet connection: This is what is not working. Some of the networked objects are either in incorrect positions or not spawned at all.

Incorrect position means it is either at the origin (0, 0, 0) or at the position of another networked object.
Because it only happens in the standalone player, I am having a hard time figuring out if the objects are not spawned, disabled, not visible or just at another position exactly. Also since it only appears when using an internet connection I assume the network delay is the reason for the inconsistencies, although it is weird that it doesn't happen if the client uses the editor (even over an internet connection).

The log of the clients player shows an out of sync exception. The network config missmatch also appears on client and server in every case, even if everything works fine, so I am not sure if it is connected to the problem.

Spoiler: output_log.txt

Initialize engine version: 5.5.0x1-CollabPreview (09b457573f85)

...


UnloadTime: 0.971923 ms
Setting up 1 worker threads for Enlighten.
Thread -> id: 1d4 -> priority: 1
Unloading 4 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 0.184320 ms

Unloading 4 unused Assets to reduce memory usage. Loaded Objects now: 331.
Total: 0.141019 ms (FindLiveObjects: 0.010240 ms CreateObjectMapping: 0.010825 ms MarkObjects: 0.115858 ms DeleteObjects: 0.004096 ms)

MatchMakingClient ListMatches :https://mm.unet.unity3d.com/json/reply/ListMatchRequest

(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

MatchMakingClient ListMatches :https://mm.unet.unity3d.com/json/reply/ListMatchRequest

(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

MatchMakingClient Join :https://mm.unet.unity3d.com/json/reply/JoinMatchRequest

(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

Network configuration mismatch detected. The number of networked scripts on the client does not match the number of networked scripts on the server. This could be caused by lazy loading of scripts on the client. This warning can be disabled by the checkbox in NetworkManager Script CRC Check.

(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

CRC Local Dump PlayerController : 0

(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

InvalidOperationException: out of sync
at System.Collections.Generic.Dictionary`2+Enumerator[System.String,System.Int32].VerifyState () [0x00000] in <filename unknown>:0
at System.Collections.Generic.Dictionary`2+Enumerator[System.String,System.Int32].MoveNext () [0x00000] in <filename unknown>:0
at System.Collections.Generic.Dictionary`2+KeyCollection+Enumerator[System.String,System.Int32].MoveNext () [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkCRC.Dump (UnityEngine.Networking.NetworkSystem.CRCMessageEntry[] remoteScripts) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkCRC.ValidateInternal (UnityEngine.Networking.NetworkSystem.CRCMessageEntry[] remoteScripts, Int32 numChannels) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkCRC.Validate (UnityEngine.Networking.NetworkSystem.CRCMessageEntry[] scripts, Int32 numChannels) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkClient.OnCRC (UnityEngine.Networking.NetworkMessage netMsg) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkConnection.HandleReader (UnityEngine.Networking.NetworkReader reader, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkConnection.HandleBytes (System.Byte[] buffer, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkConnection.TransportRecieve (System.Byte[] bytes, Int32 numBytes, Int32 channelId) [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkClient.Update () [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkClient.UpdateClients () [0x00000] in <filename unknown>:0
at UnityEngine.Networking.NetworkIdentity.UNetStaticUpdate () [0x00000] in <filename unknown>:0

(Filename: Line: -1)

Unloading 4 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 0.159744 ms
Setting up 2 worker threads for Enlighten.
Thread -> id: 3108 -> priority: 1
Thread -> id: 237c -> priority: 1

Unloading 5 unused Assets to reduce memory usage. Loaded Objects now: 1136.
Total: 0.418377 ms (FindLiveObjects: 0.043300 ms CreateObjectMapping: 0.020187 ms MarkObjects: 0.305152 ms DeleteObjects: 0.049737 ms)

I would appreciate any guesses as to why this is happening. Debugging is hard if the problem doesn't occur on your own machine..

 

One thing I've noticed is some crazy stuff can happen if both builds aren't the same (every time you want to test them together you need to rebuild so they're both up to date), most of the time stuff just doesn't spawn and collisions even happen in the wrong places. Are you sure that both the Editor and the standalone build are the same? Even if you make a tiny change that wouldn't seem like it would make a difference, it does.

 

Yes they are absolutely the same. In fact because it doesn't matter what the host uses, testing included hosting with the standalone player as well, so the host and client would both use the standalone player with the build created just seconds ago. This is not the issue

 

Hi, I just ended here because I have the exact same problem in the exact same context and I am still having a hard time trying to debug it. Have you found any solution ? Thanks by advance.