Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[NetCode] GhostCollection not working in iOS builds

Discussion in 'NetCode for ECS' started by talila, Sep 11, 2020.

  1. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    Hi,
    There seems to be an issue in NetCode 0.3 when built for iOS, that causes Exceptions in the GhostCollectionSystem (exception logs can be found below). Exceptions are thrown as soon as client tries to connect to server. A connection seems to be established for a moment, but is closed again right after.

    I discovered the issue in a custom project, but could reproduce it in the official NetCode samples (tried with the NetCube sample scene). The issue did not occur on Netcode 0.2, and everything works fine on macOS builds. It doesn't work regardless of if Burst compilation is enabled or disabled for both client or server build.

    Is this a known issue?


    Project: NetCode 0.3 NetCube sample scene, only modification made was changing the connection IP address to make the client connect to the server in a local network
    Unity version: 2020.1.4f1
    XCode version: 11.7
    iOS version: 13.7

    Setup: Netcode client running on device, server running in Unity editor (same issue when server is running as standalone macOS build).



    Exception logs in XCode for client

    InvalidOperationException: Unexpected 0 hash for type
    at Unity.NetCode.GhostCollectionSystem.HashGhostComponentSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State& comp) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.GhostCollectionSystem.AddSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State state) [0x00000] in <00000000000000000000000000000000>:0
    at Asteroids.Mixed.Generated.GhostComponentSerializerRegistrationSystem.OnCreate () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemBase.CreateInstance (Unity.Entities.World world) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.AddSystem_OnCreate_Internal (Unity.Entities.ComponentSystemBase system) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.GetOrCreateSystemsAndLogException (System.Collections.Generic.IEnumerable`1[T] types, System.Int32 typesCount) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.CreateClientWorld (Unity.Entities.World defaultWorld, System.String name, Unity.Entities.World worldToUse) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at NetCodeBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.DefaultWorldInitialization.Initialize (System.String defaultWorldName, System.Boolean editorWorld) [0x00000] in <00000000000000000000000000000000>:0
    Unity.Entities.World:GetOrCreateSystemsAndLogException(IEnumerable`1, Int32)
    Unity.NetCode.ClientServerBootstrap:CreateClientWorld(World, String, World)
    Unity.NetCode.ClientServerBootstrap:Initialize(String)
    NetCodeBootstrap:Initialize(String)
    Unity.Entities.DefaultWorldInitialization:Initialize(String, Boolean)
    (Filename: currently not available on il2cpp Line: -1)

    InvalidOperationException: Unexpected 0 hash for type
    at Unity.NetCode.GhostCollectionSystem.HashGhostComponentSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State& comp) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.GhostCollectionSystem.AddSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State state) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.Generated.GhostComponentSerializerRegistrationSystem.OnCreate () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemBase.CreateInstance (Unity.Entities.World world) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.AddSystem_OnCreate_Internal (Unity.Entities.ComponentSystemBase system) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.GetOrCreateSystemsAndLogException (System.Collections.Generic.IEnumerable`1[T] types, System.Int32 typesCount) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.CreateClientWorld (Unity.Entities.World defaultWorld, System.String name, Unity.Entities.World worldToUse) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at NetCodeBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.DefaultWorldInitialization.Initialize (System.String defaultWorldName, System.Boolean editorWorld) [0x00000] in <00000000000000000000000000000000>:0
    Unity.Entities.World:GetOrCreateSystemsAndLogException(IEnumerable`1, Int32)
    Unity.NetCode.ClientServerBootstrap:CreateClientWorld(World, String, World)
    Unity.NetCode.ClientServerBootstrap:Initialize(String)
    NetCodeBootstrap:Initialize(String)
    Unity.Entities.DefaultWorldInitialization:Initialize(String, Boolean)
    (Filename: currently not available on il2cpp Line: -1)

    InvalidOperationException: Unexpected 0 hash for type
    at Unity.NetCode.GhostCollectionSystem.HashGhostComponentSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State& comp) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.GhostCollectionSystem.AddSerializer (Unity.NetCode.LowLevel.Unsafe.GhostComponentSerializer+State state) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Transforms.Generated.GhostComponentSerializerRegistrationSystem.OnCreate () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemBase.CreateInstance (Unity.Entities.World world) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.AddSystem_OnCreate_Internal (Unity.Entities.ComponentSystemBase system) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.World.GetOrCreateSystemsAndLogException (System.Collections.Generic.IEnumerable`1[T] types, System.Int32 typesCount) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.CreateClientWorld (Unity.Entities.World defaultWorld, System.String name, Unity.Entities.World worldToUse) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientServerBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at NetCodeBootstrap.Initialize (System.String defaultWorldName) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.DefaultWorldInitialization.Initialize (System.String defaultWorldName, System.Boolean editorWorld) [0x00000] in <00000000000000000000000000000000>:0
    Unity.Entities.World:GetOrCreateSystemsAndLogException(IEnumerable`1, Int32)
    Unity.NetCode.ClientServerBootstrap:CreateClientWorld(World, String, World)
    Unity.NetCode.ClientServerBootstrap:Initialize(String)
    NetCodeBootstrap:Initialize(String)
    Unity.Entities.DefaultWorldInitialization:Initialize(String, Boolean)
    (Filename: currently not available on il2cpp Line: -1)

    ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
    Parameter name: index
    at System.ThrowHelper.ThrowArgumentOutOfRangeException (System.ExceptionArgument argument, System.ExceptionResource resource) [0x00000] in <00000000000000000000000000000000>:0
    at System.ThrowHelper.ThrowArgumentOutOfRangeException () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.GhostStatsCollectionSystem.SetGhostNames (System.String[] nameList, System.Collections.Generic.List`1[T] errorList) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.GhostCollectionSystem.OnUpdate () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystem.Update () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemGroup.UpdateAllSystems () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.NetCode.ClientSimulationSystemGroup.OnUpdate () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystem.Update () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemGroup.UpdateAllSystems () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystem.Update () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystemGroup.UpdateAllSystems () [0x00000] in <00000000000000000000000000000000>:0
    at Unity.Entities.ComponentSystem.Update () [0x00000] in <00000000000000000000000000000000>:0
    at UnityEngine.LowLevel.PlayerLoopSystem+UpdateFunction.Invoke () [0x00000] in <00000000000000000000000000000000>:0
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems()
    Unity.NetCode.ClientSimulationSystemGroup:OnUpdate()
    Unity.Entities.ComponentSystem:Update()
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems()
    Unity.Entities.ComponentSystem:Update()
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems()
    Unity.Entities.ComponentSystem:Update()
    UnityEngine.LowLevel.UpdateFunction:Invoke()
    (Filename: currently not available on il2cpp Line: -1)



    Error log in Unity Editor which is running the server only (This is probably just a followup issue, when client tries to send RPC to start ingame stream but server has no client connection)

    RpcSystem received bad protocol version from connection 0
    UnityEngine.Debug:LogError(Object)
    Unity.NetCode.RpcErrorReportingJob:Execute() (at Library/PackageCache/com.unity.netcode@0.3.0-preview.3/Runtime/Rpc/RpcSystem.cs:348)
    Unity.Jobs.JobStruct`1:Execute(RpcErrorReportingJob&, IntPtr, IntPtr, JobRanges&, Int32) (at /Users/builduser/buildslave/unity/build/Runtime/Jobs/Managed/IJob.cs:30)
     
  2. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    This is a bug with initialization order of statics when building with il2cpp, we have a fix for which should make it to the next release.
     
  3. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    Great news, thank you!
     
  4. Extrys

    Extrys

    Joined:
    Oct 25, 2017
    Posts:
    343
    Any new on this?
     
    cdm456852 likes this.
  5. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    I'd also be curious. Since the new release does not include the fix I was wondering about the ETA for this?
     
  6. anihilnine

    anihilnine

    Joined:
    Jan 29, 2016
    Posts:
    27
  7. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    Any update on this? We're unable to build on device for almost 4 month now. Or is there some kind of known workaround to be able to build NetCode projects with il2cpp?
     
  8. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The fix was in netcode 0.5, are you saying you're still seeing the same error with that version? I have not seen any issues building with il2cpp lately.
     
  9. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    I actually did not try since it was not in the changelog. Wanted to wait until update with the fix arrived so I would not have to deal with potential code adaptions caused by an update twice. Thank for the info, will try right away
     
  10. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    The issue is fixed, unfortunately I'm running into more issues.

    First I need to build as Development Build, if I don't, some generated NetCode code will cause exceptions in the BurstCompiler (although I disabled burst compilation in the editor).

    Then when trying to connect from iOS client to a server, it fails with the server logging "RpcSystem received bad protocol version from connection 0". I noticed that the client (iOS build) had 0 in the ComponentCollectionVersion field of the NetworkProtocolVersion component. I could fix this with a workaround, sending the ComponentCollectionVersion from server to client via a network broadcast and writing it into the NetworkProtocolVersion component.

    I'm now having more issues now with ECS exceptions, but still need to look into it, if the error is on my end. It all worked in the editor though.

    With that said I wonder, would you consider NetCode usable in mobile builds right now? Of course I'm aware that is a very early preview package, with all consequences that come with it and obviously no guarantee that things will be working. But I'm just curious if NetCode is tested on iOS (or generally, in IL2CPP builds) in the current state of development?
     
    AdamBebko and cdm456852 like this.
  11. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    I'm getting immediate netcode disconnects from IOS with no errors being shown. It works fine in editor on separate machines. This is on Netcode 0.5.0

    I also previously received bad protocol errors, but it seemed that restarting unity and then rebuilding the clients fixed those.

    I get the server receiving the connection, and it can send the client an RPC right then, but then after a frame or two, the server detects a NetworkStreamDisconnected component, and the entities for that connection are gone. The client app keeps running, but is not connected. Again, this doesn't occur in editor both locally (client and server playmode) and also with an editor running a client from the same project on another machine. Also works fine with a standalone OSX build running either client or server or both.

    I wasn't having this error on 0.2.0, although my project has changed a lot since then, so it's possible I broke something. I upgraded directly to 0.5.0, skipping the other versions.

    I noticed that even on the server the ComponentCollectionVersion is always 0, even after a connection. Is this what might be causing the issue?

    Edit: One thing I've changed since 0.2.0 is that I'm using FixedStrings and FixedArrays, and I also send some data along in my GoInGameRequest. I don't know if that affects anything
     
    Last edited: Dec 16, 2020
  12. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    After fixing everything else, I now have the exact same issue as you describe.
     
  13. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    Since I was also not sure if it was due to something I was doing in my code, I tried it with the NetCube example, and the same issue occurs. The client loses connection to the server after a couple of frames, without any errors or apparent reasons.
     
  14. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
  15. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    This should only happen if you do not have any serializers in the built project.

    If that is the case my first guess would be there is a problem with stripping. Can you open `Edit > Project Settings...`, go to `Player`, check `Managed Stripping Level` in the `Other settings` section and make sure it is set to `Disabled` to see if that fixes the issue?
     
  16. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    You can't set it to disabled for IOS

    Edit: Not sure it's relevant but during build to IOS, Xcode opens before the build is complete and starts indexing. Then unity's popup shows that it's "Generating Netcode", Then several seconds later (maybe 20-40 seconds or so) the build actually finishes. Could it be possible that it's not adding these to the Xcode project or something?
     
    Last edited: Dec 17, 2020
  17. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    @AdamBebko I was wondering if you made any progress on this?
    I actually managed to run a NetCode client on an iOS device and successfully connect to a server running in the Unity editor for the first time, building a minimum example with only one moving object. But in my actual project, I keep having the same issue of the device seemingly connecting successfully to lose the connection after a couple of frames again.

    @timjohansson
    As AdamBebko alraid said, there is no Disabled option for iOS, only Low, Medium and High. I also tried to uncheck the "Strip Engine Code" option, but it does not change anything.
     
    AdamBebko likes this.
  18. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    I now also noticed that the ComponentCollectionVersion is always 0, on server and on client, in editor and in builds, even in my minimal test project. I'm on NetCode version 0.5, and updated to Unity 2020.2.1f. Even though it is 0, the connection in the minimal project from iOS to a server running in the editor works and is stable, so the disconnection issue doesn't seem to be connected to that.

    I'm trying to figure out what the difference is between my minimal test project and my actual project that causes the different behaviour. If I do I will post it here, but I'd appreciate any pointers.
     
    AdamBebko likes this.
  19. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    haven’t had any success yet at all.
     
  20. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    @timjohansson @AdamBebko

    After digging deeper I found the cause for the disconnect, which seems to be a bug in the NetCode package.

    The disconnect reason in NetworkStreamDisconnected was BadProtocolVersion. After I got to print ErrorLogs on the iOS client, I got the following error:

    GhostReceiveSystem incompatible ghost version 11209181739440114952. Server: 10653054741150880776

    So I started looking for the reason of the deviating version numbers. There was one particular ghost in my case for which the hashes differed between server and client. I logged the entire version number hashing process and found the reason to be the PredictionOwnerOffset field of the GhostTypeState.

    So it appears that this is different because the order of the component NativeArray passed to the AddComponents method in GhostCollectionSystem is different between client and server. When ghostType.SnapshotSize is assigned to the ghostType.PredictionOwnerOffset in the AddComponents method, depending on the order in the components array, the SnapshotSize field has a different value between client and server at that point because the order of the components was different.

    My first naive attempt for a provisional fix was to just exclude PredictionOwnerOffset from the hash calculation. Now the connection is not immediately terminated with BadProtocolVersion anymore as expected, but I still get no correct data transport between client and server, so looks like I need to keep digging...
     
  21. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    Great detective work! Hopefully we get some official help soon. Keep me posted if you find a workaround. You’ve definitely exceeded my debugging ability :) but let me know if there’s anything I can do to help with info gathering.
     
  22. talila

    talila

    Joined:
    Mar 6, 2015
    Posts:
    26
    Got it, finally!

    In OnUpdate method in GhostAuthoringComponent.cs, the components are fetched from the DstEntityManager in this line:

    var components = DstEntityManager.GetComponentTypes(entity);


    The order is differs between iOS client and in editor, for one specific ghost in my case. I simply sorted the components array by a reliable value that is consistent between builds and platforms. The line to add right after it is

    components.Sort(new ComponentTypeComparer());


    As comparer I use this:

    Code (CSharp):
    1.  
    2.         class ComponentTypeComparer : IComparer<ComponentType>
    3.         {
    4.             public Int32 Compare(ComponentType c1, ComponentType c2)
    5.             {
    6.                 return TypeManager.GetTypeInfo(c1.TypeIndex).StableTypeHash
    7.                     .CompareTo(TypeManager.GetTypeInfo(c2.TypeIndex).StableTypeHash);
    8.             }
    9.         }
    10.        
    Maybe I could sort by something simpler, but this works.

    And btw, if someone who comes across this doesn't know how to alter the package code. Embed it into the project by copying the package from PackageChache in your Library folder to the Packages folder, and rename it to just "com.unity.netcode" (remove the version number at the end of the folder name).

    @AdamBebko Crossing my fingers the fix works for you as well.

    @timjohansson Could the fix be integrated into the next NetCode release? And could you verify that sorting the array at this point will not brake something else? So far everything seems to work fine.
     
  23. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164

    @talila My Hero!:p:p

    Your instructions worked perfectly! And I can connect fine now. At long last! @timjohansson perhaps we could get this as an update to the main package?
     
  24. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    @talila @timjohansson

    I've run into a problem that I suspect is related. When I build as a server (check box server build on build settings window), I'm getting a bad RPC version error. I thought this was something to do with the actual build, but after further testing, it happens in the editor too when the server build checkmark is on. I can turn it off, and it connects fine, but as soon as that checkbox is on, it won't connect.

    perhaps this is related to the fix above?

    Edit: I suspect this might be my fault. I'm trying to adjust quantization of one of the ghosts, and I suspect that it's an issue with the assembly definition not going to server land properly.
     
    Last edited: Jan 27, 2021
  25. AdamBebko

    AdamBebko

    Joined:
    Apr 8, 2016
    Posts:
    164
    Just want to see whether there are any updates on this issue upcoming? It's annoying to have to keep editing these lines to get things working in different versions of my projects
     
  26. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    882
    I think that fix is going to land in the next version of NetCode (0.8)
     
    AdamBebko likes this.
  27. ChrisPie

    ChrisPie

    Joined:
    Mar 5, 2015
    Posts:
    31
    Will that even release this year, or are you guys waiting for 2022.1? We haven't had a release since January.
     
  28. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    882
    We don't have any updates or ETA about that yet. My apologies.
     
  29. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,113
    @CMarastoni Is that NetCode 0.7 has been skipped and targeting NetCode 0.8?
     
  30. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    882
    Unfortunately I don't have any update in that regard yet. We are still working hard on the next release. Lots of good stuff and changes are coming but I don't have an ETA right now.
     
  31. MrEastwood

    MrEastwood

    Joined:
    Dec 9, 2013
    Posts:
    19
    I'm on netcode 0.6.0 preview 7 now and it looks like this code made it into GhostCollectionSystem.cs. The only thing is I still get
    RpcSystem received bad protocol version from connection 0
    when connecting an android build to a windows editor or other platforms connecting to each other (I'm trying OSX and Windows both in editor and builds and Android builds. Some combinations work, some give the error).

    I noticed that the ghost collection still had different orders despite being sorted on their type hash. On closer inspection i had two different Translation components that could switch places. Since they have the same type this sort would consider them interchangable.

    I changed the ComponentHashComparer at the beginning of GhostCollectionSystem.cs to this:
    Code (CSharp):
    1.         struct ComponentHashComparer : IComparer<GhostComponentSerializer.State>
    2.         {
    3.             public int Compare(GhostComponentSerializer.State x, GhostComponentSerializer.State y)
    4.             {
    5.                 var hashX = TypeManager.GetTypeInfo(x.ComponentType.TypeIndex).StableTypeHash;
    6.                 var hashY = TypeManager.GetTypeInfo(y.ComponentType.TypeIndex).StableTypeHash;
    7.              
    8.                 if (hashX < hashY)
    9.                     return -1;
    10.                 if (hashX > hashY)
    11.                     return 1;
    12.                 hashX = x.SerializerHash;
    13.                 hashY = y.SerializerHash;
    14.                 if (hashX < hashY)
    15.                     return -1;
    16.                 if (hashX > hashY)
    17.                     return 1;
    18.                 return 0;
    19.             }
    20.         }
    This fixes the rpc bad protocol version error for me. I should note connections still don't always work. I noticed my android build can't connect to the windows editor being a server, but for some strange reason the windows editor will connect as a client to the exact same android build being a server. Weird, but good enough for me as I can continue my work now :)