Search Unity

Resolved NetDbg tool doesn't work properly

Discussion in 'NetCode for ECS' started by optimise, Jun 28, 2021.

  1. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Hi @CMarastoni. When using NetDbg tool, I get the following errors and I see nothing is updated in NetCode 0.6.0-preview.7. Any idea why it like that?
    InvalidOperationException: GetSingleton<Unity.NetCode.NetworkSnapshotAckComponent>() requires that exactly one Unity.NetCode.NetworkSnapshotAckComponent exist that match this query, but there are 0.
    Unity.Entities.EntityQueryImpl.GetSingleton[T] () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/Iterators/EntityQuery.cs:863)
    Unity.Entities.EntityQuery.GetSingleton[T] () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/Iterators/EntityQuery.cs:1523)
    Unity.NetCode.GhostStatsCollectionSystem.OnUpdate () (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/Stats/GhostStatsCollectionSystem.cs:166)
    Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/SystemBase.cs:412)
    Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    UnityEngine.Debug:LogException(Exception)
    Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/Stubs/Unity/Debug.cs:19)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:477)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.NetCode.ClientSimulationSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/ClientServerWorld/ClientSimulationSystemGroup.cs:90)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ScriptBehaviourUpdateOrder.cs:333)

    IndexOutOfRangeException: Index 0 is out of range of '0' Length.
    Unity.Collections.NativeArray`1[T].FailOutOfRangeError (System.Int32 index) (at <127e81e1cb3441cc97d26b1910daae77>:0)
    Unity.Collections.NativeArray`1[T].CheckElementReadAccess (System.Int32 index) (at <127e81e1cb3441cc97d26b1910daae77>:0)
    Unity.Collections.NativeArray`1[T].get_Item (System.Int32 index) (at <127e81e1cb3441cc97d26b1910daae77>:0)
    Unity.NetCode.GhostStatsCollectionSystem.AddSnapshotStats (Unity.Collections.NativeArray`1[T] stats) (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/Stats/GhostStatsCollectionSystem.cs:98)
    Unity.NetCode.GhostSendSystem.UpdateNetStats (System.Int32& netStatSize, System.Int32& netStatStride) (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/Snapshot/GhostSendSystem.cs:2265)
    Unity.NetCode.GhostSendSystem.OnUpdate () (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/Snapshot/GhostSendSystem.cs:2279)
    Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/SystemBase.cs:412)
    Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    UnityEngine.Debug:LogException(Exception)
    Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/Stubs/Unity/Debug.cs:19)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:477)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.NetCode.ServerSimulationSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/ClientServerWorld/ServerSimulationSystemGroup.cs:75)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ScriptBehaviourUpdateOrder.cs:333)
     
    optimise, Jun 28, 2021
    #1

  2. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    mmm. The only think I can think of is a missing RequireForUpdate in the GhostStatsCollectionSystems for the first one.
    For the second one I would say, probably, if you try to connect the NetDbg tools and there are not ghost prefab setup? (this is just really guessing).
    Can give me more information about how are you running the NetDbg tool (like what project, etc)
     
    CMarastoni, Jun 28, 2021
    #2

  3. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @CMarastoni Alright u can ignore the previous errors I post. I change all the
    UNITY_EDITOR || DEVELOPMENT_BUILD defines at GhostCollectionSystem to NETCODE_DEVELOPMENT_BUILD since those debug logic makes the editor and development build too slow.

    Anyway now it just crash when NetDbg tool connects to editor. Any idea why? I believe the repro project at Case 1333309 can reproduce the bug. You can remove my custom Netcode package and restore it back to official Netcode0.6.0-preview.7. If still cannot repro the bug let me know.

    Unity.NetCode.GhostStatsCollectionSystem.BuildPacket () (at Library/PackageCache/com.unity.netcode@0.6.0-preview.7/Runtime/Stats/GhostStatsCollectionSystem.cs:254)
    Unity.NetCode.GhostStatsCollectionSystem.BeginCollection (System.UInt32 currentTick) (at Library/PackageCache/com.unity.netcode@0.6.0-preview.7/Runtime/Stats/GhostStatsCollectionSystem.cs:185)
    Unity.NetCode.GhostStatsCollectionSystem.AddSnapshotStats (Unity.Collections.NativeArray`1[T] stats) (at Library/PackageCache/com.unity.netcode@0.6.0-preview.7/Runtime/Stats/GhostStatsCollectionSystem.cs:96)
    Unity.NetCode.GhostReceiveSystem.OnUpdate () (at Library/PackageCache/com.unity.netcode@0.6.0-preview.7/Runtime/Snapshot/GhostReceiveSystem.cs:775)
    Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/SystemBase.cs:412)
    Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    UnityEngine.Debug:LogException(Exception)
    Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/Stubs/Unity/Debug.cs:19)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:477)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.NetCode.ClientSimulationSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.netcode@0.6.0-preview.7/Runtime/ClientServerWorld/ClientSimulationSystemGroup.cs:90)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:472)
    Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystemGroup.cs:417)
    Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ComponentSystem.cs:114)
    Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@bfeb4734bd/Unity.Entities/ScriptBehaviourUpdateOrder.cs:333)

    upload_2021-6-28_22-54-18.png
    upload_2021-6-28_22-55-7.png
     
    optimise, Jun 28, 2021
    #3

  4. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    I don't know now how your code looks like now.
    But given the fact that it crash inside the build packet and there is a null reference and a suspicious out of bound in a line like:
    Code (csharp):
    1.  
    2. for (int i = 0; i < m_PredictionErrors.Length; ++i)
    3. {
    4.        *(float*) (binaryData + binarySize) = m_PredictionErrors[i];  <------ ??? What going on here?
    5.        binarySize += 4;
    6. }
    7.  
    I would probably put a breakpoint and investigate a little. The pointer for the m_PredictionErrror is allocated inside the SetGhostNames, so it should be valid at this point.
     
    CMarastoni, Jun 29, 2021
    #4

  5. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Today I debug the issue and seems like the issue comes from here. Any idea?
    Unity.NetCode.DebugWebSocket.SendBinary (System.Byte[] msg, System.Int32 offset, System.Int32 len) (at Library/PackageCache/com.unity.netcode@9c36d57f37/Runtime/Stats/DebugWebSocket.cs:260)

    upload_2021-6-30_3-12-3.png
     
    Last edited: Jun 29, 2021
    optimise, Jun 29, 2021
    #5

  6. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    looks like the Len + Offset is larger than the msg size.
    So something wrong with the setup of the GhostStatsCollectionSystem packet pool or something similar.
    Is the UpdateMaxPacketSize and/or EnsurePoolSize being called ?
     
    CMarastoni, Jun 30, 2021
    #6

  7. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    UpdateMaxPacketSize method doesn't being called when the error spamming. It only being called 2 times when I enter play mode.
     
    optimise, Jun 30, 2021
    #7

  8. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Hi @CMarastoni. Have u find out the root cause of this issue?
     
    optimise, Jul 5, 2021
    #8

  9. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    Didn't look at that yet. But sounds like an issue with the EnsurePoolSize that does not reserve enough space since there are 14KB msg size but 30KB data
    Probably something you changed in the GhostCollectionSystems or other places. You also mentioned you changed EDITOR || DEVELOPMENT_BUILD => DEVELOPMENT_BUILD. Probably something does not get properly initialized anymore in the editor but only in the build for you.
    I would need the code to understand what it is wrong.
     
    CMarastoni, Jul 6, 2021
    #9

  10. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @CMarastoni I can reproduce the same error at public release NetCode 0.6.0-preview.7 without any custom modification.
     
    optimise, Jul 6, 2021
    #10

  11. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    My fault, I misunderstood the sentence
    I will check then using your project.
     
    CMarastoni, Jul 6, 2021
    #11

  12. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    900
    Looks a typo in the UpdateMaxPacketSize. The length ofd the PredictionErrors array is not multiplied by 4.
    This is the correct formula:

    Code (CSharp):
    1. 8 + 20 * 255 + 4 * m_SnapshotStats.Length + 4* m_PredictionErrors.Length + 4 * 255;
    And I think is not fixed in later versions, so may thanks for reporting. It was stomping memory at random.
     
    CMarastoni, Jul 6, 2021
    #12
    desertGhost_ and optimise like this.