Search Unity

Bug RPC FixedList512Bytes<int> 0 on receive

Discussion in 'NetCode for ECS' started by Hines94, Aug 4, 2022.

  1. Hines94

    Hines94

    Joined:
    Feb 15, 2020
    Posts:
    19
    Hi,

    I have been experimenting with FixedList512Bytes<int> for sending a list of info in an RPC. However, when I receive my request the length is correct but the values are always 0.

    For example:

    Code (CSharp):
    1.    //A request from our client to move some units somewhere
    2.     public struct MovementRequestRPC : IRpcCommand
    3.     {
    4.         public float3 DesiredLocation;
    5.         //Note this has a hard limit of 127 entities! Stored as GhostComponent ghostid!
    6.         public FixedList512Bytes<int> SelectedEntites;
    7.     }
    8.  
    When breakpoint on send RPC, values are correct, when breakpoint receive RPC they are all 0.
     
  2. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    FixedList512Byte is not a supported serialized type. We support FixedString but not FixedList.
    You need to create a template that handle the fixed list and register it the code-gen system by using the
    UserDefinedTemplates.
    Read the docs manual, there is section in that regards. Or look at the Asteroids sample too.

    In practice, create a class that implement the UserDefinedTemplates.RegisterTemplates partial method.

    Code (csharp):
    1.  
    2. public static partial class UserDefinedTemplates
    3. {
    4.     static partial void RegisterTemplates(List<TypeRegistryEntry> templates, string defaultRootPath)
    5.     {
    6.                 templates.Add(new TypeRegistryEntry
    7.                 {
    8.                     Type = "Unity.Collections.FixedList512",
    9.                     Quantized = false,
    10.                     Smoothing = SmoothingAction.Clamp,
    11.                     SupportCommand = true,
    12.                     Composite = false,
    13.                     Template = $"{TemplatesPath}DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs"
    14.                     TemplateOverride = "Path/To/MyTemplate.FixedList512.cs"
    15.                }
    16. }
    17.  
    And in your template file, you should write some fragments to handle the list. The DataStream don't support writing and reading the fixed list but you can overcome the limitation.

    The template looks like
    Code (csharp):
    1.  
    2. namespace Generated
    3. {
    4.     public struct GhostSnapshotData
    5.     {
    6.         struct Snapshot
    7.         {
    8.             #region __GHOST_FIELD__
    9.             public FixedList512 __GHOST_FIELD_NAME__;
    10.             #endregion
    11.         }
    12.  
    13.         public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    14.         {
    15.             #region __COMMAND_WRITE__
    16.             //The writer does not support WriteFixedList etc etc..
    17.             writer.WriteShort(data.__COMMAND_FIELD_NAME__.Length);
    18.             //brute force the serialiation would looks like or use the unsafe version and serialize all the bytes you need
    19.             for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    20.                 writer.WriteInt(data[i]);
    21.             #endregion
    22.         }
    23.  
    24.         public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    25.         {
    26.             #region __COMMAND_READ__
    27.             data.__COMMAND_FIELD_NAME__.Lengh = reader.ReadShort();
    28.             for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    29.                 data[i] = reader.ReadInt();
    30.             #endregion
    31.         }
    32.     }
    33. }
    34.  
    This would only work for FixedList512 when used for RPC. If you need to have this type into ghost components or commands (discouraged in both case) you need add other fragments.
     
  3. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    Another even simpler solution is to write a custom serializer for that RPC (that is probably fitting best your use case).

    Something like
    Code (csharp):
    1.  
    2. [BurstCompile]
    3. public struct MovementRequestRPC : IComponentData, IRpcCommandSerializer<MovementRequestRPC>
    4. {
    5.     public float3 DesiredLocation;
    6.     //Note this has a hard limit of 127 entities! Stored as GhostComponent ghostid!
    7.     public FixedList512Bytes<int> SelectedEntites;
    8.  
    9.  
    10.     public void Serialize(ref DataStreamWriter writer, in RpcSerializerState state, in RpcLevelLoaded data)
    11.     {
    12.         writer.WriteFloat(DesiredLocation.x);
    13.         writer.WriteFloat(DesiredLocation.y);
    14.         writer.WriteFloat(DesiredLocation.z);
    15.         writer.WriteShort(SelectedEntities.Length);
    16.         for(int i=0;i<SelectedEntities.Length;++i)
    17.             writer.WriteInt(SelectedEntities[i]);
    18.     }
    19.  
    20.     public void Deserialize(ref DataStreamReader reader, in RpcDeserializerState state, ref RpcLevelLoaded data)
    21.     {
    22.         DesiredLocation.x = writer.ReadFloat();
    23.         DesiredLocation.y = writer.ReadFloat();
    24.         DesiredLocation.z = writer.ReadFloat();
    25.         SelectedEntities.Length = writer.ReadShort();
    26.         for(int i=0;i<SelectedEntities.Length;++i)
    27.             SelectedEntities[i] = writer.ReadInt();
    28.     }
    29.  
    30.     [BurstCompile(DisableDirectCall = true)]
    31.     [MonoPInvokeCallback(typeof(RpcExecutor.ExecuteDelegate))]
    32.     private static void InvokeExecute(ref RpcExecutor.Parameters parameters)
    33.     {
    34.         RpcExecutor.ExecuteCreateRequestComponent<MovementRequestRPC, MovementRequestRPC>(ref parameters);
    35.     }
    36.  
    37.     static readonly PortableFunctionPointer<RpcExecutor.ExecuteDelegate> InvokeExecuteFunctionPointer = new PortableFunctionPointer<RpcExecutor.ExecuteDelegate>(InvokeExecute);
    38.     public PortableFunctionPointer<RpcExecutor.ExecuteDelegate> CompileExecute()
    39.     {
    40.         return InvokeExecuteFunctionPointer;
    41.     }
    42. }
    43.  
    44. class MovementRequestRPCCommandRequestSystem : RpcCommandRequestSystem<MovementRequestRPC, MovementRequestRPC>
    45. {
    46.     [BurstCompile]
    47.     protected struct SendRpc : IJobEntityBatch
    48.     {
    49.         public SendRpcData data;
    50.         public void Execute(ArchetypeChunk chunk, int orderIndex)
    51.         {
    52.             data.Execute(chunk, orderIndex);
    53.         }
    54.     }
    55.     protected override void OnUpdate()
    56.     {
    57.         var sendJob = new SendRpc{data = InitJobData()};
    58.         ScheduleJobData(sendJob);
    59.     }
    60. }
    61. }
    62.  
    Check also the RPC documentation for more details
     
    Last edited: Aug 13, 2022
  4. Hines94

    Hines94

    Joined:
    Feb 15, 2020
    Posts:
    19
    Thanks for your reply. I ended up using the fixed list 512 byte with the int param as I will require this for a number of different RPC's (movement/attack etc).

    I spent a significant amount of time getting this working but got there in the end. The main issue was that my template was not working and seemed to do nothing (not picked up by the autogenerator) despite running ForceCodeGeneration. I still have no idea why it happened but at some point it seems that it just got picked up (I don't think I even changed anything!)

    My UserTemplates file:

    namespace Unity.NetCode.Generators
    {
    public static partial class UserDefinedTemplates
    {
    public const string TemplatesPath = "Packages/com.unity.netcode/Editor/Templates/";
    static string OverridesPath = "Assets/Scripts/NetCodeGen/Templates";

    static partial void RegisterTemplates(List<TypeRegistryEntry> templates, string defaultRootPath)
    {
    templates.Add(new TypeRegistryEntry
    {
    Type = "Unity.Collections.FixedList512Bytes",
    Quantized = false,
    Smoothing = SmoothingAction.Clamp,
    SupportCommand = true,
    Composite = false,
    Template = $"{TemplatesPath}DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs",
    TemplateOverride = $"{OverridesPath}/IntFixedList512.cs"
    //TemplateOverride = ""
    });
    }
    }
    }

    It has the Assembly Reference too, just as in the samples.


    My Template

    namespace Generated
    {
    public struct GhostSnapshotData
    {
    struct Snapshot
    {
    #region __GHOST_FIELD__
    public FixedList512 __GHOST_FIELD_NAME__;
    #endregion
    }
    public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    {
    #region __COMMAND_WRITE__
    //The writer does not support WriteFixedList etc etc..
    writer.WriteInt(data.__COMMAND_FIELD_NAME__.Length);
    //brute force the serialiation would looks like or use the unsafe version and serialize all the bytes you need
    for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    writer.WriteInt(data.__COMMAND_FIELD_NAME__[i]);
    #endregion
    }
    public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    {
    #region __COMMAND_READ__
    data.__COMMAND_FIELD_NAME__.Length = reader.ReadInt();
    for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    data.__COMMAND_FIELD_NAME__[i] = reader.ReadInt();
    #endregion
    }
    }
    }

     
  5. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    Hey,

    mmm.. the template registration looks incorrect.
    You should register the concrete type:

    Code (csharp):
    1.  
    2.             templates.Add(new TypeRegistryEntry
    3.             {
    4.                 Type = "Unity.Collections.FixedList512Bytes<int>",
    5.                 ...
    6.  
    If you don't do that, the type cannot find a match. This is probably why is was not working (you may have changed this?)

    It is great that you get the template working. This is always good to know how that thing work, in case you need to add any other custom type. +1000 Kodus!

    However, while this work in your specific use case, I would suggest to override also the following regions (eventually just empty)
    __GHOST_WRITE__
    __GHOST_READ__
    __COMMAND_WRITE_PACKED__
    __COMMAND_READ_PACKED__

    The reason being, the template you wrote only work for RPC: if this list is added to a command or a replicated component, then thing will break (because it wants to serialize a FixedString32).

    I will use this occasion to talk a bit more about this use case (override template or adding template for new types).
    This may be useful to anyone that would read this tread (just in case).

    The template you written, CAN ONLY WORK FOR RPC and it is not overriding all the required fragments.
    So, while it work for you, it is by no mean a "best practice" to write a partial template for a type in this way.

    When you make a template override that uses a base template (in this case the FixedString32) and you are changing the snapshot type you are using (in this case is a FixedList512<int> instead of FixedString32) you need to check how the base template implements the various fragments and overrides all the ones that are not compatible with the new type. In particular, for this one:
    __GHOST_WRITE__
    __GHOST_READ__
    __COMMAND_WRITE__
    __COMMAND_READ__
    __COMMAND_WRITE_PACKED__
    __COMMAND_READ_PACKED__

    Also, in general, as soon as you register a type to code-gen, some assumptions are made:

    1- it is supposed to work for type that are present in replicated component. The following regions must be present (even if empty):
    __GHOST_WRITE__
    __GHOST_READ__
    __GHOST_PREDICT__
    __GHOST_COPY_TO_SNAPSHOT__
    __GHOST_COPY_FROM_SNAPSHOT__
    __GHOST_RESTORE_FROM_BACKUP__
    __GHOST_CALCULATE_CHANGE_MASK__
    __GHOST_CALCULATE_CHANGE_MASK_ZERO__
    __GHOST_REPORT_PREDICTION_ERROR__
    __GHOST_GET_PREDICTION_ERROR_NAME__

    2- if the template SupportCommand is true, the following regions must also be present:
    __COMMAND_WRITE__
    __COMMAND_READ__
    __COMMAND_WRITE_PACKED__
    __COMMAND_READ_PACKED__
     
  6. Hines94

    Hines94

    Joined:
    Feb 15, 2020
    Posts:
    19
    I changed the user defined template to your suggestion:

    Type = "Unity.Collections.FixedList512Bytes<int>",


    I think there may be a bug in the code generation.... It is extremely flaky! Sometimes it works and picks up my user changes but at other times it will not generate no matter how many times I force generation!
     
  7. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    If you add a template you must force code-generation. If you change the template, you must force a code-generation.
    Template are not detected by Unity compilation pipeline, so it is required to invoke the force-generation every time you make a change.

    In any event, can you please report a bug for that? So we can track it.
    Please add as much details as you can about your workflow or what you were doing.
    If you can add you template and/or mini-project would be also great.
     
  8. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,761
    I don't really want to start a new thread just for this but on a related note, I found it a bit weird 'double' isn't a natively supported serialized type. I've added my own TypeRegistryEntry with a quantized supported version, it just seemed a bit odd not to be supported by default.
     
  9. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    I agree, that was a missing basic type we should support.

    I think we can also try to add support for generic types like FixedList<> or in general other fixed size container of us. But it is not on the road map yet.

    For double and other types you guys tend to use and you are seeing we are missing some support please continue to report.
     
  10. FaithlessOne

    FaithlessOne

    Joined:
    Jun 19, 2017
    Posts:
    320
    I would appreciate native support for FixedList<> by Netcode for Entities for serializing RPC commands and synchronizing ghosts.

    I tried making a FixedList code generation template like described in this thread, but failed, because the template was never picked up by the code generation itself. Don't know why. Tried various things mentioned here but had no luck. Also it seems in version 1.0.0 of Netcode a few things changed regarding the templating which may be the reason.

    As a workaround I now use the FixedStringXyzBytes types for transferring list data as part of RPC commands.
     
  11. Occuros

    Occuros

    Joined:
    Sep 4, 2018
    Posts:
    300
    I would also like to add my +1 to have FixedLists supported in RPCs.


    Until that is the case, it would be great if the manual would indicate which types are and aren't supported for rpc's.
     
  12. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
  13. wechat_os_Qy01ogVe2VXok7r9tzZwtkL2U

    wechat_os_Qy01ogVe2VXok7r9tzZwtkL2U

    Joined:
    Jul 28, 2023
    Posts:
    2
    using System.Collections.Generic;

    namespace Unity.NetCode.Generators
    {
    public static partial class UserDefinedTemplates
    {
    public const string TemplatesPath = "Library/PackageCache/com.unity.netcode@1.0.11/Editor/Templates/DefaultTypes/";
    static partial void RegisterTemplates(List<TypeRegistryEntry> templates, string defaultRootPath)
    {
    templates.AddRange(new[]{
    new TypeRegistryEntry
    {
    Type = "Unity.Collections.FixedList512Bytes<int>",
    Quantized = false,
    Smoothing = SmoothingAction.Clamp,
    SupportCommand = true,
    Composite = false,
    Template = $"{TemplatesPath}GhostSnapshotValueFixedString32Bytes.cs",
    TemplateOverride = "Packages/com.meta.framework/NetCodeGen/Templates/FixedList512.NetCodeSourceGenerator.additionalfile",
    },
    }) ;
    }
    }
    }
    #templateid: Custom.FixedList512
    namespace Generated
    {
    public struct GhostSnapshotData
    {
    struct Snapshot
    {
    #region __GHOST_FIELD__
    public FixedList512Bytes<int> __GHOST_FIELD_NAME__;
    #endregion
    }

    public void Serialize(ref Snapshot snapshot, ref Snapshot baseline, ref DataStreamWriter writer, ref StreamCompressionModel compressionModel, uint changeMask)
    {
    #region __GHOST_WRITE__

    #endregion
    }

    public void Deserialize(ref Snapshot snapshot, ref Snapshot baseline, ref DataStreamReader reader, ref StreamCompressionModel compressionModel, uint changeMask)
    {
    #region __GHOST_READ__

    #endregion
    }

    public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    {
    #region __COMMAND_WRITE__
    writer.WriteShort(data.__COMMAND_FIELD_NAME__.Length);
    for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    writer.WriteInt(data.__COMMAND_FIELD_NAME__);
    #endregion
    #region __COMMAND_WRITE_PACKED__

    #endregion
    }

    public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
    {
    #region __COMMAND_READ__
    data.__COMMAND_FIELD_NAME__.Lengh = reader.ReadShort();
    for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
    data.__COMMAND_FIELD_NAME__ = reader.ReadInt();
    #endregion
    #region __COMMAND_READ_PACKED__

    #endregion
    }
    }
    }
    this is my template and register, but it didn't work, and has an error " C:\projects\dots\Packages\com.unity.netcode\Runtime\SourceGenerators\Source~\NetCodeSourceGenerator\Generators\TemplateFileProvider.cs(81,1): error NetCode: NetCode AdditionalFile 'NetCodeGen/Templates/FixedList512.NetCodeSourceGenerator.additionalfile' (named 'Custom.FixedList512') is a valid Template, but it cannot be matched with any UserDefinedTemplate (probably a typo). Known user templates:[Unity.Collections.FixedList512Bytes<int>[Library/PackageCache/com.unity.netcode@1.0.11/Editor/Templates/DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs]]." I don't know how to fix it
     
  14. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Is that FixedList<> RPC will be supported at next 1.1 exp release?
     
  15. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    895
    No it is not.
     
  16. wechat_os_Qy01ogVe2VXok7r9tzZwtkL2U

    wechat_os_Qy01ogVe2VXok7r9tzZwtkL2U

    Joined:
    Jul 28, 2023
    Posts:
    2
    My netcode version is 1.2.0-pre.6, and i write my .additionalfile and UserDefinedTemplates correctly according to project NetcodeSamples, but unity Editor has error : error NetCode: NetCode AdditionalFile 'E:/GOODS/Unity/YIEntity/Assets/NetCodeGen/Templates/Rotation2d.NetCodeSourceGenerator.additionalfile' (named 'Custom.Translation2d') is a valid Template, but it cannot be matched with any UserDefinedTemplate (probably a typo). Known user templates:[]. I don't know why, and I would appreciate it if you can give some advice.
     
  17. NikiWalker

    NikiWalker

    Unity Technologies

    Joined:
    May 18, 2021
    Posts:
    316
    This is saying it cannot find any UserDefinedTemplates, implying this step was not performed correctly. Can you double check your .asmref?