Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Sync texture over network using netcode

Discussion in 'Netcode for GameObjects' started by vickyyy0221, Dec 4, 2023.

  1. vickyyy0221

    vickyyy0221

    Joined:
    Oct 24, 2023
    Posts:
    5
    Hi all, I am a newer in netcode and currently struggling in sending/receiving Texture(1024*1024) over network for few days.
    I need to update the texture from local client to server when they first connected and share the new texture to each connected clients, I've tried by Rpc and custom messaging(seems this can send bigger package), but still get the same error even setting Max Payload Size to 500000:
    OverflowException: Writing past the end of the buffer

    Code (CSharp):
    1.  
    2. public void SendTexture()
    3. {
    4.    
    5.     Texture clientTex = gameObject.GetComponentInChildren<Renderer>().material.mainTexture;
    6.     byte[] clientTex2Bytes = ConvertTexture2DToBytes(matchShapesTest.TextureToTexture2D(clientTex));
    7.     Debug.Log($"client {NetworkManager.LocalClientId} spawn with tex2bytes length {clientTex2Bytes.Length}");
    8.     PrintByteArray(clientTex2Bytes);
    9.     var msgManager = NetworkManager.Singleton.CustomMessagingManager;
    10.     var writer = new FastBufferWriter(FastBufferWriter.GetWriteSize(clientTex2Bytes), Allocator.Temp);
    11.  
    12.     using (writer)
    13.     {
    14.         writer.WriteValueSafe(clientTex2Bytes);
    15.         msgManager.SendNamedMessage("UpateTexture", NetworkManager.ServerClientId, writer, NetworkDelivery.Reliable);
    16.     }
    17. }
    18.  
    19. public void UpdateTexture(ulong senderId, FastBufferReader payload)
    20. {
    21.     payload.ReadValueSafe(out byte[] bytes);
    22.     Debug.Log($"receive {bytes.Length} from {senderId}");
    23.     PrintByteArray(bytes);
    24.     Texture2D tex2d = ConvertBytesToTexture2D(bytes);
    25. }
    26.  
    One more question, the network object texture(default) that default using is showing different size on server and local client, why?
    (server)
    upload_2023-12-4_14-52-53.png
    (local client)
    upload_2023-12-4_14-53-39.png
    Is there any way that I can reduce the texture byte array or I just having wrong settings?
    Can anyone help?
     
  2. vickyyy0221

    vickyyy0221

    Joined:
    Oct 24, 2023
    Posts:
    5
    Oh, the error end by changing NetworkDelivery.Reliable to NetworkDelivery.ReliableFragmentedSequenced, but I still cannot see received output on server side.

    Code (CSharp):
    1. public override void OnNetworkSpawn()
    2. {
    3.     base.OnNetworkSpawn();
    4.     NetworkManager.Singleton.CustomMessagingManager.RegisterNamedMessageHandler("UpdateTexture", UpdateTexture);
    5. }
     
    Last edited: Dec 4, 2023
  3. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,117
    For one, it‘s best to send the original image. It‘llbe a png or jpg and thus much smaller than the texture bytes. Next, the texture bytes require a specific texture format but a client‘s hardware may not support that specific format. Again sending the original image‘s bytes is preferred as it provides you with all the options of loading the image and using it for various purposes.

    To send large chunks you could manually split the data into xx kb chunks with staggered ROC messages. Or use a file hosting service and only share the download url. A lot easier!
     
  4. RikuTheFuffs-U

    RikuTheFuffs-U

    Unity Technologies

    Joined:
    Feb 20, 2020
    Posts:
    437
    If all builds of the game are aware of the texture, you don't need to send it. Just send an ID, and use it to retrieve the texture from the local files.
     
    NoelStephens_Unity likes this.
  5. NoelStephens_Unity

    NoelStephens_Unity

    Unity Technologies

    Joined:
    Feb 12, 2022
    Posts:
    247
    What RikuTheFuffs-U points out, unless you are capturing a picture via webcam or the like it would be much easier to create an ID for the texture. You can actually look at how we generate the NetworkObjectIdHash value to get an idea of how you could go about doing this.
     
  6. vickyyy0221

    vickyyy0221

    Joined:
    Oct 24, 2023
    Posts:
    5
    Hi all, thanks for answering my question.
    Sorry for not explained my case clearly. Actually I am working on sync texture which is captured via device camera, every client should have their unique texture when they start connection.
    Currently, server can get the new texture when client connected, but I am struggling on how to synchronize to other existing and late join clients.
    Client update their own texture to server when they connected, server share new received texture to other clients (by custom messaging). Then I need a list to store and sync texture to each client (seems NetworkList is the only way to go).
    I am working on NetworkList with struct which contains clientId and texture byte array, but stuck on error below:
    Error CS8377 The type 'DisplayPattern.ClientPatternData' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'NetworkList<T>'
    upload_2023-12-5_11-19-51.png

    Code (CSharp):
    1. public NetworkList<ClientPatternData> connectClients = new NetworkList<ClientPatternData>();
    2.  
    3. public struct ClientPatternData
    4. {
    5.     public ulong clientId;
    6.     public byte[] textureBytes;
    7.     public bool isUpdated;
    8.    
    9.     public ClientPatternData(ulong id, byte[] bytes, bool isupdated)
    10.     {
    11.         clientId = id;
    12.         textureBytes = bytes;
    13.         isUpdated = isupdated;
    14.     }
    15.  
    16.     public void NetworkSerialize<T>(BufferSerializer<T> serializer) where T : IReaderWriter
    17.     {
    18.         serializer.SerializeValue(ref clientId);
    19.         serializer.SerializeValue(ref textureBytes);
    20.         serializer.SerializeValue(ref isUpdated);
    21.     }
    22. }
    Actually I am not sure if this is a good way to go, please give me some advice...qq
     
  7. vickyyy0221

    vickyyy0221

    Joined:
    Oct 24, 2023
    Posts:
    5
    Okay, I can now have the new texture on both sides.
    I have a local dictionary connectedClients to record income textures and another dictionary syncedClients to record texture update status. serverRpc call by client to change texture on gameobject when client received new updated texture from server, but still cannot sync texture on clients.
    Code (CSharp):
    1. private Dictionary<ulong, byte[]> connectedClients = new Dictionary<ulong, byte[]>();
    2. private Dictionary<ulong, bool> syncedClients = new Dictionary<ulong, bool>();
    upload_2023-12-5_19-8-17.png
    Code (CSharp):
    1. [ServerRpc(RequireOwnership = false)]
    2. public void SetTextureServerRpc(ServerRpcParams serverRpcParams = default)
    3. {
    4.     ulong senderId = serverRpcParams.Receive.SenderClientId;
    5.     Debug.Log("ServerRpc called by " + senderId);
    6.     if (NetworkManager.ConnectedClients.ContainsKey(senderId))
    7.     {
    8.         //sync texture for sender client
    9.         foreach(var syncClient in syncedClients)
    10.         {
    11.             //if not synced on client yet
    12.             if(syncClient.Value == false && NetworkManager.Singleton.ConnectedClients.TryGetValue(syncClient.Key, out NetworkClient clientObject))
    13.             {
    14.                 NetworkObject obj = clientObject.PlayerObject;
    15.                 if (connectedClients.TryGetValue(syncClient.Key, out byte[] receivedBytes))
    16.                 {
    17.                     //clientTexture.Value = receivedBytes;
    18.                     Debug.Log($"sync {syncClient.Key} owned object on {senderId}");
    19.                     Texture2D tex2d = ConvertBytesToTexture2D(receivedBytes);
    20.                     obj.GetComponentInChildren<Renderer>().material.mainTexture = tex2d;
    21.                     syncedClients[syncClient.Key] = true;
    22.                 }
    23.                 else
    24.                 {
    25.                     Debug.LogError($"{syncClient.Key} texture not found");
    26.                 }
    27.             }
    28.         }
    29.     }
    30.  
    31. }
    Actually I don't know how NetworkVariable work on my case...