Search Unity

How to send large amounts of data on reliable fragmented sequenced channel using NetworkTransport

Discussion in 'Multiplayer' started by warrickct, Feb 14, 2018.

  1. warrickct

    warrickct

    Joined:
    Oct 26, 2017
    Posts:
    4
    I'm currently trying converting send certain model properties (verts, uvs, triangles) and eventually entire model data over a network when spawned locally and then recreate them on the clientside. Right now I'm currently trying to send over a serialized class with the 2 lists + 1 array. When serialized it's byte[] length is 1499351 which is over the sendable limit as far as I know. I get errors saying it's over the limit of ~65kb limit.

    I've considered compressing the model using MessagePack but I think there will be even bigger models needing to be sent in the future so that won't solve things properly.

    I'm currently doing this all using the reliable, fragmented, sequenced channel which I thought would break large sized byte arrays into fragments then send and reconstruct the data on the receiving side. Am I using this channel wrong?

    I also attempted to just implement my own fragmentation on the same channel by sending the array in 1024 sized portions but I can't think of an elegant way to receive it on the receiving side.

    So to summarise I have about 3 questions:
    - Does reliable, fragmented, sequenced handle byte arrays over the 65kb limit?
    - If yes then how would I go about sending the large array using NetworkTransport API
    - If no to the above question, what's could be a good way to set up custom fragmentation?
    - Is there another method for sending gameobjects/models that are only present on one client to another client that I'm missing completely?

     
  2. l3fty

    l3fty

    Joined:
    Mar 23, 2013
    Posts:
    87
  3. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    - Does reliable, fragmented, sequenced handle byte arrays over the 65kb limit? - no
    - If no to the above question, what's could be a good way to set up custom fragmentation? - use reliable sequenced qos (it's going like tcp stream) and fragment by yourself.

    for sending array you don't specially need to use fragmented channel:
    QOS: reliable sequenced:
    1. Send(arrayLength)
    2. for( int i =0; i < arrayLength; ++i)
    send(object);
     
  4. warrickct

    warrickct

    Joined:
    Oct 26, 2017
    Posts:
    4
    I think I tried something similar to what you've suggested but ran into a problem. I tried deserializing the model mesh vertices into a serializable class with 3 floats then serialized it into a large array and tried to send that across the network to deserialize later but it doesn't.

    Model serialization and sending function:
    Code (CSharp):
    1. Mesh modelMesh = model.GetComponent<MeshFilter>().mesh;
    2.  
    3. //converts into serializable object contianing list with 3 float coordinates.
    4. Vertices vertices = new Vertices(modelMesh.vertices);
    5.  
    6. //serializes the vertices object
    7. MemoryStream ms = new MemoryStream();
    8. BinaryFormatter bf = new BinaryFormatter();
    9. bf.Serialize(ms, vertices);
    10.  
    11. //converts to a usually large byte array[]
    12. byte[] data = ms.ToArray();
    13.  
    14. //send the array at cut off point every 1024 bytes
    15. for (int i=0; i < data.Length; i += 1024) {
    16.     byte[] chunk;
    17.     //Takes the remaining bytes if less than 1024 remaining
    18.     if (less than 1024 bytes remaining){
    19.         chunk =  take remaining bytes
    20.         break;
    21.     }
    22.     //take 1024 bytes
    23.     else {
    24.         chunk = data.Skip(i).Take(1024);
    25.     }
    26. }
    27. byte error;
    28. NetworkTransport.Send(hostId, connId, channId, chunk, chunk.Length, out error);
    29.  
    I noticed that even when the final byte is less than 1024 bytes[] in size the received buffer ends up being 1024 due to the receive function buffer being set to length 1024 which doesn't cause errors when receiving but I think it causes errors when serializing the data back into the triple float serializable object. I know the serializable object works because I've used it in the exact same way with regular UDP sending.
     
    Last edited: Feb 18, 2018
  5. warrickct

    warrickct

    Joined:
    Oct 26, 2017
    Posts:
    4
    Update: I have managed to send the large serializable class after fiddling with things.

    I think the problem was the received method uses a fixed receive buffer size (in my case 1024 bytes).

    When I tried to cast the combined buffer it would therefore be multiple of 1024 in size when the actual size is usually something else e.g (real object byte[] length when serialized: 1900bytes, received byte[] size: 2048. This in combination with the casting method I used:

    Code (csharp):
    1. MeshWireData meshWireData = (MeshWireData)binaryFormatter.Deserialize(memoryStream);
    Which is a more strict type conversion than the one I'm now using:

    Code (csharp):
    1. MeshWireData meshWireData = binaryFormatter.Deserialize(memoryStream) as MeshWireData
    So changing from the former to the latter seems to have fixed the problem as far as I can tell. There is the chance a vertex coordinate is off but I can't see it when I add the resulting meshfilter to a mesh renderer.