Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Sending Large Structs over Network

Discussion in 'Multiplayer' started by Grey_Cloak, Jun 14, 2019.

  1. Grey_Cloak

    Grey_Cloak

    Joined:
    Jun 16, 2014
    Posts:
    6
    Hello everyone, I'm having an issue with coming up with a good solution on how to (quickly) send a large struct or file via the network using C#. I've been working on this issue for the better part of a week and I've now become desperate enough to make a forum post.

    My first solution was to use the LLAPI for network connections and try sending the data through that method. After optimizing the code as much as I was able to, I was still only able to send a 5.8 MB struct within the span of around 24 seconds.
    That's about 242 KB/s, which is awfully slow for a local network connection (I was testing this by connecting to my own PC). I would like to be able to transfer structs at least as fast as 1 MB/s.

    After that attempt, I decided to rewrite my netcode in terms of using TcpListener and TcpClient. After taking a day or two to do that, I finished and the result is now taking 28 seconds to transfer a 5.8 MB struct. And it also seems to be blocking the main thread while doing so, so it certainly has not been an improvement.

    I'll post the code here for my (very simple) sending and receiving functions:

    Sending:
    Code (CSharp):
    1. private void Send(NetMsg msg)
    2.     {
    3.         if (!socketReady)
    4.             return;
    5.  
    6.         try {
    7.             BinaryFormatter formatter = new BinaryFormatter();
    8.          
    9.             formatter.Serialize(stream, msg);
    10.         }
    11.         catch (Exception e) {
    12.             Debug.Log("Write error : " + e.Message + " to server");
    13.         }
    14.     }
    Receiving:
    Code (CSharp):
    1. private void ReceiveData()
    2.     {
    3.         BinaryFormatter formatter = new BinaryFormatter();
    4.      
    5.         NetMsg receivedMsg = (NetMsg)formatter.Deserialize(stream);
    6.         HandleMessage(receivedMsg);
    7.     }
    If anyone has any suggestions as to how to increase the speed of the transfer, I'd very much appreciate it.
     
    Last edited: Jun 14, 2019
  2. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    You should show sample of data format, you try send over the network.

    Chances are, you have bloated data set, with unimportant informations. But you can look into data compression.

    However, why are you concerned, how long it takes to send 5 MB. It wont be real time game anyway, with such large data. At most streamed, or even only on start / load / save. Whatever you need.

    Also, don't expect everyone to have such speed of network.
     
  3. Grey_Cloak

    Grey_Cloak

    Joined:
    Jun 16, 2014
    Posts:
    6
    The struct I am sending is a representation of all the tiles in the world's tilemap. It is a map that is procedurally generated and the tiles can be altered during runtime, so it is necessary to store and send information about all of the tiles whenever a player joins the server. Every tile struct takes up approximately 94 B, and the world currently consists of a maximum of 62,500 tiles, hence the 5.8 MB size of the world structure that I am transferring.

    I am concerned about the speed because it is exceptionally slow considering that it is transferring over a LAN (file transfer over LAN is typically around 12 MB/s), and the fact that internet connections will be even slower is worrisome. I'm also only in the beginning stages of organizing the structures, so there's also the issue that this timespan will only become larger as more variables are added.

    Here's the code for the different structs involved, although I'm not sure it's particularly relevant since it's all just turned into bytes upon serialization anyway (which is a process that only takes around 1-2 seconds).

    Here's the message struct, which holds the SerializedTiles struct (the 5.8 MB struct):
    Code (CSharp):
    1. [System.Serializable]
    2. public class Net_WorldTiles : NetMsg
    3. {
    4.     public Net_WorldTiles()
    5.     {
    6.         OP = NetOP.WorldTiles;
    7.     }
    8.  
    9.     public long sendTime;
    10.     public World.SerializedTiles data;
    11. }
    Here's the SerializedTiles struct itself, which is just a multidimensional array of serialized metatiles:
    Code (CSharp):
    1.     [System.Serializable]
    2.     public class SerializedTiles
    3.     {
    4.         public MetaTile.Serialized[,,,] serializedMetaTileMap;
    5.  
    6.         public SerializedTiles(MetaTile.Serialized[,,,] _serializedMetaTileMap)
    7.         {
    8.             serializedMetaTileMap = _serializedMetaTileMap;
    9.         }
    10.     }
    And finally, the metatile struct itself:
    Code (CSharp):
    1. [System.Serializable]
    2.     public class Serialized
    3.     {
    4.         public int posX;
    5.         public int posY;
    6.         public int posZ;
    7.  
    8.         public int zoneX;
    9.         public int zoneY;
    10.  
    11.         public int xFlip;
    12.         public int yFlip;
    13.  
    14.         public int tileType;
    15.         public int variation;
    16.         public bool isPathable;
    17.         public float vegetation;
    18.  
    19.         public bool isTransitionTile;
    20.         public int[] quadrants;
    21.         public int transitionIndex;
    22.         public int transitionToType;
    23.  
    24.         public Serialized(int _posX, int _posY, int _posZ, int _zoneX, int _zoneY, int _xFlip, int _yFlip, int _tileType, int _variation, bool _isPathable, float _vegetation,
    25.             bool _isTransitionTile, int[] _quadrants, int _transitionIndex, int _transitionToType)
    26.         {
    27.             posX = _posX;
    28.             posY = _posY;
    29.             posZ = _posZ;
    30.  
    31.             zoneX = _zoneX;
    32.             zoneY = _zoneY;
    33.  
    34.             xFlip = _xFlip;
    35.             yFlip = _yFlip;
    36.  
    37.             tileType = _tileType;
    38.             variation = _variation;
    39.             isPathable = _isPathable;
    40.             vegetation = _vegetation;
    41.  
    42.             isTransitionTile = _isTransitionTile;
    43.             quadrants = _quadrants;
    44.             transitionIndex = _transitionIndex;
    45.             transitionToType = _transitionToType;
    46.         }
    47.     }
     
  4. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    Consider replace int for short int, float for halve, and byte, when applicable.
    Also, instead sending whole map, send seed. Just like many games does.
     
  5. Grey_Cloak

    Grey_Cloak

    Joined:
    Jun 16, 2014
    Posts:
    6
    Yes there are some small optimizations I could make to the variables to improve loading speed, but those would not have a particularly dramatic effect.

    And like I said, the tiles can be altered during runtime, so just sending the seed would not be enough to accurately generate the world in its current state. However, I could add a "dirty" bool to indicate which tiles have changed since the initial generation, which would greatly reduce the amount of data that needs to be transferred (except in the worst case scenario of every single tile being different since initial generation).

    However, I seem to have found another solution. I once again switched my netcode, this time to incorporate the Telepathy library (https://github.com/vis2k/Telepathy), and with changing the max packet size to 10 MB (using the MaxMessageSize variable) I am now loading the 5.8 MB struct within 6 seconds.
    That's 960 KB/s, which is a significant improvement. My deepest gratitude to vis2k who made this library and commented it out so thoroughly.
     
  6. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    I would say, you got major flow in your game design / network solution. Look how for example minecraft works.
    Or other tail based games. Or even RTS games, with thousands of units.

    You should get back to drawing board, and rethink you design.

    You only should send over the network changes, or sync data.
    Seed is to ensure that initial game state is the same for either client / server.
    And saves on network bandwidth. Client can generate map on their device.
     
  7. Grey_Cloak

    Grey_Cloak

    Joined:
    Jun 16, 2014
    Posts:
    6
    Hmm...you don't seem to understand what I've already said. Generating the map on the client's machine with a transferred seed won't be beneficial if the tiles have all been altered from the original procedural generation. At any rate, the current transfer speed is about equivalent to how long it takes to generate the map anyway.

    I did at least take your advice about turning the ints into shorts, though I have no idea how to change the float into a "half." That made a 5.8 MB struct into a 3.9 MB struct, which is a decent reduction. However, the transfer still took 6 seconds, oddly enough...which suggests that those 6 seconds might actually just be the serialization and de-serialization of the struct.
     
  8. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    1. How often you generate whole map?
    2. How often you change n number of tiles, and what % of whole map is each change contributes to?
    3. How often you need send tiles map update over the network?

    You can always convert also float to int, short, byte if you like. That may be more convenient. Just multiply by 10^n, to remove unwanted precision.
     
  9. Grey_Cloak

    Grey_Cloak

    Joined:
    Jun 16, 2014
    Posts:
    6
    1.) The whole map is only generated once.
    2.) The number of tiles changed will be dependent upon a variety of factors, like how long the host has played on the world, how the NPCs have affected the tiles, etc. I haven't progressed far enough in development to get any sort of hard statistics.
    3.) The entire tilemap will only need to be sent whenever a player joins the server. So it should not have a significant amount of impact on the performance, since my game is currently only meant to be played by a maximum of 5 simultaneous users.

    And yeah, I did convert the float to a byte actually, since I did not really need the precision of a float. I also converted all the shorts to bytes or sbytes since I probably won't need more than 256 values for each variable. That reduced the entire map size down to 2.8 MB, which is less than half the original size.
     
  10. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    Good stuff.

    Hence you can send seed to client, client will generate own exactly copy of base map.

    If players are currently online, you send only currently changed tails, and only these, which are in view range (depending how much player see of the whole map). This way keeping small footprint.

    See point one first. If players can join in middle of the game, then you send only changed tiles from beginning, to update, for which this information can be stored on server / authoritative client. Otherwise, see point 2.
     
    Munchy2007 likes this.