BinaryFormatter vs. Encoding.GetBytes

Which workflow is better to use upon receiving a DataEvent?
the UNET documentation examples convert string messages to a byte array using BinaryFormatter()

Code (CSharp):

1. byte[] buffer = new byte[1024];
2. Stream message = new MemoryStream(buffer);
3. BinaryFormatter formatter = new BinaryFormatter();
4. formatter.Serialize(message , "test");

However, I have been using this...

Code (CSharp):

1. byte[] buffer = Encoding.Unicode.GetBytes("test");

the ``buffer`` used in Encoding seems more compact, only 8 bytes, versus the BinaryFormatter which upon inspection shows up to 27 bytes being used before all 0s showing...

 

BinaryFormatter includes meta data describing the type of object you are serializing. This allows it to be used for complex objects without having to know how all of the members' data is being stuffed into the byte array. This also makes it space inefficient, because if you already know you are receiving a string you don't need data saying things like yes this is a string packed into the data you are sending.

My approach when packing multiple types or even multiple strings into the same byte array is to use Encoding.UTF8.GetBytes, and then use BitConverter.GetBytes on the length of the resulting byte array from the string. I then pack those together into a new byte array starting with the 4 bytes from BitConverter to tell me just how many bytes in that array need to be deserialized into the string. I can then pack as many variable length strings as I want into my network messages, or mix in other serialized data types.

By the way, I'm not doing this with Unet, as I moved off Unet recently, but the principle should be the same. Additionally I use UTF8 instead of UTF16 (Unicode) because it should be more space efficient for the most common characters I generally send, which are the basic ASCII characters. Unless I'm mistaken, Encoding.UTF8.GetBytes("test") should result in a 4 byte array instead of your code's resulting 8 byte array.

 

I see, so basically you may have something like this?
[3, 0, 0, 0, 23, 84, 192, 2, 0, 0, 0, 88, 77]?
string-----> ^^^^^^^^^ and here--->^^^^^

and you limit it to 4 bytes to cover variable length strings up till length 255*4 which works well because you wouldn't want to be sending messages much longer anyways.

The only thing though is that I dont plan on sending that many variable length strings.
I imagine 90% of the traffic will just be in movement data. I figured I could just do something like
reserve the first 3 or 4 bytes for the net id of the object i want and the next 3 bytes for the argument types and the remaining be the actual message (a vector, bool, string, etc)

 

Arrays are indexed with a type of "int" so you can't actually have a larger byte array than max int as far as I'm aware anyway. UTF16 (Unicode) and UTF8 are variable length encoding systems. Even if you are sending the same number of characters, you can be sending a variable number of bytes to represent those characters.

If you want to avoid sending a variable that says how many bytes your string is I'd just send a single string in the message (so the length of the string in bytes is the length of the entire message), or use a fixed length encoding like UTF32 or ASCII. UTF32 is size inefficient for packing into a byte array, and ASCII is a limited character set and isn't localization friendly.

 

https://wobesdev.wordpress.com/2017/12/02/serialization-stuff/