Question How to serialize Dictionary<string, string> and Dictionary<string, CustomClass[]> in MessagePack?

So I seem to have hit this roadblock in building my app where an attempt to serialize these two objects of called "Dictionary<string, string> stat" and "Dictionary<string, Property[]> properties" just seem impossible. my code is sequential of course and an exception is returned at the point of serializing "stat" which comes before the serialization of"properties", meaning the code never even reaches the point of serializing properties dictionary, thus my first challenge is getting past serializing the stat dictionary. I'm asking both questions so I won't have to come back to type another question.

Error Message: (UnityFramework)[8340] <Notice>: FormatterNotRegisteredException: System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] is not registered in resolver: MessagePack.Resolvers.StaticCompositeResolver

Things I've tried:
Initialized messagePack composite resolvers before attempting to serialize or deserialize anything.:

Code (CSharp):

1. //initialize messagepack
2.             StaticCompositeResolver.Instance.Register(
3.                  MessagePack.Resolvers.GeneratedResolver.Instance,
4.                  MessagePack.Resolvers.StandardResolver.Instance,
5.                  MessagePack.Resolvers.BuiltinResolver.Instance
6.                
7.             );
8.  
9.             var option = MessagePackSerializerOptions.Standard.WithResolver(StaticCompositeResolver.Instance);
10.  
11.             MessagePackSerializer.DefaultOptions = option;

The generated resolver does seem to cover the Property class, however, the code never reaches there and it throws an error when serializing the "stat" dictionary which comes before the "properties" dictionary. Neither the standard resolver nor the built-in resolver solves the challenge of serializing the "stat" dictionary.

Code (CSharp):

1. static GeneratedResolverGetFormatterHelper()
2.         {
3.             lookup = new global::System.Collections.Generic.Dictionary<Type, int>(3)
4.             {
5.                 { typeof(global::AchievementManager.Property_ActivationRules), 0 },
6.                 { typeof(global::AchievementManager.Property_UpdateRules), 1 },
7.                 { typeof(global::AchievementManager.Property), 2 },
8.             };
9.         }

What in the world am I supposed to do to get message pack to serialize the two dictionaries. COncise help will be much appreciated.
Unity Version:2019.4.17f1
ScriptingBackend: IL2CPP
Api Compatibility Level: .Net 4.x
Strip Engine Code: False
Build Platform: IOS and Android

 

Can you get any Dictionaries to work at all through that message pack thingy? It might be that it requires a custom resolver just for the Dictionary itself.

Any particular reason you're using MessagePack instead of something tried and true like JSON? (I mean apart from speculative optimization reasons.)

 

Considering that this problem has little to do with Unity, and more to do with MessagePack and the library you're using to serialize into MessagePack. That's more where you should be focusing. (the .net version that unity targets will be important in that context of course).

I'd probably suggest asking around any communities that focus on MessagePack, especially the specific library you're using.

Speaking of which... which library are you using?

Is it this one?
https://github.com/neuecc/MessagePack-CSharp
(when having issues with 3rd party libs, you should include what lib you're using)

You should maybe check out the discussion there. They would likely have the beset information about using the library than any of us. Most of us here in the unity forums generally use json both the built-in unity json serializer, and newtonsoft, are easily integrated into a unity project.

Lastly, I'm with @Kurt-Dekker, is there a reason you're specifically targeting MessagePack? Are you maybe communicating with a web interface that expects messages in MessagePack format?

If not... why not just use JSON, Newtonsoft JSON support dictionary right out of the box.

...

In tangential news... that API for the MessagePack lib you're using... ain't going to lie, talk about hork.

 

Thanks so much for the suggestions @Kurt-Dekker and @lordofduct. I am using the library from here https://github.com/neuecc/MessagePack-CSharp, and yes I was bought over by the "infinitely fast" advert tag that attracted me to the library while googling for "fast unity serializer".

After endless searches online, the docs and intuition... I figured out how to register the dictionaries(The irrelevant public fields have to be tagged as ignore). This:

Code (CSharp):

1. public class StatsManager : MonoBehaviour, IGameManager {
2.     //The player statistics and achievements are managed here
3.     [IgnoreMember]
4.     public ManagerStatus status { get; private set; }
5.  
6.     [Key(0)]
7.     public Dictionary<string, string> stats;
8. }

Then for the "properties" dictionary:

Code (CSharp):

1. [MessagePackObject]
2. public class AchievementManager
3. {
4.         // public ManagerStatus status { get; private set; }
5.  
6.     [MessagePackObject]
7.     public class Property{    //a number of these are used to activate an achievement
8.         // internal string name;
9.         // internal float value;
10.         [Key(0)]
11.         public float activationValue; //threshold value for activation
12.         [Key(1)]
13.         public Property_ActivationRules activationRule;   //rule to activate the property
14.         [Key(2)]
15.         public Property_UpdateRules updateRule;   //rule to update the existing property value
16.         [Key(3)]
17.         public float intialValue; //current value of the property
18.         //value is compared to activation value using activation rule to check if to activate property
19.         [Key(4)]
20.         public bool activated;    //check of activated
21.         [Key(5)]
22.         public string percentComplete; //percentage comparison between initial value(current value) and activation value
23.         [Key(6)]
24.         public string name;
25.         [Key(7)]
26.         public string description;
27.  
28.  
29.     }
30.     public enum Property_ActivationRules{ //condition to get a property activated
31.         greaterthan,
32.         lessthan
33.     }
34.     public enum Property_UpdateRules{ //
35.         additive,   //add incoming value to existing value
36.         replace,    //replace existing value with incoming value after performing some check
37.      
38.     }
39. //I prepared the second dictionary for registration here
40.     [Key(0)]
41.     public Dictionary<string, Property[]> properties;

I think the picture is clear here, everything must ultimately be in a class labeled as messagepackobject, then all the dictionaries should be given a key label and the same for the class whose type are the values in the dictionary "properties". Then I ran the code generator from the inspector and it worked. The generated resolver code now indicated

Code (CSharp):

1. //code omitted
2. static GeneratedResolverGetFormatterHelper()
3.         {
4.             lookup = new global::System.Collections.Generic.Dictionary<Type, int>(8)
5.             {
6.                 { typeof(global::AchievementManager.Property[]), 0 },
7.                 { typeof(global::System.Collections.Generic.Dictionary<string, global::AchievementManager.Property[]>), 1 },
8.                 { typeof(global::System.Collections.Generic.Dictionary<string, string>), 2 },
9.                 { typeof(global::AchievementManager.Property_ActivationRules), 3 },
10.                 { typeof(global::AchievementManager.Property_UpdateRules), 4 },
11.                 { typeof(global::AchievementManager), 5 },
12.                 { typeof(global::AchievementManager.Property), 6 },
13.                 { typeof(global::StatsManager), 7 },
14.             };
15.         }
16. //more code

my types have been registered.

 

I'd love to see the speed differences from this and newtonsoft. Get a feel for what "infinity" is like.

 

I know this is more than a year old, but I wanted to add up to it.
I'm facing the same problem with IL2CPP runtime dictionary serialization, and I'll try out this solution. I've seen another person with the same issue and he did the same solution: Put the dictionary inside a class and give it a MessagePack Key attribute.
as for the comparison between MessagePack and Newtonsoft json, MessagePack is way faster. Newtonsoft is the slowest out there actually.
I have been building custom C# real-time game servers for a few years, and as a part of upgrading them, I constantly look for faster serialization. MessagePack gives the highest speed. The only problem is you have to make a custom serializer for some types, and you'll fall into some problems like this Dictionary problem the thread is about.
The second best solution would be Utf8Json ( from the same developer actually )

 

Well "speed" is relative. I don't see any advantage of MessagePack over Json in a context of a game, unless you're serializing/deserializing every frame or several times per second, like in network transmission for eg. (in this case even MessagePack might be slow...) I don't see how the player would be bothered for few-tens milliseconds of difference. I'd say the player won't even notice the difference between 2 secs and 2.2 secs, while you, the developer, will gain peace of mind.

My game saves are around 1 mb (Json + gzipped) and I can say that they contain A LOT of data, are loaded and saved in around a second, on 5 years old PC.

 

In an offline game, it won't matter much. Json would be completely fine.
But in a game server ( or game client ), you send and receive many messages per second, for example the position updates of all game players, that need to be deserialized and processed. In this case serialization makes a difference.
Since I'm building both server and client, they both use the same serialization, and the server handles many rooms at the same time, so, hundreds of updates per second.
But yes, usually developers shouldn't bother, and they should gain peace of mind ( just using json )

 

That's exactly what I pointed out in my reply. And also, as I pointed out, MessagePack might not be the optimal solution for fast paced multiplayer games. Sending raw (and optimized) packets over UDP, RUDP or even Quic might be better.