Problem Deserializing to Non-Serializable Collections

I'm trying to use ISerializationCallbackReceiver to allow me to serialize/deserialize a HashSet<int> to/from a List<int>. However, I'm running into an issue where I can't change the size (and thus, the contents) of the underlying List. Here's some code, which I've tried in 5.0, 5.1, and 5.2.

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using System.Collections.Generic;
5.  
6. public class HashSetSerializer : MonoBehaviour, ISerializationCallbackReceiver {
7.  
8.     [SerializeField]
9.     private List<int> serializedField;
10.  
11.     private HashSet<int> runtimeField;
12.  
13.     #region ISerializationCallbackReceiver implementation
14.     public void OnBeforeSerialize ()
15.     {
16.         this.serializedField = new List<int> (this.runtimeField);
17.         Debug.Log ("HashSet script BeforeSerialize -- # of elements ser / runtime: " + this.serializedField.Count + "/ " + this.runtimeField.Count);
18.     }
19.  
20.     public void OnAfterDeserialize ()
21.     {
22.         this.runtimeField = new HashSet<int> (this.serializedField);
23.         Debug.Log ("HashSet script AfterDeserialize -- # of elements ser / runtime: " + this.serializedField.Count + "/ " + this.runtimeField.Count);
24.     }
25.     #endregion
26.  
27. }
28.  

When I go into the editor, and change the size of the "serializedField" List from 0 to say, 5, the size instead clamps to 1 -- the only sizes that take seem to be 0 and 1. Attempting to change the size does trigger deserialization though, which provides console output that reads: "HashSet script AfterDeserialize -- # of elements ser / runtime: 5 / 1." I get the same effect if I instead manually iterate over the List's elements to populate the HashSet.

Here's where it gets a bit stranger: If I create a different MonoBehaviour that changes the runtime field to a List, then it all works! Like this:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using System.Collections.Generic;
5.  
6. public class ListSerializer : MonoBehaviour, ISerializationCallbackReceiver {
7.  
8.     [SerializeField]
9.     private List<int> serializedField;
10.  
11.     private List<int> runtimeField;
12.  
13.     #region ISerializationCallbackReceiver implementation
14.     public void OnBeforeSerialize ()
15.     {
16.         this.serializedField = new List<int> (this.runtimeField);
17.         Debug.Log ("HashSet script BeforeSerialize -- # of elements ser / runtime: " + this.serializedField.Count + "/ " + this.runtimeField.Count);
18.     }
19.     public void OnAfterDeserialize ()
20.     {
21.         this.runtimeField = new List<int> (this.serializedField);
22.         Debug.Log ("HashSet script AfterDeserialize -- # of elements ser / runtime: " + this.serializedField.Count + "/ " + this.runtimeField.Count);
23.     }
24.     #endregion
25. }
26.  

Note that the UNDERLYING SERIALIZED TYPE in both cases is a List, which Unity can serialize -- all that's changed is the runtime type that we're deserializing to, so this should have nothing to do with that, yet it somehow seems to. Note that replacing the HashSet with a Dictionary (another type that Unity can't serialize) breaks in the same way that a HashSet does. You can confirm this by trying out the code that Unity provides as the ISerializationCallbackReceiver example, found here: http://docs.unity3d.com/ScriptReference/ISerializationCallbackReceiver.OnBeforeSerialize.html. In that case, the underlying Lists are stuck with their initialized contents, and the size of the Lists can't be changed.

For the HashSet example, changing the collection type from <int> to that of a nullable, serializable object (like an Object tagged with [Serializable]), the maximum size of the serialized list becomes 0, instead of 1.

All told, there are 3 possibilities that I can think of right now:
1) I've just done something incorrectly, but if that's the case then Unity needs to update their example because that seems just as broken.
2) This is a bug in Unity. Possible, given that the problem only seems to arise when working with types that Unity can't serialize.
3) This has to do with C#. The change in behavior between collections of non-nullable ints vs nullable Objects suggests that it might have to do with how C# deals with empty or empty-ish collections, but I'd expect normal behavior out of the List<int>, which is NOT empty when resized (the default int is 0, not Null).

Anyways, I'd love to hear people's thoughts on this. Are people out there getting ISerializationCallbackReceiver to work properly? Are you experiencing similar issues?

 

Looking into this a bit more, I've found that the problem is only with increasing the size of the underlying list. Existing values can be changed, and the list can be trimmed down without problem. For example, I copied the Unity example code for serializing a dictionary and tried making changes in the editor. That code is here:

Code (csharp):

1.  
2. using UnityEngine;
3. using System;
4. using System.Collections;
5. using System.Collections.Generic;
6.  
7.  
8. public class SerializationCallbackScript : MonoBehaviour, ISerializationCallbackReceiver
9. {
10.     public List<int> _keys = new List<int> { 3, 4, 5};
11.     public List<string> _values = new List<string> { "I", "Love", "Unity"};
12.    
13.     //Unity doesn't know how to serialize a Dictionary
14.     public Dictionary<int,string>  _myDictionary = new Dictionary<int,string>();
15.    
16.     public void OnBeforeSerialize()
17.     {
18.         _keys.Clear();
19.         _values.Clear();
20.         foreach(var kvp in _myDictionary)
21.         {
22.             _keys.Add(kvp.Key);
23.             _values.Add(kvp.Value);
24.         }
25.     }
26.    
27.     public void OnAfterDeserialize()
28.     {
29.         _myDictionary = new Dictionary<int,string>();
30.         for (int i=0; i!= Math.Min(_keys.Count,_values.Count); i++)
31.             _myDictionary.Add(_keys[i],_values[i]);
32.     }
33.    
34.     void OnGUI()
35.     {
36.         foreach(var kvp in _myDictionary)
37.             GUILayout.Label("Key: "+kvp.Key+ " value: "+kvp.Value);
38.     }
39. }
40.  

Changing the value "Unity" to "Cheese" worked fine, as did reducing the size of the keys list. Trying to increases the size of the keys list (even after it had been reduced already) causes the odd behavior.

Furthermore, there's some weird stuff going on with initialized values (like the "I Love Unity" list). The only way to get the initialized values to appear is to add an emtpy script to a GameObject using "Add -> New Script" and then edit the script (like pasting the code in above). Creating the script code first and THEN adding it to the GameObject leaves the lists empty and uneditable. Using "reset component" also results in empty, non-editable lists.

This is all massively messed up, and I'm frankly shocked that I haven't been able to find any other examples or discussion on the Unity forums. Is nobody else in the world trying to serialize HashSets or Dictionaries using ISerializationCallbackReceiver?

 

ALL RIGHT! I found the cause of this. I wouldn't call it a bug with Unity, but it's definitely something that should be mentioned somewhere.

The issue is that that keys in a HashSet need to be unique. When you change the size of the underlying list in the editor, the empty values are set to the default for the key type (like 0 if you're working with ints). Changing the size of the list causes Unity to deserialize the script, which means trying to put a list with a bunch of 0s in it into a HashSet. The extra 0s get thrown out. The HashSet (which is still too small) gets serialized back to the List, and that's the issue!

3 ways around this I can see:
1) Use preprocessor directives to make things behave differently in the editor
2) Add a "drity" flag to the script so that the HashSet only gets serialized if you've actually got content to serialize (or if you want to)
3) Use OnBeforeSerialize as normal, but use Awake as a soft "deserialize" instead of using OnAfterDeserialize. This allows you to use a HashSet at runtime without gunking up the editor.

 

It's a good idea if you can report a Documentation feedback to Unity, so they can add this to the Docs for future references. Thanks.

 

...2018 and this is still an issue... yay...

 

They just don't have priority over all other, more important, issues.

 

Fair, enough, but they could at least change the example