Bug Field with SerializeReference attribute serializing to wrong RID in .asset file

In my project (mainly for learning Unity and experimenting with stuff, thus the reason I might be over complicating things), I have a custom (recursive) data structure which aims to be able to store any kind of data but with some runtime type safety. It's called ItemData.

I had to write a custom serialization process for this class, as it uses a dictionary during runtime to store its values. I serialize the class by converting this dictionary to a list of SerializableKeyValue, which holds a string for the key and a value of type ValueBase which is tagged with [SerializeReference] to store the value:

Code (CSharp):

1.  
2. [Serializable]
3. public class SerializableKeyValue {
4.    public string key;
5.    [SerializeReference] public ValueBase value;
6.  
7.    public SerializableKeyValue(string key, ValueBase value) {
8.       this.key = key;
9.       this.value = value;
10.    }
11. }
12.  

And the serialization code:

Code (CSharp):

1.  
2. public void OnBeforeSerialize() {
3.    serializedData = new SerializableKeyValue[tags.Count]; // tags variable is the dictionary.
4.  
5.    int i = 0;
6.    foreach ((IKeyBase key, object value) in tags) {
7.       // Because the dictionary stores the actual object value,
8.       // rather than the wrapped value in a ValueBase, there is a method to transform
9.       // an object (of the correct runtime type) to a ValueBase on the Key class.
10.       serializedData[i] = new SerializableKeyValue(key.ID, key.AsValueBase(value));
11.       i++;
12.    }
13. }
14.  

Code (CSharp):

1.  
2. public void OnAfterDeserialize() {
3.    tags = new Dictionary<IKeyBase, object>();
4.  
5.    foreach (SerializableKeyValue kvp in serializedData) {
6.       // GetValueAsObject is a method in ValueBase which simply
7.       // gets the value in a ValueBase as an object.
8.       tags[keyMap[kvp.key]] = kvp.value.GetValueAsObject();
9.    }
10.  
11.    serializedData = null;
12. }
13.  

ValueBase is a base class which other classes derive from, so they can be used as values in my custom data structure, thus the reason why it has to be tagged with [SerializeReference]. I am currently testing with two subclasses of ValueBase: One which stores an Integer, and one which stores ItemData (the recursive part):

Code (CSharp):

1.  
2. [Serializable]
3. public sealed class IntValue : DataValue<IntValue, int> { // The DataValue class has some helper methods.
4.    [SerializeField] private int value;
5.    public override int Value {
6.       get => value;
7.       set => this.value = value;
8.    }
9. }
10.  

Code (CSharp):

1.  
2. [Serializable]
3. public sealed class ItemDataValue : DataValue<ItemDataValue, ItemData> {
4.    [SerializeField] private ItemData value = new ItemData();
5.    public override ItemData Value {
6.       get => value;
7.       set => this.value = value;
8.    }
9. }
10.  

Serialization works fine for the Integer version (lets call it Integer1), and also for the plain ItemData version (ItemData1). But, when I add, say, an Integer (Integer2) to the nested ItemData the serializer seems to get confused. Now I don't mean that it simply does not serialize, in fact, it serializes completely. The problem is that the RID used in the created .asset file does not match the RID of Integer2, it is one too high. This of course causes the deserializer to give an error stating the RID is missing. I will include the asset file as raw text, because I can't upload it as a .asset

I hope my explanation is somewhat understandable, and if anything or everything is unclear, just ask. I won't include my entire project here (I don't even know if that's possible), but I'll include all relevant files. You should be able to reproduce the issue with just those files.

[Update]

I've attempted to exactly replicate the bug from an empty slate, For some reason the code I made serializes to the exact same .asset file (at least, the ItemData part, rest was left out), except the RIDs are now correct for the nested integer value...

As I can not upload any more files, the code for this is pasted below:

ItemDataNonBugged.cs:

Code (CSharp):

1. using System;
2. using UnityEngine;
3.  
4. [Serializable]
5. public class ItemDataNonBugged : ISerializationCallbackReceiver {
6.  
7.     public SerializableKeyValue[] serializedData;
8.  
9.     public ItemDataNonBugged(SerializableKeyValue[] serializedData) {
10.         this.serializedData = serializedData;
11.     }
12.  
13.     [Serializable]
14.     public class SerializableKeyValue {
15.         public string key;
16.         [SerializeReference] public ValueBase value;
17.  
18.         public SerializableKeyValue(string key, ValueBase value) {
19.             this.key = key;
20.             this.value = value;
21.         }
22.  
23.         public override string ToString() {
24.             return key + ": " + value;
25.         }
26.     }
27.  
28.     public abstract class ValueBase {
29.        
30.     }
31.  
32.     [Serializable]
33.     public class IntValue : ValueBase {
34.         public int value;
35.  
36.         public IntValue(int value) {
37.             this.value = value;
38.         }
39.  
40.         public override string ToString() {
41.             return value.ToString();
42.         }
43.     }
44.  
45.     [Serializable]
46.     public class ItemDataValue : ValueBase {
47.         public ItemDataNonBugged value;
48.  
49.         public ItemDataValue(ItemDataNonBugged value) {
50.             this.value = value;
51.         }
52.  
53.         public override string ToString() {
54.             return value.ToString();
55.         }
56.     }
57.  
58.     public override string ToString() {
59.         string str = "{";
60.         foreach (SerializableKeyValue serializableKeyValue in serializedData) {
61.             str += serializableKeyValue + ", ";
62.         }
63.  
64.         str = str.Substring(0, str.Length - 2) + "}";
65.         return str;
66.     }
67.  
68.     public void OnBeforeSerialize() {
69.         Debug.Log("start ser");
70.         foreach (SerializableKeyValue serializableKeyValue in serializedData) {
71.             Debug.Log(serializableKeyValue.ToString());
72.         }
73.     }
74.  
75.     public void OnAfterDeserialize() {
76.         Debug.LogWarning("start deser");
77.         foreach (SerializableKeyValue serializableKeyValue in serializedData) {
78.             Debug.LogWarning(serializableKeyValue.ToString());
79.         }
80.     }
81. }
82.  

Data.cs:

Code (CSharp):

1. using UnityEngine;
2.  
3. [CreateAssetMenu]
4. public class Data : ScriptableObject {
5.  
6.     public ItemDataNonBugged defaultItemData;
7.    
8. }
9.  

DataEditor.cs:

Code (CSharp):

1. using UnityEditor;
2. using UnityEngine;
3.  
4. [CustomEditor(typeof(Data))]
5. public class DataEditor : Editor {
6.  
7.     public override void OnInspectorGUI() {
8.         Data data = (Data)target;
9.  
10.         if (GUILayout.Button("Set Value")) {
11.             data.defaultItemData = new ItemDataNonBugged(new[] {
12.                 new ItemDataNonBugged.SerializableKeyValue("integer", new ItemDataNonBugged.IntValue(1)),
13.                 new ItemDataNonBugged.SerializableKeyValue("itemData", new ItemDataNonBugged.ItemDataValue(new ItemDataNonBugged(new[] {
14.                     new ItemDataNonBugged.SerializableKeyValue("integer2", new ItemDataNonBugged.IntValue(2))
15.                 })))
16.             });
17.             EditorUtility.SetDirty(target);
18.         }
19.  
20.         if (GUILayout.Button("Read Data")) {
21.             Debug.Log(data.defaultItemData);
22.         }
23.     }
24. }

If someone is able to find why this discrepancy occurs, I would be very grateful.

 

Update, for anyone which has a similar problem:

After *a lot* of testing, I was able to figure out the couple of conditions that were causing the bug.
It turned out that creating a new object (which is serialized as reference) in the OnBeforeSerialize method causes it the incorrectly serialize the reference, specifically this line of code was causing it:

Code (CSharp):

1. serializedData[i] = new SerializableKeyValue(key.ID, key.AsValueBase(value)); // value is of type System.Object, and is wrapped into a ValueBase

The way I fixed it in the main version of my project is by already storing the serializable wrapper objects, rather than wrapping the actual values in OnBeforeSerialize:

Code (CSharp):

1. serializedData[i] = new SerializableKeyValue(key.ID, value); // value is already of type ValueBase

Apparently creating new objects that are serialized directly (so not by reference) works fine, as evident by creating the new SerializableKeyValue.

I'm going to assume this is a bug with Unity's serialization so I guess it's time to send in a bug report