Proper use of OnAfterDeserialize

The following is an example of a custom serialization that does not work.
Calling mySubClass.LoadData does not work as expected. I would expect OnAfterDeserialize to be called on MyClass after all of its fields have been deserialized. But it is behaving the opposite, as if it had not finished deserializing all of the fields from mySubClass.
Here goes the example that does not work:

Code (CSharp):

1. class MyClass : MonoBehaviour, ISerializationCallbackReceiver {
2.     [SerializeField] SubClass mySubClass;
3.  
4.     public void OnBeforeSerialize () {
5.         mySubClass.SaveData ();
6.     }
7.  
8.     public void OnAfterDeserialize () {
9.         mySubClass.LoadData(); // For some reason, this will fail. It is as if the serialized data was still being deserialized while we're trying to read from it.
10.     }
11. }
12.  
13. class SubClass: ScriptableObject {
14.     [SerializeField] DataSerial serializedData; // Unity will serialize this
15.     [HideInInspector] Data unserializedData;    // Unity will not serialize this
16.  
17.     // And many other variables that unity serializes normally.
18.  
19.     public void SaveData () {
20.         // Save the data to the serializedData variable.
21.     }
22.  
23.     public void LoadData () {
24.         // Load the data from the serializedData variable.
25.     }
26.  
27.     class DataSerial : ScriptableObject {
28.         // Lots of  stuff unity can serialize.
29.     }
30.  
31.     class Data {
32.         // Lots of stuff unity can't serialize.
33.     }
34. }
35.  

On the other hand, if we put the ISerializationCallbackReceiver on the SubClass, it seems to work correctly.

Code (CSharp):

1. class MyClass : MonoBehaviour {
2.     [SerializeField] SubClass mySubClass;
3. }
4.  
5. class SubClass: ScriptableObject, ISerializationCallbackReceiver {
6.     [SerializeField] DataSerial serializedData; // Unity will serialize this
7.     [HideInInspector] Data unserializedData;    // Unity will not serialize this
8.  
9.     // And many other variables that unity serializes normally.
10.  
11.     public void OnBeforeSerialize () {
12.         // Save the data to the serializedData variable.
13.     }
14.  
15.     public void OnAfterDeserialize () {
16.         // Load the data from the serializedData variable.
17.     }
18.  
19.     class DataSerial : ScriptableObject {
20.         // Lots of  stuff unity can serialize.
21.     }
22.  
23.     class Data {
24.         // Lots of stuff unity can't serialize.
25.     }
26. }
27.  

Am I doing something wrong on the first case? Is the second case reliable? I really don't fully understand why the first case won't work. I would expect the OnAfterDeserialize to be called after everything of the class has been fully deserialized, but it does not seem to work this way.

EDIT: After some more testing, not even the second case seems to work 100% of the time. Is this a bug?

 

OnAfterDeserialize is called after the object the interface is attached to is deserialized.

Not after EVERYTHING is deserialized.

So your 'MyClass' is done deserializing, but since 'SubClass' is a ScriptableObject and therefore serialized by reference... that means it gets serialized/deserialized at its own pace, and it's not necessarily ready yet.

This is not a bug, this is just how unity serializes/deserializes all UnityEngine.Object's. It is multi-threaded and out of order (or rather in an order not necessarily defined, and can change).

It's even in the documentation:
https://docs.unity3d.com/ScriptReference/ISerializationCallbackReceiver.html

 

Sorry for the necro, is there any way in 2020.1 to ensure that a referenced ScriptableObject instance is loaded when working with it in

OnAfterDeserialize()

?

Wouldn't this mean that there are cases in which you use a referenced SO instance in a MonoBehaviour with the referenced instance not being deserialized and thus returning null? How would one prevent this?

 

ISerializationCallbackReceiver should only handle data of itself.

What is it you're attempting to do that you need to modify the data of a reference ScriptableObject during the serialization/deserialization of some other object?

What is your end-goal, maybe we can assist in coming up with another way to accomplish said goal. Instead of using ISerializationCallbackReceiver in a manner it wasn't designed to be used.

 

Thanks for coming back to me, I made a thread about it here: ISerializationCallbackReceiver fails to get referenced ScriptableObject instances?

Basically, I want to use SO instances instead of C# enumeration values (because it scales well and easily allows changing existing values without breaking existing serialized objects), but because I cannot get the SO instance, not itself nor its data, I cannot work with the instances.

 

I stumbled across the same problem.

My use case is somehow similar to @Ardenian's:
I want to transform the data of a scriptable object before it is used, because I need the data in a different form at runtime than I want it to edit in the inspector. (E.g: I serialize the object with a

List<A>

but at runtime I need a

Dictionary<K, A>

where the keys are some property of

A

.)

It is not clear to me, why an "after deserialization callback" is called before the instance is fully deserialized, where I would define an instance to be "deserialized" if and only if it is in such a state that I one use it without unexpected errors like null references which are not there because of my failure to assign something (e.g. in the inspector for scriptable objects).
@lordofduct: In fact, this seems even more like a bug to me after reading the documentation to the end, where is stated:

"The order of callback execution is not guaranteed between such objects. However it is guaranteed that the main object's

OnBeforeSerialize

callback would be invoked before those implemented by the referenced objects. And

OnAfterDeserialize

on the main object would be invoked after it is called on all the referenced objects."
​

This - from my perspective - implies what @Ardenian and I expected - that referenced objects are deserialized in the OnAfterDeserialize callback in the referencing class.


Some workaround suggestions:

• One could avoid the direct usage of a scriptable object and instead wrap it in a mono behaviour that transforms and exposes the given data.
  While this would work this seems for me to undermine the idea of scriptable objects
• It is worth it to think again about the structure of the data and if it can be represented in a state near to the desired. For example, you can use a serializable dictionary instead of the common "key-value pair" list. Of course, this is not always possible or sensible.

Note that

Awake()

apparently is not called on the scriptable object at all (at least, I could not achieve it to be called).

TLDR: I think this is a bug and it should be possible to access referenced objects in the OnAfterDeserialization callback relying on those not being null.

Edit: I submitted a bug report, let's see what comes out.

 

Because this is a chicken and egg problem. ScriptableObjects are standalone assets. They have their own "serialization stream" or serialization data. So fields that reference other UnityEngine.Objects can only be filled after all objects have been deserialized. Keep in mind that two UnityEngine.Objects can happily reference each other with a field. So there is absolutely no way you get a clear order as you want.

When the documentation talks about the "main" object and "child" objects it talks about serializable classes that are serialized inside the main class, not referenced UnityEngine.Object classes. Their use of "referenced classes" is in deed a bit misleading.

You can solve all those problems by not deriving your child data object from ScriptableObject.

As lordofduct said, the ISerializationCallbackReceiver is only for transforming any non serializable data into serializable data for this object only. Since those callbacks run on the serialization thread you can not use anything of the Unity API that isn't threadsafe in those callbacks.

ScriptableObjects should only be used when you need standalone data / functionality that can be referenced from one or multiple places.

Nobody in this thread here who has problems with the interface have given a clear example of their problem and what data this is about. The case that Ardenian presented in the other question was not really about serialization. He just wanted to do "post processing" on the referenced serialized data. This should be done in Awake, that's what Awake is good for. This had little to do with the serialization of the data itself. The dictionary was just for convenient access of the data. You can't rely on data that is serialized seperately in those callbacks. Awake is guaranteed to run after the complete object initialization has finished. Keep in mind that there is no relationship (in the serialization sense) between an object that references a ScriptableObject or any other UnityEngine.Object.

This is impossible in many cases as mentioned above. See this example:

Code (CSharp):

1.  
2. public class A : ScriptableObject, ISerializationCallbackReceiver
3. {
4.     public B bRef;
5.     public void OnBeforeSerialize()
6.     {
7.     }
8.     public void OnAfterDeserialize()
9.     {
10.     }
11. }
12.  
13. public class B : ScriptableObject, ISerializationCallbackReceiver
14. {
15.     public A bRef;
16.     public void OnBeforeSerialize()
17.     {
18.     }
19.     public void OnAfterDeserialize()
20.     {
21.     }
22. }
23.  

This is a classical circular reference which is totally possible and allowed since object A and object B are completely seperately serialized objects. Of course there is absolutely no way to have OnAfterDeserialize to run after the reference to the other object has been setup / restored. One has to run before the other. So one of the object can not possibly be finished with its deserialization. This is not a bug, just an impossiblility. Again you all try to use that interface to do things it's not meant to do. You try to off load certain data to other objects that are part of a seperate serialization stream / object.

 

It is called. In editor, when you create it, in build, when the game starts (it gets created).
Here is an example how you can achieve "Awake" in editor when you enter play mode.

 

This is unbelievable. Thank you for the suggestion but this is just not the way I want to write code.

@Bunny83: It should not be a problem to enforce a sensible deserialization order up to cyclic dependencies in my opinion.

What should be done in this method, if not "post-processing" of deserialized data? We already gave rather concrete use case scenarios. What should this callback be used for, in your opinion?

Let's imagine an even more concrete example:

- We want to have a class

Card

for a card game
- We want to have a class

CardCollection

for a collection (deck?) of cards
- Each card has the properties attack and color
- A card collection should contain the properties

totalAttackSum

and a dictionary

colorsToCards

mapping colors to all cards of that color in a deck

We also want to create new cards and card collections in the editor using the inspector so we let them be scriptable objects.
Obviously, the card collection has a list of cards that is serialized such that you can then assign them in the inspector.
The properties

totalAttackSum

and

colorsToCards

should not be serialized but, since they are dependent of the cards assigned to a deck, they should be initialized based on that information.

The most intuitive and in my opinion sensible way to achieve this without unity or inspectors would be to call the constructor

new CardCollection(cards)

which then generates additional data based on the parameter and populates things like to

colorsToCards

-structure. Since we can not use constructors in unity such initialization is often moved to

Awake / Start

where the inspector serves as the constructor (or actually the field-injection mechanism). Since those methods serve another purpose in the case of Scriptable Objects the only reasonable place for initializations of this kind which I see is the OnAfterDeserialize callback.

 

Every asset behaves the same. It is consistent. ScriptableObjects in an asset file are assets, they behave like it.

Well, that's up to you. I choose working solutions over code beauty 100% of the time.

 

The only thing I can tell from this thread is that some people are like "I don't like the way Unity has designed it".

It's not that it's technically "wrong", Unity did it a specific way. Y'all just don't like it that way.

Which... is fine. There's many things I don't like about the choices Unity makes time and time again. But that's what you get when you use a tool designed by a 3rd party... you have to accept, or work around (like @Lurking-Ninja has demonstrated), that design.

There's really nothing more that we can say on this aside from giving you examples of how you may work around it in the cases where people have gave specific examples of what they wanted. Or shrug in those cases where examples aren't given.

Other than that it's up to Unity... and well, we're not Unity.

Maybe they'll see this and change it... but I wouldn't hold my breathe. Changes to the way serialization works is few and far between (we waited YEARS to get generics serializable). I have a strong feeling it's because serialization is the back bone of quite a bit of Unity that any changes to it have far sweeping ramifications (that's just speculation based on decades of experience working on my own teams in the business world where tightly integrated tool sets like serialization have made changes very difficult).

So yeah... if anyone has a specific thing they'd like a work around for. Ask away.

Otherwise... it's just a bitch fest.

Which I mean... to be fair. Bitch fests are very cathartic at times.

 

You are taking my statements way to personal, sorry for that.
I like unity and it is not my aim to complain.

Besides that, I gave an example which is as concrete as one can get and asked for specific suggestions how you would implement this structure.

Code Quality is a big important factor especially in game design. To restate my question:

"How to implement initialization of scriptable objects, at what point at runtime I can rely on the field injection to have been finished?"

 

You missed the first sentence of that paragraph, which is very important: "This interface is supported on objects that are referenced by

SerializeReference

"

SerializeReference

is kind of another beast entirely, I'll leave it to you to check the docs to understand how it works completely, but basically serializing using that does impose a strict tree-like ordering for the serialization (self-referential recursion is disallowed), which means that the inner objects can be guaranteed to be fully initialized.

(Though I am fairly certain that paragraph and the description above also applies to plain nested objects too, as long as they don't inherit

UnityEngine.Object

)

In the context of plain ol'

public

/

SerializeField

initialization, then the ordering issues are valid, and really not possible to solve at all. Since instances of

UnityEngine.Object

(Such as

MonoBehaviour

or

ScriptableObject

) can only be serialized like that, not through

SerializeReference

, then indeed, you cannot assume that the object reference to a

UnityEngine.Object

will be valid at

OnAfterDeserialize

, only from

Awake

onwards.

As others explained above, you can only have "strict deserialization order" OR "possibility of circular references". Unity chose the later for

UnityEngine.Object

, so we can't have the former.

To solve the "serialize a dictionary" use-case specifically, instead of deducing the key at deserialization time, you could instead serialize the key together with the value, using a

List<MyKeyValuePair>

instead of

List<MyValue>

, where

MyKeyValuePair

is a private nested serializable struct that stores the key and the value.

If that is not viable, because the key is dependent on runtime data, then you need to decide when the key becomes valid using your own lifecycle. If the dictionary will only be used after the keys are known to be valid, you could also lazy-initialize the dictionary at first use instead.

Finally, to answer your restated question directly: You can rely on serialization to have been completely resolved from

Awake

onwards.

 

Would OnAfterDeserialize be even more useful if it followed the Script Execution Order in your Project Settings?

Where do folks think would be the best place to petition for this? It seems like it would be possible to do. It could still be multi-threaded but done in batches by order.

And it would make serializing dictionaries that reference other things a good bit easier in some cases.

Until then Awake can work I suppose. One thing for folks to consider is pairing Awake with the ExecuteAlways attribute if you need things to work a certain way in the editor.