Scriptable Objects, Deep Copy / Serialization on Run Time

Hi everyone.
I think by now, anyone who had worked with scriptable objects knows that the serializations and tweaking of the asset was meant to happen in editor and not at run time. But there is a question that kept me busy for quit some time. first of all, lets take a look at one extension method :

Code (CSharp):

1. public static class ExtensionMethods
2.     {
3.         public static T DeepClone<T>(this T toClone)
4.         {
5.             using (var stream = new MemoryStream())
6.             {
7.                 var formatter = new BinaryFormatter();
8.                 formatter.Serialize(stream, toClone);
9.                 stream.Seek(0, SeekOrigin.Begin);
10.                 return (T)formatter.Deserialize(stream);
11.             }
12.         }
13.     }

so, let me break the questions one by one and hopefully someone, specially would be happy day in life if that person is from unity developers, would answer these questions :

1- what if i use that extension method on a scriptable object at run time? for example, lets say i made Item class a scriptable object and the item data base another one and the inventory the third one, now, when i want to give an item to a player, i have to make a new deep copy of the Item scriptable object from the database scriptable object and put that deep copy one into the inventory. so far so good. but.... normally for a deep copy on scriptable objects, people write manual code to assign values, for something like an item class that is not a problem, but for a nested objected oriented hierarchy of scriptable objects, that would become a nightmare. could the code above, produce a functional and correctly working scriptable object in MEMORY? (after all scriptable objects are c# objects at the end of the day?)


ok, now for question two :

2- Anyone familiar with Deep Copy by Expression Trees? (I saw it here and don't know anything about it LINK). Would this solution work with unity's objects? The article mentions that even if the object is not marked serializable it would work, so the big question here is would it work with almost all of the unity object types? (Again thinking about scriptable objects)

And finally last question :

3- WHAT IF, I serialize a deep copy of a scriptable object that exists in memory, for example, at the end of a game level and do it by things like json or bson or anything that would let me keep that data on a hard drive or any other storage type? Would I be able to read those data back at the start of another session and deserialize it, instantiate a new scriptable object in memory and using one the above deep cloning methods assign the deserialized data back? would this solution work or not?

If the answer to one of the first questions would be yes and the 3rd question get a positive answer, wouldn't it mean that we found a way to serialize/deserialize and use those fancy scriptable objects outside of editor? although they only sit in memory but wouldn't that make scriptable objects a hell lot more usefull during the run time?

thanks for helping and answering.

 

That's pretty deep. I don't think I have all the answers, but I can offer a little help.

1 - Runtime generation of scriptable objects is a reasonable approach. ScriptableObject already has a CreateInstance static method on it, which will create an in-memory instance of a ScriptableObject. I use this approach when assigning configs at runtime to objects, where I don't want actual assets in my project for every possible config. I don't see any meaningful difference between CreateInstance and your deep copy, as they're both additional run-time instances of a ScriptableObject. As for actually producing the copy of the object, I think it's possible you could create a ScriptableObject that can't be fully copied depending on the data types of all the properties and child properties. But chances are, if it's a field you can set in the inspector, it's a field that will copy properly. So I'd be somewhat optimistic about your approach.

2 - Not sure.

3 - I think that should work. I created a base class for most of the ScriptableObject in my project, called GameSettingsBase. The main value of the base class is that I added methods to persist the object to disk, and load it from disk. Technically I extract certain fields, store them in an XML file, and then restore those fields to the ScriptableObject later, but it's similar to your idea. This allows me to "Save" my scriptable object, then make changes it to, and then "load" the old version if I don't like my changes. So, your approach would likely work, though you may have to reset to saving the field data to a simpler data type.

 

1) serializing ScriptableObject like you describe

ScriptableObject is a UnityEngine.Object. Since this is so this means that there is a 'unity' portion to the object, and a 'C#/mono' portion of the object. These 2 objects are associated by the 'instanceId' which is a member stored on all UnityEngine.Objects:

Code (csharp):

1.  
2. using System;
3. using System.Runtime.CompilerServices;
4. using System.Runtime.InteropServices;
5. using System.Security;
6. using UnityEngine.Internal;
7. using UnityEngine.Scripting;
8. using UnityEngineInternal;
9.  
10. namespace UnityEngine
11. {
12.   /// <summary>
13.   ///   <para>Base class for all objects Unity can reference.</para>
14.   /// </summary>
15.   [RequiredByNativeCode]
16.   [StructLayout(LayoutKind.Sequential)]
17.   public class Object
18.   {
19.     internal static int OffsetOfInstanceIDInCPlusPlusObject = -1;
20.     private IntPtr m_CachedPtr;
21.     private int m_InstanceID; //<---------- this bad boy right here
22.     private string m_UnityRuntimeErrorString;
23.  

By serializing and deserializing the object in the manner you just described. What you end up with is a clone of the C# portion, but it's still pointing at the same 'unity' object on the C++ side of things. This being the result of the fact that 'm_InstanceID' has be preserved.

That is to say... if you even could. But the serializer will probably fail since ScriptableObject isn't marked as serializable. Though you can get around this using a SerializerSurrogate of course. But why would you since the problem I just described.

Instead you should use Object.Instantitate, which clones UnityEngine.Objects. And does so on both the C# and Unity side of things.

2) Member clone by expression trees

This will suffer from the issue as I described before.

Note that using reflection, or expression trees, to generate objects at runtime are basically like calling 'new SomeUnityObject'. And that's a big no-no in unity. You need to let unity create unity objects for you.

3) WHAT IF, I serialize a deep copy...

This can work to an extent.

First off, you need to change the way you make a deep clone... or just not do it at all since really it's not really necessary.

Next, you'll need to change the way you instantiate the object since you need Unity to create it for you. Don't use the Object.Instantiate though since you're creating instances... use ScriptableObject.CreateInstance (this is like the difference between cloning and a constructor respectively).

Furthermore, unity is going to fill out all fields in the hierarchy of objects. So if it has references to other unity objects, those will be filled in the way unity does so. So be careful how you fill in those values and what not. Depending how you do it you can end up with duplicates of objects, or you could end up modifying references to objects you didn't mean to modify.

But yeah, you can then set all the fields based on that.


...

With that said I usually tokenize my objects for serialization. And usually allow an object to implement some interface for tokenizing so it can control the manner in which it gets tokenized (if it needs to). Essentially making shallow clones in the form of a token, and if they NEED to be deep doing so on an adhoc basis. Essentially I convert the ScriptableObjects into pure C# objects so I'm not restricted by the unity stuff.

In theory sure I could probably devise some set of rules to then convert into an algorithmic process of serializing ScriptableObjects in an implied manner. But the amount of up front work for that, as well as limitations that'd end up getting built into it (flagging what gets serialized and not, so on)... is it really all that worth it?

If you think it is becuase you will have tens of hundreds of different 'types' that will be serialized that have deep hierarchies. Well yeah, it may be necessary.

But if you want to do that, it sounds like you need to first wrestle with and learn the whole way unity objects are created. And understand the impact that has on how you can treat those objects. Meaning any ol' C# serialization practices just don't apply without heavy modification.

 

Well, the reason i was looking for deep copy, was that i am building a behavior tree that is using scriptable objects as it's actual nodes, so, on the agent at the start of game, i need a copy of the tree without any reference to the original graph (both graph and nodes are scriptable objects). that is why i thought a deep copy is needed. would create Instance of the graph itself give me a copy like that with 0 relation to the original graph? (think of 10 AI in the scene that has assigned the same behavior tree)

about the last question, i was thinking more toward things like Inventory, game settings and stuff like that because showing the data in debugging time for these things are easier on a scriptable object than base c# objects unless i would go for custom editor windows (no big deal for creating custom windows but if you can just select the scriptable object, you can not only see the result, but for a debug scene all you would need is to read info from the scriptable object without a need for many of the other systems to be there). in this regard, i think, for these type of things, a custom serializer to collect info from the inventory and it's items, for example hand in hand with a deserializer for loading is needed? to manually assign data back to the newly generated scriptable objects? this can work like a deep copy, i mean, you can write serialization/deserialization per scriptable object instance and it would simply return for example json text or get a jason text and return a new instantiated scriptable object with data?

sigh, this is exactly why i was having hope on those two first solutions. but your answer is explanatory enough to know that the chances are really high that it wouldn't work in most cases.