How to use JSON.net to deserialize into a scriptable object

I am using JSON.net to serialize my scriptable object.

I am not using JSONutility because my scriptable object is nested with references and dictionaries which JsonUtility does not support.

While serializing the SO works well, I am unable to deserialize the json string and load it back into the SO.

Running the code below gives a warning saying SO must be created using CreateInstance instead of new and the data in the SO is not updated.

Code (CSharp):

1. data = JsonConvert.DeserializeObject<ScriptableObject>(jsonstring, settings);

Running the code below, gives a JSON serialization exception since it is unable to create an instance of a type which is an abstract class in the nested SOs

Code (CSharp):

1. JsonConvert.PopulateObject(jsonstring, data);

This is the full error:

Code (CSharp):

1. JsonSerializationException: Could not create an instance of type Raraland.BaseCategory. Type is an interface or abstract class and cannot be instantiated.

I cannot use JsonUtility.FromJsonOverwrite since any lists and nested scriptable objects in my json, don't get parsed on loading.

How do I deserialize json into my SO using JSON.net? Or is there any other json library that will allow this?

 

Not sure about serializing scriptable objects in particular, but serializers by default don't handle polymorphism. You have to tell them explicitly to do that, because it's more costly, and they may require additional information to keep the cost relatively low.
Typically that involves providing them with a list of known types, or handling conversions more explicitly.

The easiest way in Json.Net (which leads to larger file sizes, and potential security risks) is to tell it to store the full types. Play with the TypeNameHandling setting when serializing the files

.Auto - should provide explicit types only where it's needed (polymorphic objects)
.All - I think this option provides explicit types even when they aren't needed

Code (CSharp):

1.                     JsonSerializerSettings settings = new JsonSerializerSettings()
2.                     {
3.                         TypeNameHandling = Newtonsoft.Json.TypeNameHandling.Auto
4.                     };

There are other ways to do it in Json.Net as well. You can look them up.

 

Also, it's advisable to let unity serialize it's own weird types. If you're gonna serialize with JSON, use a plain old C# class.
ScriptableObjects are actually wrappers around native C++ objects. That's why they're created via ScriptableObject.Instantiate() and not via new(). JSON.NET can't now about this and sooner or later something will break.

If you want to have stuff both serialized with Jason and as assets, mark your custom class as [Serializable] and have scriptableObject that holds a reference to it. If you're on 2019.3+ , you can use [Serialize reference] for the added flexibility.

 

I actually did use the AutoSettings, forgot to include it in the code. It still however doesn't seem to work.

There are still some SO types which it warns about handling incorrectly i.e the new operator is being called on them instead of CreateInstance. These SO's are actually nested deep within the parent SO i'm trying to de/serialize

 

My current architecture is based on loading initial values from SOs. I then manipulate values in the SOs during gameplay and save then out to file as JSON. The next time the game loads I want to now fill the SO values from the saved json file.

Do you have any suggestions on what I can do instead?

SOs made creating and storing these initial game values a breeze.

 

Can you even serialize the most basic ScriptableObject using Json.Net?
Perhaps you can't, it's a unity type after all.

And by serialize, I also mean deserialize.

 

i am able to serialize

I am able to serialize it without any issues. Just unable to deserialize it.

Internally Json.net creates a new instance on deserialization whereas scriptable objects don't allow creation using the new keyword. Instead they need to use instantiate or createInstance.

That's where the problem lies.

 

Based on this post here https://stackoverflow.com/questions/42904122/unity-deserialize-into-scriptable-object
I figured json.net maybe has something similar. So I wonder if this may do the same thing as what they mention for jsonutility.

https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_JsonConvert_PopulateObject_1.htm

I did not test it as I'm currently in the middle of something else, just throwing this out there.

 

Yes I did try it but it doesn't work either. My SOs are deeply nested and this ends up giving me a warning about deeper nested SOs that they must be created using CreateInstance instead of new and the data in the SO is not updated.

 

What I did in my serialization tool ( I user Odin serializer instead of JSON.net so I could choose JSON for debugging purposes, and binary for speed in the final product, but I'm going off-topic) is the following:

Create an abstract class that has a GUID, and implements

ISerializationCallbackRecevied

so the GUID persists. Let's call that "Saveable". Mark this class with

[Serializable]

so you can save it along with any Unity type, be it MonoBehaviours, SOs, etc.

Then you create concrete classes to hold your data, for example

class SaveableTransform : Saveable {

       public Vector3 position, scale;

       public Quaternion rotation;

}

Now let's say you want to save this as part of your player data. In your

Player : Monobehaviour

class, add a variable of type

SaveableTransform

The second part of this is accessing your saved data. What I did was create a global manager using a

ScriptableObjectSingleton

, the code for which you can easily find on google. This

SaveableManager : ScriptableObjectSingleton

has a

Dictionary<GUID, byte[]>

that holds the GUID and saved data of all the

Saveable

objects. The object themselves are self-registering on startup.

This way, you provide a path to your saved data (I chose to have a .zip file with a lot of plain text files inside, each one named as the same as the GUID for easy lookup, but again this is for ease of debugging. For production you probably want a binary blob of data that can be accessed faster).

On application start, you load all the saved data to your dictionary and, when a

Saveable

is accessed, you go there to fetch your data and populate the object in a lazy manner. If the data is not found, you can fallback to the default values saved alongside your Monobehaviour/ScriptableObject.

This is a solution I've come along with recently, and is in no way battle-tested. If you need help PM me and I can send you my code.

Good luck!

 

Did you Solve it?

 

For anyone reading this, and in case it helps somehow. I must admit I did not read the full thread but if the issue is Unity Deserialize Into Scriptable Object the below link from S.O. might help.

https://stackoverflow.com/questions/42904122/unity-deserialize-into-scriptable-object

 

OP is asking about json.net, not the Unity native serializer

 

So basically I came across this and didn't find a answer that I found I liked, so I kinda F***ed around.

What I did was "Sister-classes" for the Scriptable objects.
Both the scriptable object and the class itself have a way to funnel data from one to the other.

Kinda like this

This is the class I convert to and from JSON essentially:

Code (CSharp):

1. using System.Collections.Generic;
2. using System.Linq;
3. using UnityEngine;
4. using Utility;
5.  
6. namespace Data.Serialize
7. {
8.     public class FunctionDataSerialize
9.     {
10.         public FunctionUtility.FunctionNames functionName;
11.         public int totalGamesPlayedWithThisFunction;
12.         public float totalTimePlayedWithThisFunction;
13.         public Vector2Int winLose = Vector2Int.zero;
14.         public float winLostRatioWithFunction;
15.         public int maxScoreWithFunction;
16.         public int minScoreWithFunction;
17.         public float averageScoreWithFunction;
18.         public List<int> totalScoreGathered;
19.  
20.         public FunctionDataSerialize()
21.         {
22.             functionName = FunctionUtility.FunctionNames.Abs;
23.             totalGamesPlayedWithThisFunction = 0;
24.             totalTimePlayedWithThisFunction = 0f;
25.             winLose = Vector2Int.zero;
26.             winLostRatioWithFunction = 0f;
27.             maxScoreWithFunction = 0;
28.             minScoreWithFunction = 0;
29.             averageScoreWithFunction = 0f;
30.             totalScoreGathered = new List<int>();
31.         }
32.  
33.         public FunctionDataSerialize(TetrisFunctionData data)
34.         {
35.             functionName = data.functionName;
36.             totalGamesPlayedWithThisFunction = data.totalGamesPlayedWithThisFunction;
37.             totalTimePlayedWithThisFunction = data.totalTimePlayedWithThisFunction;
38.             winLose = data.winLose;
39.             winLostRatioWithFunction = data.winLostRatioWithFunction;
40.             maxScoreWithFunction = data.maxScoreWithFunction;
41.             minScoreWithFunction = data.minScoreWithFunction;
42.             averageScoreWithFunction = data.averageScoreWithFunction;
43.             totalScoreGathered = data.totalScoreGathered.ToList();
44.         }
45.     }
46. }
47.  

And this is the main part of the ScriptableObject:

Code (CSharp):

1.     public class TetrisFunctionData : SerializedScriptableObject
2.     {
3.         public FunctionUtility.FunctionNames functionName;
4.         [SerializeField] public int totalGamesPlayedWithThisFunction;
5.         [SerializeField] public float totalTimePlayedWithThisFunction;
6.         [SerializeField] public Vector2Int winLose = Vector2Int.zero;
7.         [SerializeField] public float winLostRatioWithFunction;
8.         [SerializeField] public int maxScoreWithFunction;
9.         [SerializeField] public int minScoreWithFunction;
10.         [SerializeField] public float averageScoreWithFunction;
11.         [SerializeField] public List<int> totalScoreGathered;
12.  
13.         public void Setup(FunctionDataSerialize data)
14.         {
15.             functionName = data.functionName;
16.             totalGamesPlayedWithThisFunction = data.totalGamesPlayedWithThisFunction;
17.             totalTimePlayedWithThisFunction = data.totalTimePlayedWithThisFunction;
18.             winLose = data.winLose;
19.             winLostRatioWithFunction = data.winLostRatioWithFunction;
20.             maxScoreWithFunction = data.maxScoreWithFunction;
21.             minScoreWithFunction = data.minScoreWithFunction;
22.             averageScoreWithFunction = data.averageScoreWithFunction;
23.             totalScoreGathered = data.totalScoreGathered.ToList();
24.         }
25. .....

Then somewhere in the code I do the JSON magic:

Code (CSharp):

1.         public void Serialize() // send to DB
2.         {
3.             var jsonSerializerSettings = new JsonSerializerSettings();
4.             jsonSerializerSettings.Converters.Add(new Newtonsoft.Json.Converters.StringEnumConverter());
5.  
6.             var data = new PlayerSerialize(this);
7.         }

And this:

Code (CSharp):

1.             if (result.Data != null && result.Data.ContainsKey("Data"))
2.             {
3.                 var da = JsonConvert.DeserializeObject<PlayerSerialize>(result.Data["Data"].Value);
4.                 score = da.score;
5.                 playerName = da.playerName;
6.                 tetrisGameData.Setup(da.tetrisGameData);
7.              
8.                 Debug.Log("I am done with player setup");
9.             }

Note that this "da.tetrisGameData" is a ScriptableObject that has other ScriptableObjects in it. in that setup it loops through its list and calls their Setup() methods. And so on.
And of Course this PlayerSerialize is a Sister class here for the total info of the Player Data I hold (Which has SOs within SOs with lists of them....)

I am unsure if this is the best practice, but it is a workaround if you still wish to use the Newtonsoft Json and ScriptableObjects in runtime.

Hope this helps someone

 

Did you ever find a solution to this?

 

Did you ever find a solution to this?

 

Yup, I think having a "POCO sister object" which contains the same fields is probably the best method for now. If someone made code generation for this I could see that making things easier.

FWIW I also found this, but haven't tested it yet: https://gist.github.com/sebtoun/205d455bbb9fedc5d6b976c4e588c03f

 

No, not really. Json.NET has several different methods and constructs to deal with such things. First of all the equivalent method to Unity's FromJsonOverwrite would be PopulateObject as it was already mentioned in the top post. The issue he had was that he seems to have polymorphic classes which isn't supported by Unity's JsonUtility and of course creates issues with Json.NET as well. Polymorphic data can only be supported when the serializer stores metadata about the actual instance type.

Json.NET also has support for a CustomCreationConverter. So that converter would take care of creating the scriptable object. You could create a single implementation that could be used for any ScriptableObject.

Code (CSharp):

1. public class SOConverter<T> : CustomCreationConverter<T> where T : ScriptableObject
2. {
3.     public override T Create(Type aObjectType)
4.     {
5.         if (typeof(T).IsAssignableFrom(aObjectType))
6.             return (T)ScriptableObject.CreateInstance(aObjectType);
7.         return null;
8.     }
9. }
10.  

With that class you should be able to use DeserializeObject and provide the necessary converters.

Code (CSharp):

1. YourSOType obj = JsonConvert.DeserializeObject<YourSOType>(json, new SOConverter<YourSOType>(), new SOConverter<SomeOtherSubSOType>());

Here Json.NET should use our converter to actually create the SO instances. Our converter uses the proper CreateInstance call to create the instances. However in theory you could implement any kind of lookup / conversion / replacement with those custom converters. You may want to look into how those converters work. A converter can even customize the generated json and also apply some custom parsing to it. So it would be possible to store some kind of ID for certain SO assets and when deserializing, lookup those IDs in your own database and return the shared asset instead. It's a bit of work, but the library is quite powerful when utilized correctly.

To be honest, I have barely used the Newtonsoft Json.NET library as I wasn't in need for any more advanced serializing features. Everyone has to decide for themselfs which approach would fit the usecase best.

 

This is the way.

 

Incredible insight, thank you

 

This is the way.