What is this symptom of? Newtonsoft.Json StackOverflowException

Code (CSharp):

1. Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b0] in <97722d3abc9f4cf69f9e21e6770081b3>:0
2.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x00057] in <97722d3abc9f4cf69f9e21e6770081b3>:0
3.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b0] in <97722d3abc9f4cf69f9e21e6770081b3>:0
4.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary (Newtonsoft.Json.JsonWriter writer, System.Collections.IDictionary values, Newtonsoft.Json.Serialization.JsonDictionaryContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x00149] in <97722d3abc9f4cf69f9e21e6770081b3>:0
5.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x0013a] in <97722d3abc9f4cf69f9e21e6770081b3>:0
6.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x00057] in <97722d3abc9f4cf69f9e21e6770081b3>:0
7.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b0] in <97722d3abc9f4cf69f9e21e6770081b3>:0
8.   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00047] in <97722d3abc9f4cf69f9e21e6770081b3>:0
9.   at Newtonsoft.Json.JsonSerializer.SerializeInternal (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00253] in <97722d3abc9f4cf69f9e21e6770081b3>:0
10.   at Newtonsoft.Json.JsonSerializer.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00000] in <97722d3abc9f4cf69f9e21e6770081b3>:0
11.   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal (System.Object value, System.Type type, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00028] in <97722d3abc9f4cf69f9e21e6770081b3>:0
12.   at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value, System.Type type, Newtonsoft.Json.Formatting formatting, Newtonsoft.Json.JsonSerializerSettings settings) [0x0000e] in <97722d3abc9f4cf69f9e21e6770081b3>:0
13.   at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value, Newtonsoft.Json.Formatting formatting, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <97722d3abc9f4cf69f9e21e6770081b3>:0

Unity doesn't just crash, it outright closes on me. The line that breaks it, which is the very first time it's triggered:

Code (CSharp):

1. string data = JsonConvert.SerializeObject(jsonify, Formatting.Indented, new JsonSerializerSettings { ReferenceLoopHandling = ReferenceLoopHandling.Ignore });

Before this line, I Debug.Logged, jsonify, which is an object. I try to turn class into JSON. `Debug.Log(jsonify);` names the expected class (BrigBlock.CMData), the expected class looks as follows:

Code (CSharp):

1. namespace BrigBlock
2. {
3.    public class CMData
4.    {
5.        public string myId;
6.  
7.        public byte[] soundData;
8.  
9.        public int sampleSize;
10.  
11.        public Dictionary<DataType, Color> albumsColors;
12.    }
13. }

I don't know why it fails, looks normal to me. It doesn't self-reference. myId is a string containing hash, soundData is read from disk using File.ReadAllBytes (absolutely unrelated to that application, it's a standalone file), sampleSize is set to 150, albumsColor is typical Dictionary.Add using DataType as Key and Color as Value. Loaded file is 279kB.

 

A stack overflow, in any context, is almost always the result of infinite recursion.

Why are you specifying

ReferenceLoopHandling = ReferenceLoopHandling.Ignore

if your data type is not self-referential? I bet if you removed that parameter the serializer will give you a different (and possibly more helpful) error.

 

Could you give me an example on when it isn't?

I forgot, and after removing it, I remembered, within this Dictionary there's a

UnityEngine.Color

, which causes this error:

JsonSerializationException: Self referencing loop detected for property 'linear' with type 'UnityEngine.Color'

This is why I used

ReferenceLoopHandling.Ignore

, it used to work before and render somehow limited although working Color JSON object, I believe it simply dropped variables that were too deep. Now it doesn't work at all.

I made a workaround by storing Color r, g, b in Vector3 but obviously that's not how it's supposed to be. If anybody reads it and you like a puzzle or have a solution on top of your head, please be my guest.

 

So newtonsoft just serializes every read/write property of the type being serialized. Unforunately unity has a bunch of useful read/write properties that are just helpers.

Usually you'd attribute properties that need properties ignored... but since you didn't write the 'Color' struct, you can't do this.

Also the ReferenceLoopHandling.Ignore isn't really going to fix your problem since Color isn't a reference, it's a value. And since Color.linear is a property of type Color, you're serializing a Color with a Color with a Color with a Color with a Color.... ad infinitum. (thusly infinite loop)

...

You'll likely have to write your own surrogate converter for it. In newtonsoft it's called a 'JsonConverter', see here:
https://www.newtonsoft.com/json/help/html/CustomJsonConverter.htm

Write a converter that handles the 'Color' type and only write the a,r,g,b values into the jsonwriter, ignoring all other properties.

 

Well, a stack overflow means that you're trying to store more data on the stack than will fit there.

Infinite recursion guarantees* this will happen eventually, because every time you call a function you have to add something to the stack, and something times infinity is always bigger than however big the stack is.

But it's also possible to get too big without actually being "infinite". You just have to be big enough. For example, a recursive function that counts from 1 to a trillion will cause a stack overflow, even though (on a computer with infinite memory) it would eventually terminate.

But modern stacks are so big that it's very unusual to get a stack overflow without going infinite.

Also, the "stack" is an abstraction, and abstractions are leaky. Theoretically something could happen on a lower level that would violate the abstraction and change your result without going through usual channels (e.g. someone waves a strong magnet over your computer).



*There's actually a neat exception called tail recursion, but it isn't used in C#.

 

For me it solved the issue when applying the additional settings like such:

Code (CSharp):

1. public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
2. {
3.     ....
4.     // Fix: Ignore loops
5.     serializer.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;

 

Oh wow, I didn't realize option existed in JSON.NET. I am SO using that from now on.