Search Unity

JSON .NET for Unity

Discussion in 'Assets and Asset Store' started by Dustin-Horne, Sep 13, 2013.

  1. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Ahh sorry, it's a little bit different... the settings are on the serializer itself. I'll test it with the converters but here is an example of how to do it in a console application. In this particular sample I created a class with circular references and serialized it by setting up the properties on the JsonSerializer:

    Code (csharp):
    1.  
    2. public class Foo
    3. {
    4.     public int Value { get; set; }
    5.     public Foo Sibling { get; set; }
    6. }
    7.  
    8. static void Main(string[] args)
    9. {
    10.     var foo1 = new Foo
    11.     {
    12.         Value = 4
    13.     };
    14.  
    15.     var foo2 = new Foo
    16.     {
    17.         Value = 1,
    18.         Sibling = foo1
    19.     };
    20.  
    21.     foo1.Sibling = foo2;
    22.  
    23.     byte[] data;
    24.  
    25.     using (var ms = new MemoryStream())
    26.     {
    27.         using (var writer = new BsonWriter(ms))
    28.         {
    29.             var serializer = JsonSerializer.Create();
    30.                  
    31.             serializer.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
    32.             serializer.Serialize(writer, foo1);
    33.  
    34.             data = ms.ToArray();
    35.         }
    36.              
    37.     }
    38.  
    39.     using (var ms = new MemoryStream(data))
    40.     {
    41.         using (BsonReader reader = new BsonReader(ms))
    42.         {
    43.             JsonSerializer serializer = new JsonSerializer();
    44.             var newFoo = serializer.Deserialize<Foo>(reader);
    45.  
    46.             Console.WriteLine(newFoo.Value); //4
    47.             Console.WriteLine(newFoo.Sibling.Value); //1
    48.             Console.WriteLine(ReferenceEquals(null, newFoo.Sibling.Sibling)); //true (ignored)
    49.         }
    50.     }
    51.  
    52.     Console.ReadKey();
    53.  
    54.  
    55. }
    56.  
     
  2. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Here is the same example, but this time using ReferenceLoopHandling.Serialize and PreserveReferencesHandling.All which keeps track of references. You'll see that newFoo.Sibling.Sibling is no longer null because it keeps track of the references.. but I need to test this with structs.

    Code (csharp):
    1.  
    2. static void Main(string[] args)
    3. {
    4.     var foo1 = new Foo
    5.     {
    6.         Value = 4
    7.     };
    8.  
    9.     var foo2 = new Foo
    10.     {
    11.         Value = 1,
    12.         Sibling = foo1
    13.     };
    14.  
    15.     foo1.Sibling = foo2;
    16.  
    17.     byte[] data;
    18.  
    19.     using (var ms = new MemoryStream())
    20.     {
    21.         using (var writer = new BsonWriter(ms))
    22.         {
    23.             var serializer = JsonSerializer.Create();
    24.  
    25.             serializer.ReferenceLoopHandling = ReferenceLoopHandling.Serialize;
    26.             serializer.PreserveReferencesHandling = PreserveReferencesHandling.All;
    27.  
    28.             serializer.Serialize(writer, foo1);
    29.  
    30.             data = ms.ToArray();
    31.         }
    32.  
    33.     }
    34.  
    35.     using (var ms = new MemoryStream(data))
    36.     {
    37.         using (BsonReader reader = new BsonReader(ms))
    38.         {
    39.             JsonSerializer serializer = new JsonSerializer();
    40.             var newFoo = serializer.Deserialize<Foo>(reader);
    41.             serializer.ReferenceLoopHandling = ReferenceLoopHandling.Serialize;
    42.             serializer.PreserveReferencesHandling = PreserveReferencesHandling.All;
    43.  
    44.             Console.WriteLine(newFoo.Value); //4
    45.             Console.WriteLine(newFoo.Sibling.Value); //1
    46.             Console.WriteLine(ReferenceEquals(null, newFoo.Sibling.Sibling)); //false
    47.         }
    48.     }
    49.  
    50.     Console.ReadKey();
    51.  
    52.  
    53. }
    54.  
    55. public class Foo
    56. {
    57.     public int Value { get; set; }
    58.     public Foo Sibling { get; set; }
    59. }
    60.  
     
  3. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Here's an example... setting ReferenceLoopHandling to Ignore and using it with Structs instead of Classes (which mirrors Vectors.

    Code (csharp):
    1.  
    2.         static void Main(string[] args)
    3.         {
    4.             var foo1 = new Foo
    5.             {
    6.                 Value = 4
    7.             };
    8.  
    9.             byte[] data;
    10.  
    11.             using (var ms = new MemoryStream())
    12.             {
    13.                 using (var writer = new BsonWriter(ms))
    14.                 {
    15.                     var serializer = JsonSerializer.Create();
    16.  
    17.                     serializer.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
    18.                     serializer.PreserveReferencesHandling = PreserveReferencesHandling.Objects;
    19.  
    20.                     serializer.Serialize(writer, foo1);
    21.  
    22.                     data = ms.ToArray();
    23.                 }
    24.  
    25.             }
    26.  
    27.             using (var ms = new MemoryStream(data))
    28.             {
    29.                 using (BsonReader reader = new BsonReader(ms))
    30.                 {
    31.                     JsonSerializer serializer = new JsonSerializer();
    32.                     var newFoo = serializer.Deserialize<Foo>(reader);
    33.  
    34.                     Console.WriteLine(newFoo.Value); //4
    35.                     Console.WriteLine(newFoo.Sibling.Value); //1
    36.                 }
    37.             }
    38.  
    39.             Console.ReadKey();
    40.         }
    41.  
    42.         public struct Foo
    43.         {
    44.             public int Value { get; set; }
    45.  
    46.             public Foo Sibling { get { return new Foo { Value = 1 }; } }
    47.         }
    48.  
     
  4. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    One final example for your @SureSight and I think this is the path you want to take. Since you have to create a new instance of Serializer for BSON it doesn't use the "default converters". Instead you'll need to populate the Converters on the Serializer object. The below example just adds a new Vector3 converter and shows serialization of a Vector3. In your case you'll want to do this:

    Code (csharp):
    1.  
    2. serializer.Converters.AddRange(JsonConvert.DefaultSettings.Converters);
    3.  
    The above will take all of the default converters and put them into your converter list. But the below example shows it working with the Vector3 converter:

    Code (csharp):
    1.  
    2.             var vect = new Vector3(20.0f, 35.3f, 127.2f);
    3.  
    4.             byte[] data;
    5.  
    6.             using (var ms = new MemoryStream())
    7.             {
    8.                 using (var writer = new BsonWriter(ms))
    9.                 {
    10.                     var serializer = JsonSerializer.Create();
    11.  
    12.                     serializer.Converters.Add(new VectorConverter());
    13.  
    14.                     serializer.Serialize(writer, vect);
    15.  
    16.                     data = ms.ToArray();
    17.                 }
    18.  
    19.             }
    20.  
    21.             using (var ms = new MemoryStream(data))
    22.             {
    23.                 using (BsonReader reader = new BsonReader(ms))
    24.                 {
    25.                     JsonSerializer serializer = new JsonSerializer();
    26.                     serializer.Converters.Add(new VectorConverter());
    27.  
    28.                     var newVect = serializer.Deserialize<Vector3>(reader);
    29.  
    30.                     Console.WriteLine(newVect.x); //20
    31.                     Console.WriteLine(newVect.y); //35.3
    32.                     Console.WriteLine(newVect.z); //127.2
    33.                 }
    34.             }
    35.  
    36.             Console.ReadKey();
    37.  
     
    SureSight likes this.
  5. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    85
    Found a temporary solution. This only happens when trying to serialize / deserialize properties. The fix is to convert your properties to regular fields (remove {get;set;}). After that it works. Though we'd probably need a better, more long-term solution.
     
  6. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Are the properties that are erroring out all collections? Also could you please reproduce and send a bug report to Unity. This is a bug that they seem to fix and then it pops back up again in a later version.
     
  7. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    85
    Hmm, not sure if it was just collections, I converted all my models to regular fields and it worked. Will test it out and send a bug report to Unity & post it here.
     
    Dustin-Horne likes this.
  8. SureSight

    SureSight

    Joined:
    Aug 2, 2013
    Posts:
    65
    @Dustin-Horne The code for the default converters won't compile. See attachments for compile errors.
    Everything else is bang on.

    Many thanks.

    Code (csharp):
    1.  
    2. serializer.Converters.AddRange(JsonConvert.DefaultSettings.Converters);
    3.  
    [/QUOTE]
     

    Attached Files:

  9. KeeLo

    KeeLo

    Joined:
    Nov 1, 2013
    Posts:
    76
  10. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    What you want is JsonConvert.PopulateObject(jsonString, objectToOverwrite);

    There's also a second overload that accepts a JsonSerializerSettings instance as a third parameter.

    https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_JsonConvert_PopulateObject.htm
     
  11. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    [/QUOTE]

    Sorry, I forgot that's a Collection<T> and not a List<T>. You'll have to do something like:

    Code (csharp):
    1.  
    2. foreach(var converter in JsonConvert.DefaultSettings().Converters)
    3. {
    4.     serializer.Converters.Add(converter);
    5. }
    6.  
    Or you could just write an extension method to populate it for you.
     
    SureSight likes this.
  12. KeeLo

    KeeLo

    Joined:
    Nov 1, 2013
    Posts:
    76
    Hello! Do you support product? why asset is free?)))
     
  13. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Yes I still support it. :) It's free now because Google approached me and wanted to include it as part of their Poly API for Unity. So, I worked with them and made it free.
     
  14. KeeLo

    KeeLo

    Joined:
    Nov 1, 2013
    Posts:
    76
    That sounds good! Good luck ! Asset is very useful! Thank you!
     
    Dustin-Horne likes this.
  15. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    You're quite welcome. :)
     
  16. Karsnen_2

    Karsnen_2

    Joined:
    Nov 28, 2011
    Posts:
    89
    An Extreme Issue:

    Let's the file gets corrupted & the string has (let's say) 3 newline characters or System.String.Empty.

    Now I do the following

    Code (CSharp):
    1.  
    2.     currentDirectoryInformation = JsonConvert.DeserializeObject<JSONEntities.DirectoryInformation> (_jsonBuffer);
    3.  
    4.                 Debug.Log (currentDirectoryInformation.FileNames.Count);
    5.  
    6.                 if (currentDirectoryInformation.FileNames.Contains (fileName) == true)
    7.                 {
    8.                     currentDirectoryInformation.FileNames.Remove (fileName);
    9.                     currentDirectoryInformation.Count.FileCount--;
    10.                 }
    11.  
    The above results in

    Object reference not set to an instance of an object


    on
    Debug.Log (currentDirectoryInformation.FileNames.Count);



    My Question is, let's say I roll this for production, what is the best way to handle it?

    Thanks,
    Karsnen

    Code (CSharp):
    1.     [System.Serializable]
    2.     public sealed class DirectoryInformation // from which currentDirectoryInformation is init
    3.     {
    4.         public ContentCount Count;
    5.         public System.Collections.Generic.List<string> FolderNames;
    6.         public System.Collections.Generic.List<string> FileNames;
    7.    
    8.         public string CreationTime;
    9.         public string LastTouchedTime;
    10.  
    11.         public DirectoryInformation()
    12.         {
    13.             Count = new ContentCount ();
    14.             FolderNames = new System.Collections.Generic.List<string> ();
    15.             FileNames = new System.Collections.Generic.List<string> ();
    16.         }
    17.     }
     
  17. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I'm not sure I follow. You mean you want to know what to do if your file is corrupt and invalid?
     
  18. Karsnen_2

    Karsnen_2

    Joined:
    Nov 28, 2011
    Posts:
    89
    Hello Dustin,

    Let me elaborate.

    Scenario :
    1. I save JSON strings in a file and then use them on demand
    2. I check if that JSON file exists then extract the string from that file to

     
    currentDirectoryInformation = new JSONEntities.DirectoryInformation ();
    if (FileHelper.FileExists (relativePathToMetaFile) == true)
    {
    string _jsonBuffer = System.String.Empty;
    _jsonBuffer = FileHelper.ReadJsonFromFile (relativePathToMetaFile);
    currentDirectoryInformation = JsonConvert.DeserializeObject<JSONEntities.DirectoryInformation> (_jsonBuffer);
    Debug.Log (currentDirectoryInformation.FileNames.Count);

    if (currentDirectoryInformation.FileNames.Contains (fileName) == true)
    {
    currentDirectoryInformation.FileNames.Remove (fileName);
    currentDirectoryInformation.Count.FileCount--;
    }
    }


    At this time, now if the string (_jsonBuffer) has an empty string or is corrupted - then the statement
    Debug.Log (currentDirectoryInformation.FileNames.Count);


    throws the following error


    NullReferenceException: Object reference not set to an instance of an object

    The above error is thrown for
    Debug.Log (currentDirectoryInformation.FileNames.Count);

    statement.

    I'm curious as to why an error is thrown when the class is being initialized properly from the statement

    currentDirectoryInformation = new JSONEntities.DirectoryInformation ();



    The below class is the class, I intend to extract information.
    Code (CSharp):
    1.     [System.Serializable]
    2.     public sealed class DirectoryInformation
    3.     {
    4.         public ContentCount Count;
    5.         public System.Collections.Generic.List<string> FolderNames;
    6.         public System.Collections.Generic.List<string> FileNames;
    7.    
    8.         public string CreationTime;
    9.         public string LastTouchedTime;
    10.  
    11.         public DirectoryInformation()
    12.         {
    13.             Count = new ContentCount ();
    14.             FolderNames = new System.Collections.Generic.List<string> ();
    15.             FileNames = new System.Collections.Generic.List<string> ();
    16.         }
    17.     }

    I guess I'm clear this time.

    I highly appreciate your help. Thanks.
     
  19. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Oh, you're calling DeserializeObject but the Json string is empty so it's returning null, making your whole object null.

    To avoid that, use JsonConvert.PopulateObiect inetead which will attempt to Deserialize the string on top of your existing object.
     
  20. Karsnen_2

    Karsnen_2

    Joined:
    Nov 28, 2011
    Posts:
    89
    Thank you Dustin.

    I think that helped but if this error is returned -


    JsonSerializationException: No JSON content found. Path '', line 0, position 0.


    What is the best approach? I'm just trying to create a failsafe if the file is corrupted. Please advise me, if I have to take another approach.

    I highly appreciate your time, Dustin.

    Thanks.
     
  21. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Code (csharp):
    1.  
    2. if(!string.IsNullOrWhitespace(_jsonBuffer))
    3.     //populate object
    4.  
     
  22. Karsnen_2

    Karsnen_2

    Joined:
    Nov 28, 2011
    Posts:
    89
    Thank you Dustin.
     
    Dustin-Horne likes this.
  23. AlbinZosimusJulius

    AlbinZosimusJulius

    Joined:
    Nov 21, 2015
    Posts:
    7
    Hi Dustin, first of all thank you for this great asset!
    I have a problem using JSON.NET in a WebGL build. I send small json data packets from the browser (multiple times per second) to Unity using 'SendMessage'. There I read the data manually by using the JsonTextReader:
    Code (CSharp):
    1.  
    2. using(var sr = new StringReader(json))
    3. using(var reader = new JsonTextReader(sr))
    4. {
    5.     while(reader.Read())
    6.     {
    7.         if((reader.TokenType == JsonToken.PropertyName) && (reader.Depth == 1))
    8.         {
    9.             switch(reader.Value.ToString())
    10.             {
    11.                 case "action":
    12.                     blabla = reader.ReadAsString();
    13.                     break;
    14.  
    15.                 case "uid":
    16.                     uid = reader.ReadAsString();
    17.                     break;
    18.  
    19.                 // four other cases...
    20.             }
    21.         }
    22.     }
    23. }
    But after some time I will eventually face a WebGL "Out of memory!" error. It seems that the instantiation of text readers or something will produce a lot of garbage that will overflow the allocated memory in the browser. Here's the stack trace from the browser console:
    Code (csharp):
    1. Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 268435456, (2) compile with -s ALLOW_MEMORY_GROWTH=1 which adjusts the size at runtime but prevents some optimizations, (3) set Module.TOTAL_MEMORY to a higher value before the program runs, or if you want malloc to return NULL (0) instead of this abort, compile with -s ABORTING_MALLOC=0
    2.  
    3. uncaught exception: abort("Cannot enlarge memory arrays. Either (1) compile with  -s TOTAL_MEMORY=X  with X higher than the current value 268435456, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which adjusts the size at runtime but prevents some optimizations, (3) set Module.TOTAL_MEMORY to a higher value before the program runs, or if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 ") at jsStackTrace@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:1154:12
    4. stackTrace@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:1168:11
    5. abort@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:19862:43
    6. abortOnCannotGrowMemory@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:1210:2
    7. enlargeMemory@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:1213:2
    8. _sbrk@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1613815:1
    9. _GC_unix_sbrk_get_mem@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1661116:1
    10. _GC_expand_hp_inner@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1140644:1
    11. _GC_collect_or_expand@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1059476:1
    12. _GC_alloc_large@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1444558:1
    13. _GC_generic_malloc@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1398371:1
    14. _GC_malloc_atomic@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1593790:1
    15. __ZN6il2cpp2vm6Object15AllocatePtrFreeEjP11Il2CppClass [il2cpp::vm::Object::AllocatePtrFree(unsigned int, Il2CppClass*)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1568980:1
    16. __ZN6il2cpp2vm5Array11NewSpecificEP11Il2CppClassj [il2cpp::vm::Array::NewSpecific(Il2CppClass*, unsigned int)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1234718:1
    17. __Z10SZArrayNewP11Il2CppClassj [SZArrayNew(Il2CppClass*, unsigned int)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1749523:1
    18. _BufferUtils_RentBuffer_m2229979349@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1598652:1
    19. _JsonTextReader_EnsureBuffer_m4043226489@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1687525:1
    20. _JsonTextReader_Read_m2461759731@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:848925:1
    21. dynCall_iii@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1789585:1
    22. invoke_iii@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:16480:10
    23. _APIData_PopulateFromJSON_m2325856188@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:231577:1
    24. _AirConsoleAgent_ProcessData_m3389771770@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:688831:1
    25. _VolplaneController_ProcessData_m1038396144@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1730861:1
    26. __Z45RuntimeInvoker_Void_t1185182177_RuntimeObjectPFvvEPK10MethodInfoPvPS4_ [RuntimeInvoker_Void_t1185182177_RuntimeObject(void (*)(), MethodInfo const*, void*, void**)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1725810:1
    27. dynCall_iiiii@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1752953:1
    28. invoke_iiiii@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:16048:10
    29. __ZN6il2cpp2vm7Runtime6InvokeEPK10MethodInfoPvPS5_PP15Il2CppException [il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1424762:1
    30. _il2cpp_runtime_invoke@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1668541:1
    31. __Z23scripting_method_invoke18ScriptingMethodPtr18ScriptingObjectPtrR18ScriptingArgumentsP21ScriptingExceptionPtrb [scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1534593:1
    32. __ZN19ScriptingInvocation6InvokeEP21ScriptingExceptionPtrb [ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1042699:1
    33. __ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtrP21ScriptingExceptionPtr [MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingExceptionPtr*)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:260511:1
    34. __ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtr [MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodPtr, ScriptingObjectPtr)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1013842:1
    35. __ZN9Scripting20SendScriptingMessageER10GameObjectPKc18ScriptingObjectPtr [Scripting::SendScriptingMessage(GameObject&, char const*, ScriptingObjectPtr)]@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:982724:1
    36. _SendMessageString@blob:http://localhost:7860/21dcbdcf-0f46-4247-b4de-cbb54b245cbb:1370316:1
    37. UnityLoader["8b0b7c8bcbfc7a9b0673754cb4c9e96b"]/asm._SendMessageString@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:19199:9
    38. ccallFunc@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:740:13
    39. SendMessage@blob:http://localhost:7860/4bef3bd2-f4fe-43f7-b36f-ad3fbebaa529:343:144
    40. SendMessage@http://localhost:7860/build/Build/UnityLoader.js:275:18
    41. Agent.prototype.sendToUnity@http://localhost:7860/build/javascript/unity-airconsole-agent.js:353:17
    42.  
    (The VolplaneController / AirConsoleAgent objects don't do anything here but calling the method APIData.PopulateFromJSON() which reads the json)

    What is the best way of deserializing incoming json data packets that operates as memory friendly as possible? The data will sometimes be sent multiple times per second and control of this behavior is out of my hands. So there is no way for me of changing this frequency.
    Is it better to just use the deserializing method?
    I am using Unity 2017.3.1f1 and the latest JSON.NET version on the asset store.

    Some insights are greatly appreciated!
    Julian
     
  24. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I'm not sure if there will be a more efficient way. You may have to keep track of the number of times you've called this method and manually call a GC.Collect(true) every so many iterations to get the heap cleaned up. WebGL isn't as good at managing memory. I would start, though, by following those instructions at the top of the exception you're getting and play around with the memory values to see if you can find a larger one that works for you.
     
  25. Laicasaane

    Laicasaane

    Joined:
    Apr 15, 2015
    Posts:
    361
    Hi, I have some classes like these:
    Code (CSharp):
    1.  
    2. public class ControlProperties
    3. {
    4.     public Guid id;
    5.     public string name;
    6.     public Type controlType;
    7.     // Control's type: BaseControl, ObjectControl, TextControl, ImageControl
    8. }
    9.  
    10. public class ObjectProperties : ControlProperties
    11. {
    12.     public Vector3 position;
    13.     public Vector3 rotation;
    14. }
    15.  
    16. public class TextProperties : ObjectProperties
    17. {
    18.     public string content;
    19.     public Color fontColor;
    20.     public float fontSize;
    21. }
    22.  
    23. public class ImageProperties : ObjectProperties
    24. {
    25.     public bool isUniform;
    26.     public float uniformScale;
    27.     public Vector2 nonUniformScale;
    28. }
    Currently the controlType field is serialized into this
    Code (CSharp):
    1. MyGame.TextControl, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
    Is there anyway to remove the assembly info so the result could be just MyGame.TextControl?
     
  26. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    JsonSerializerSettings.DefaultSettings().TypeNameAssemblyFormat = AssemblyFormatterStyle.Simple;
     
  27. Laicasaane

    Laicasaane

    Joined:
    Apr 15, 2015
    Posts:
    361
    Yes, I did try that before asking the question here. The result is the same. :(

    "$type":"MyGame.TextProperties, Assembly-CSharp"
    "controlType":"MyGame.TextControl, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null"
     
  28. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Json .net is only writing the $type property. Where is "controlType" coming from? The simple output writes the namespace, type and assembly name so it knows where to find it. That's the shortest it's going to get.
     
  29. Laicasaane

    Laicasaane

    Joined:
    Apr 15, 2015
    Posts:
    361
    The "controlType" is a field in my example class in the #1625 post (public Type controlType). I put the 2 fields there just for comparision. I mean, when I use the AssemblyFormatterStyle.Simple setting, it does make the "$type" field short. But that field has nothing to do with me, it's the data for JsonConverter to use. What I care is the "public Type controlType", which I rely on to do other things, and I just want it to be short: it should be serialized into, for example, "MyGame.TextControl", without a bunch of assembly info behind.

    So the question is: Could I do that through Settings only? Or must I write a custom serializer? Thanks :)
     
  30. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    ahhh see. You'll have to write a custom JsonConverter for "Type" to control how it serialized and deserializes.
     
    Laicasaane likes this.
  31. DMCwildcard

    DMCwildcard

    Joined:
    Oct 18, 2017
    Posts:
    7
    hello,
    i have some problems with IList,....
    is this a known bug?
     
  32. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    What problem exactly? If your type is IList, the deserializer won't know what concrete type to actually instantiate. You'll need to use TypeNameHandling to store the type name in the serialized json. If it's an issue of not being able to construct the list (getting an exception where it can't find the constructor), then the issue is with types being stripped by IL2CPP. Your easiest option with collections is probably to use the actual List<T> concrete type rather than IList.
     
  33. layola

    layola

    Joined:
    Aug 6, 2013
    Posts:
    94
    hi,I got a strange bug.
    it's work on other platform well,but in ios.it's throw some error like
    ----------------
    Unable to find a constructor to use for type ServerJson. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, System.String id) [0x00000] in <filename unknown>:0
    ---------------

    I check my SeverJson,and it's have no constructor,but work on android or other platform.
    how to fixed that.
     
  34. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Most likely you're not constructing this class anywhere manually. Since JSON .NET does it via reflection, the Unity stripping engine (which is always on to some degree for IL2CPP) will sometimes strip the constructor. It can also strip properties. The default parameterless constructor is implicit, but it'll still get stripped.

    There are a couple of options. You can add a parameterless constructor and put a [Preserve] attribute on it which will prevent Unity from stripping it, but it's still possible that Unity will strip getters and/or setters from your properties. I would recommend using a link.xml file to specify classes and/or namespaces that shouldn't be stripped. There is a link.xml file in the /JsonDotNet folder that I use to prevent stripping of important things that Json .NET uses. Don't modify that one as it'll get replaced with updates but you can create an additional link.xml file somewhere in your assets folder and use mine as an example of how to preserve classes and namespaces.

    As an additional note, I know it seems weird that it works on some platforms but breaks on iOS... even if you're using IL2CPP on Android, but for some reason the stripping engine is not consistent between platforms and will break or not break things differently.
     
  35. gemelo

    gemelo

    Joined:
    Apr 11, 2016
    Posts:
    3
    Are there any updates on this issue beside of the "not using properties" workaround? We ran into the same issue and avoiding all properties is not an option in our case. I can confirm that it doesn't only affect collection properties.

    Thanks

    Peer
     
  36. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    can you drop me an email? dustin at parentelement.com. I'll send you an updated build for .net 4.6 and we'll see if that will work. This issue has been happening with other assets as well I've seen. It's a regression bug in Unity that got reintroduced with the .net 4.6 upgrade, but I might be able to work around it.
     
  37. dfraska

    dfraska

    Joined:
    Mar 1, 2018
    Posts:
    3
    Is this fix available in a build yet? The dates on the Asset Store for 2.0.1 don't seem to line up with this thread so I'm not sure how old this is.
     
  38. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    It did not as I haven't pushed a 2.0.2 version. I'd only had the single request for that change and I've been working on bigger changes such as .NET 4.6 support across platforms.
     
    Laicasaane likes this.
  39. Laicasaane

    Laicasaane

    Joined:
    Apr 15, 2015
    Posts:
    361
    I have to switch to the full .NET 4.x profile when building for Windows standalone. Will the upcoming version support .NET Standard 2.x?
     
  40. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Drop me an email and I'll send you an update to test.
     
  41. Laicasaane

    Laicasaane

    Joined:
    Apr 15, 2015
    Posts:
    361
    Nah, I'm just asking about it for now. It's not something urgent atm. It would be nice if you have some ETA then I could put it on my team's backlog.
     
    Dustin-Horne likes this.
  42. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I'll try to do some additional testing over the next week and put together an ETA.
     
  43. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    For everyone wanting to use .NET 4.6, I've attached a package. It's marked experimental but is being used by several people. You have to do a couple of things to use it in multiple platforms (and the editor) for this package but I'll get updated packages that no longer require the change. Use this package and the following instructions if you're targeting .NET 4.6:

    I think the attached package has all of the platform DLLs but only the IL2CPP DLL is .NET 4.6. When you import, uncheck the other DLLs and only import the IL2CPP one. Then in the inspector, select the /jsondotnet/assemblies/il2cpp/newtonsoft.json.dll and change the mapping to All Platforms and save.

    I'm working on an update and going to see if I can tackle mixed platforms (.NET 3.5 vs .NET 4.6) with asmdefs. I'll update the GitHub repo with the new projects hopefully in the next 3 weeks and push to the asset store as well.
     

    Attached Files:

    SugoiDev, BBB_brunoma and Laicasaane like this.
  44. imajuk2

    imajuk2

    Joined:
    Jul 16, 2015
    Posts:
    3
    Hi,

    I have a question about deserialization with reference loop.
    In my sample code, I expect a result "me", but actually It's Null. Why?

    Code (CSharp):
    1.  
    2. public class MyClass
    3. {
    4.     public MyClass Friend;
    5.     public string Name { get; private set; }
    6.  
    7.     public MyClass(string name)
    8.     {
    9.         Name = name;
    10.     }
    11.  
    12.     override public string ToString()
    13.     {
    14.         return Name;
    15.     }
    16. }
    17.  
    18.  
    19. public class Sample : MonoBehaviour {
    20.         var c1 = new MyClass("me");
    21.         var c2 = new MyClass("you");
    22.         c1.Friend = c2;
    23.         c2.Friend = c1;
    24.         Debug.Log(c1.Friend.Friend); // output: "me"
    25.  
    26.         var s = JsonConvert.SerializeObject(c1, Formatting.Indented, new JsonSerializerSettings
    27.                 {
    28.                     PreserveReferencesHandling = PreserveReferencesHandling.Objects,
    29.                     TypeNameHandling = TypeNameHandling.All
    30.                 });
    31.  
    32.         Debug.Log(s); // the result of serialization looks good.
    33.         /**
    34.         {
    35.           "$id": "1",
    36.           "$type": "MyClass, Assembly-CSharp",
    37.           "Friend": {
    38.             "$id": "2",
    39.             "$type": "MyClass, Assembly-CSharp",
    40.             "Friend": {
    41.               "$ref": "1"
    42.             },
    43.             "Name": "you"
    44.           },
    45.           "Name": "me"
    46.         }
    47.         */
    48.  
    49.         var d = JsonConvert.DeserializeObject<MyClass>(
    50.                 s,
    51.                 new JsonSerializerSettings
    52.                 {
    53.                     TypeNameHandling = TypeNameHandling.All
    54.                 }
    55.             );
    56.  
    57.         Debug.Log(d.Friend.Friend); // but the output is  Null
    58. }
     
  45. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    You need to include ReferenceLoopHandling when you deserialize as well. Your JsonSerializerSettings when you deserialize only have TypeNameHandling specified and not ReferenceLoopHandling.
     
  46. imajuk2

    imajuk2

    Joined:
    Jul 16, 2015
    Posts:
    3
    Thank you for reply.

    I tried to add ReferenceLoopHandling, but the result is same.. Any idea?


    Code (CSharp):
    1. var d =
    2.     JsonConvert.DeserializeObject<MyClass>(
    3.         s,
    4.         new JsonSerializerSettings
    5.         {
    6.             TypeNameHandling = TypeNameHandling.All,
    7.             ReferenceLoopHandling = ReferenceLoopHandling.Serialize
    8.         }
    9.     );
    10.  
    11. Debug.Log(d.Friend.Friend); // but the output is  Null
     
  47. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I'm sorry... I was replying on my phone and I didn't pay attention. I meant to say PreserveReferencesHandling. That needs to be used on both Serialization and Deserialization.

    https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_PreserveReferencesHandling.htm
     
  48. imajuk2

    imajuk2

    Joined:
    Jul 16, 2015
    Posts:
    3
    I put PreserveReferencesHandling not only Serialization but also Deserialization, but didn't work.
    I can't solve the reference loop.

    maybe my code's something wrong.. but not sure what.

    Code (CSharp):
    1.  
    2. var c1 = new MyClass("me");
    3. var c2 = new MyClass("you");
    4. c1.Friend = c2;
    5. c2.Friend = c1;
    6. Debug.Log(c1.Friend.Friend); // output "me"
    7.  
    8. var s = JsonConvert.SerializeObject(c1, Formatting.Indented, new JsonSerializerSettings
    9.         {
    10.             PreserveReferencesHandling = PreserveReferencesHandling.Objects,
    11.             TypeNameHandling = TypeNameHandling.All
    12.         });
    13.  
    14. Debug.Log(s); // the result of serialization looks good.
    15. /**
    16. {
    17.   "$id": "1",
    18.   "$type": "MyClass, Assembly-CSharp",
    19.   "Friend": {
    20.     "$id": "2",
    21.     "$type": "MyClass, Assembly-CSharp",
    22.     "Friend": {
    23.       "$ref": "1"
    24.     },
    25.     "Name": "you"
    26.   },
    27.   "Name": "me"
    28. }
    29. */
    30.  
    31. var d =
    32.     JsonConvert.DeserializeObject<MyClass>(s, new JsonSerializerSettings
    33.         {
    34.             PreserveReferencesHandling = PreserveReferencesHandling.Objects,
    35.             TypeNameHandling = TypeNameHandling.All,
    36.         }
    37.     );
    38.  
    39. Debug.Log(d.Friend.Friend); // but the output is Null
    40.  
     
    Last edited: Jun 28, 2018
  49. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Hmm ok I'll have to dig in and take a look. I wonder if it's trying to deserialize the whole object tree before adding the object to the reference cache, so when it deserializes the first child and tries to deserialize the Friend property, the reference for the parent hasn't been added yet.
     
  50. WheresMommy

    WheresMommy

    Joined:
    Oct 4, 2012
    Posts:
    890
    Thanks for putting that package online, but now Unity does not seem to find the Framework at all? Can you help me out. I deleted the folder, imported your package and unchecked all other DLLs then the IL2CPP file.

    Assets/Scripts/DuelManager.cs(9,7): error CS0246: The type or namespace name `Newtonsoft' could not be found. Are you missing an assembly reference?