JSON .NET for Unity

@Justin,

Thanks for the great package.

I just saw that Json.NET official version is 6.01, which version are you using for this package.

 

*Dustin

You're welcome. It's actually a mishmash to be honest which is why I want to get the whole shebang updated. The primary package is using a Version 4 release but I don't remember which one off the top of my head... it's what I had to use initially to get the iOS support working within Unity. The problem with it is that particular version doesn't support multidimensional arrays, only jagged and some nice newer features have been added such as asynchronous serialization and deserialization.

In order to implement support for Windows Store Apps I had to use a newer version, however the editor uses Mono and Windows Store Apps don't which is why there is a completely separate version of JSON .NET inside the WinRT folder and that one is actually Version 5 Release 8 which was the newest when I converted it, so technically multidimensional arrays and async serialization / deserialization are supported in Windows Store Apps but not in the editor.

Version 6 was just released and I want to try to update everything to that version at some point.

 

Wow, you had to do quite a bit of juggling with all of the different versions.

Would be grand to have the whole shebang upgraded.
Less variations to deal with and easier on gray matter.

 

It will certainly make things easier for me in the long run as well. It's very time consuming though. There are a lot of changes for iOS and WebPlayer to make them work properly. The architecture changed a lot between the v4 and v5 (and now v6) releases as well so I'll basically have to find the equivalent code and make all of my changes since a lot of it changed and moved to utility classes.

I'm waiting though for the next Unity release before I make the change. They shimmed in a bunch of stuff for Windows Store Apps and Windows Phone... the problem is that JSON .NET already assumes those reflection objects / methods don't exist and codes around them. Unfortunately, they completely broke it for Windows Phone 8 and the 4.3.3 and 4.3.4 releases because they broke some very necessary methods in System.Type and added BindingFlags (which already existed) so I have no way to properly test an updated version for WP8 without using a workaround so I'm waiting for the update. (They've said the problem is fixed and will be in the next non 4.3.x update)

 

Just wondering about Json.net performance.

It would be nice to have some metrics for Json.NET. Also be good to see how various versions of Json.NET perform.


http://theburningmonk.com/2011/08/performance-test-binaryformatter-vs-protobuf-net/

http://simplespeedtester.codeplex.c...dTester.Example/BinarySerializersSpeedTest.cs

 

Hi Dustin, do you have any updates on this? It looks like the next update will be 4.5 - are you on the beta program?

 

This is on my todo list. The real issue is that performance is going to differ wildly based on a lot of factors, such as what platform you are using, the size and type of data you're serialize / deserializing (i.e. reference vs value types and the affect on garbage collection) and whether you're using JSON or BSON. I'm going to develop some performance stress tests once I get everything updated to Version 6 of Newtonsoft.

Unfortunately I don't have access to the beta. I won't even begin work on the conversion until 4.5 is released because I'm not positive what all will change. The conversion itself will take some time so I don't really have an ETA. The most difficult bit will be troubleshooting and testing all of the AOT issues that occur on iOS.

 

Hi Dustin. Thanks for the great work on your port of JsonDotNet. I am currently running v1.3.3 download from the Asset Store.

I've unfortunately ran into a few AOT errors on iOS. I'll talk about the first issue here, and follow up in another post for the other.

I've found a workaround for the below issue, and I'll post the general idea here, hopefully it will help others. Or, if I'm missing something obvious, and my workaround is unnecessary, I sure would like to know!

Generally, it seems that if I am deserializing generic (List<T>, Dictionary<K,V>, where the T or K is a enum) objects that are never created in code, the AOT compiler doesn't generate constructors. I've managed to work around this issue by explicitly calling the constructor. For example:

Code (csharp):

1.  
2. public enum FooType
3. {
4.     foo,
5.     bar
6. }
7.  
8. public class TestClass
9. {
10.     public static string testData = @"
11. {
12.     'myList': [ 'foo', 'bar' ],
13.     'myDict': { 'foo': 1, 'bar': 2 }
14. }";
15.     List<FooType> myList;
16.     Dictionary<FooType, string> myDict;
17.  
18.     public static TestClass Create( string data )
19.     {
20.         var j = JsonConvert.DeserializeObject<TestClass> (data);
21.         Debug.Log (JsonConvert.SerializeObject (j));
22.         return j;
23.     }
24. }
25.  
26.  

calling TestClass.Create();

will generate the following error:

Code (csharp):

1. ExecutionEngineException: Attempting to JIT compile method 'System.Collections.Generic.List`1<FooType>:.ctor ()' while running with --aot-only.
2.  
3.   at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0
4. Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
5.   at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0
6.   at System.Reflection.MonoCMethod.Invoke (BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0
7.   at System.Reflection.ConstructorInfo.Invoke (System.Object[] parameters) [0x00000] in <filename unknown>:0
8.   at System.Activator.CreateInstance (System.Type type, Boolean nonPublic) [0x00000] in <filename unknown>:0
9.   at System.Activator.CreateInstance (System.Type type) [0x00000] in <filename unknown>:0
10.   at Newtonsoft.Json.Utilities.CollectionUtils.CreateAndPopulateList (System.Type listType, System.Action`2 populateList) [0x00000] in <filename unknown>:0
11.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateAndPopulateList (Newtonsoft.Json.JsonReader reader, System.String reference, Newtonsoft.Json.Serialization.JsonArrayContract contract) [0x00000] in <filename unknown>:0
12.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String reference) [0x00000] in <filename unknown>:0
13.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue) [0x00000] in <filename unknown>:0
14.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueProperty (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonProperty property, System.Object target, Boolean gottenCurrentValue, System.Object currentValue) [0x00000] in <filename unknown>:0
15.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00000] in <filename unknown>:0
16.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, System.String id) [0x00000] in <filename unknown>:0
17. UnityEngine.GameObject:SendMessage(String, Object, SendMessageOptions)
18. UICamera:Notify(GameObject, String, Object)
19. UICamera:ProcessTouch(Boolean, Boolean)
20. UICamera:ProcessTouches()
21. UICamera:Update()
22.  

So that's the problem, and the work around is to create a class method (or extension method), that looks something like this:

Code (csharp):

1.    
2. public static class TestClassExt
3. {
4.     private static void __hintAOT( this TestClass pThis )
5.     {
6.         pThis.myList = new List<FooType>();
7.         var dummy1 = pThis.myList.Count;
8.         pThis.myDict = new Dictionary<FooType, string>();
9.         var dummy2 = pThis.myDict.Count;
10.     }
11. }
12.  

This seems to be enough to hint the AOT compiler that it needs to generate the constructor code, and after that everything works fine. I've actually cooked up an editor script that walks my assembly namespace and generates the __hintAOT methods for each class it encounters.

So, is this normal? Is there a better way?

Thanks,

Travis

 

Hi Dustin, this is my other issue with JsonDotNet 1.3.3

There are issues that only happen when I'm using it as a dll. Compiled in VS 2013, targeting .Net FW 3.5

I'm pretty sure this is compatible with Unity's flavour of Mono as it works fine under Windows. Not sure about Android. On iOS however, I've run into a couple of problems.

The first was just a code stripping problem that for some reason only happens when using as a dll - I solved that by adding some lines to link.xml

The next problem has me stuck. If I take the same code from the previous post:

Code (csharp):

1.  
2. public enum FooType
3. {
4.     foo,
5.     bar
6. }
7.  
8. public class TestClass
9. {
10.     public static string testData = @"
11. {
12.     'myList': [ 'foo', 'bar' ],
13.     'myDict': { 'foo': 1, 'bar': 2 }
14. }";
15.     public List<FooType> myList;
16.     public Dictionary<FooType, string> myDict;
17.    
18.     public static TestClass Create( string data )
19.     {
20.         var j = JsonConvert.DeserializeObject<TestClass> (data);
21.         Debug.Log (JsonConvert.SerializeObject (j));
22.         return j;
23.     }
24. }
25. public static class TestClassExt
26. {
27.     private static void __hintAOT( this TestClass pThis )
28.     {
29.         pThis.myList = new List<FooType>();
30.         var dummy1 = pThis.myList.Count;
31.         pThis.myDict = new Dictionary<FooType, string>();
32.         var dummy2 = pThis.myDict.Count;
33.     }
34. }

I get this when I call TestClass.Create( TestClass.testData );

Code (csharp):

1. ExecutionEngineException: Attempting to JIT compile method '(wrapper dynamic-method) TestClass:CreateTestClass ()' while running with --aot-only.
2.  
3.   at System.Delegate.CreateDelegate (System.Type type, System.Object firstArgument, System.Reflection.MethodInfo method, Boolean throwOnBindFailure) [0x00000] in <filename unknown>:0
4.   at System.Delegate.CreateDelegate (System.Type type, System.Reflection.MethodInfo method, Boolean throwOnBindFailure) [0x00000] in <filename unknown>:0
5.   at System.Delegate.CreateDelegate (System.Type type, System.Reflection.MethodInfo method) [0x00000] in <filename unknown>:0
6.   at System.Reflection.Emit.DynamicMethod.CreateDelegate (System.Type delegateType) [0x00000] in <filename unknown>:0
7.   at Newtonsoft.Json.Utilities.DynamicReflectionDelegateFactory.CreateDefaultConstructor[Object] (System.Type type) [0x00000] in <filename unknown>:0
8.   at Newtonsoft.Json.Serialization.DefaultContractResolver.GetDefaultCreator (System.Type createdType) [0x00000] in <filename unknown>:0
9.   at Newtonsoft.Json.Serialization.DefaultContractResolver.InitializeContract (Newtonsoft.Json.Serialization.JsonContract contract) [0x00000] in <filename unknown>:0
10.   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateObjectContract (System.Type objectType) [0x00000] in <filename unknown>:0
11.   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract (System.Type objectType) [0x00000] in <filename unknown>:0
12.   at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract (System.Type type) [0x00000] in <filename unknown>:0
13.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.GetContractSafe (System.Type type) [0x00000] in <filename unknown>:0
14.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadForType (Newtonsoft.Json.JsonReader reader, System.Type t, Newtonsoft.Json.JsonConverter propertyConverter) [0x00000] in <filename unknown>:0
15.   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0
16.   at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0
17.   at Newtonsoft.Json.JsonSerializer.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0
18.   at Newtonsoft.Json.JsonConvert.DeserializeObject (System.String value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0
19.   at Newtonsoft.Json.JsonConvert.DeserializeObject[TestClass] (System.String value, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0
20.   at Newtonsoft.Json.JsonConvert.DeserializeObject[TestClass] (System.String value) [0x00000] in <filename unknown>:0
21.   at TestClass.Create (System.String data) [0x00000]  

Any thoughts?

Thanks,

Travis

 

That's actually normal and there isn't a better way.. but there are different ways. For instance, you could have a static class the instantiates all of the types you need and just let them go immediately out of scope, so when you startup your application you could execute it then do a GC.Collect to clean it all up. For example:

Code (csharp):

1.  
2. public static class AotHint
3. {
4.      public static void CreateHints()
5.      {
6.           new List<FooType>();
7.           new Dictionary<FooType, string>();
8.           GC.Collect();
9.      }
10. }
11.  

Then just call AotHint.CreateHints() somewhere when your game first starts. Now for the reasoning behind it...

In traditional .NET (and Mono), the entire Assemblies are JIT compiled so the CLR knows how to construct and execute everything. For iOS, it uses AOT for everything. The AOT compiler is pretty dumb and doesn't know how to construct certain via reflection unless it has already "seen" you do it in your code. So, if you have never created a new Dictionary<FooType, string> then it doesn't know how to execute the constructor for it. More specifically, it doesn't know how to inject the generic arguments.

You would think the declaration would be enough, but unfortunately it's not as the AOT compiler can't figure out the correct types to use when constructing the generic type. The same thing will happen with generic classes (for instance if you have MyClass<T>). But, once you have constructed them one time in your code and the AOT compiler has seen it... then it knows how to create those types later.

The AotHint method I showed above is very much like what you did, the difference is that it only constructs everything once and lets it go out of scope right away. Now you don't have to new those properties up on your classes every time you instantiate them (or deserialize them). That will save you from generating unnecessary garbage later on if those properties are really meant to be null.

I wish there was a better way to handle it but it's a limitation of the AOT compiler. Great work coming up with the solution on your own though! I'm glad you took the time to work it out.

 

I've not done it myself, but instead of instantiating empty objects (which has performance and code clarity issues) could one simply do the following?

Code (csharp):

1.  
2. public static class AotHint
3. {
4.      public static void CreateHints()
5.      {
6.           AddHint<List<FooType>>();
7.           AddHint<Dictionary<FooType, string>>();
8.      }
9.  
10.      public static void AddHint<T>() { }
11. }
12.  

------

Was working with one of my developers using iOS with the .net 2.0 subset. Turns out that this causes issues with polymorphism - specifically when using a class (e.g. dictionary, list, array) that are defined in mscorlib.

Solution? Define the dictionary/list in our own project

Code (csharp):

1.  
2. public Dictionary<string, Config.SaveData> SaveData; //Bad :(
3.  
4. public SaveData SaveData; //Good :)
5. public class SaveData : Dictionary<string, object> { }
6.  

Works even with full stripping

 

Great information guys, thanks! @npsf3000, yes that works as well as it gives the AOT compiler enough info to go off of. One thing I'm still looking at though is stripping in combination with using the .NET Subset as it appears to be creating some issues where there are references to Reflection.Emit. They shouldn't be required since it's not used in iOS or Android so I need to get those removed to make sure it is fully working with the .NET Subset as well as full stripping.

Thanks again for the info!

 

Just to clarify, I actually never call my hint method, therefore no need to worry about garbage. Just having the code in the assembly is seems to do the trick in mostcases.. However, I have found that there are some cases where just calling

Code (csharp):

1. new CustomGenericType()

is sometimes not enough.
For example:

Code (csharp):

1.  public class FooTypeDict : Dictionary<FooType, string> {}

write a function/method that calls

Code (csharp):

1. var container = new FooTypeDict

isn't quite enough, you also need to write a line of code that calls a method/property on the object like

Code (csharp):

1. var c = container.Count;

. But again, you still don't need to call the top level code, so this code never actually executes. it's just there for the AOT compiler.

 

Sorry to pester, but any thoughts on my DLL only issues?

http://forum.unity3d.com/threads/20...-AOT-Support?p=1558324&viewfull=1#post1558324

 

Yes, this is a similar reason though. It doesn't know how to create the Enumerator, so if you have the .Count method in your code it discovers how to create that enumerator.

If you're compiling to a DLL you're going to need to make sure you add the appropriate symbol in your compiler settings. If you're only targeting iOS you can do this by going to your project properties and making sure you add UNITY_IOS or UNITY_IPHONE before you compile. This will make sure the appropriate AOT corrected code for iOS gets compiled.

 

Right, that makes sense. What if I am also targetting android? It seems I would need 2 different versions of the assembly, but how can I tell Unity which assembly belongs to which project?

Thanks for the help!

Travis

 

I know Unity supports special folders for at least some platforms (like Windows Store) but I'm not sure if they do for all platforms. In those cases you can create DLLs for different platforms and put them in the corresponding folders but I'm not entirely sure how that works. It looks like they have ones for Android and iOS according to the Wiki so there are a couple of things you'd want to do...

First you'll want to compile a version for the editor and put it in the special Editor folder... then you'll want to do:

UNITY_IOS and compile for IOS, then UNITY_ANDROID to compile for Android and put it in the special Android folder. That way you have one version that will run on everything in the editor and one for each platform.

http://wiki.unity3d.com/index.php/Special_Folder_Names_in_your_Assets_Folder

Edit: That might only work for platform specific files and not DLLs, I'm not sure.

 

Yes, I'm not sure about that. I've done some experiments, and the Editor/Build system doesn't complain when you have multiple copies of the same Assembly dll in the project, but it's also unclear which one gets picked up for the actual build. Even if the dlls are in sibling folders eg

Code (csharp):

1.  
2. Assets/libs/JsonDotNet.dll
3. Assets/libs2/JsonDotNet.dll
4.  

the build doesn't complain.

It seems like it might be the classic, "Behaviour is undefined."

I've investigated further, and it appears that the using the plugins folder does not work (undefined at best).

Code (csharp):

1.  
2. Assets/plugins/iOS/JsonDotNet.dll
3. Assets/plugins/Android/JsonDotNet.dll
4.  

My iOS build appears to link against the Android version, without the AOT helper modules. I get a JIT exception consistent with what I saw before when trying to link the wrong dll.

Another approach could be an Editor script that intelligently renames the dlls (dll<->dlx) depending on the build target. I've seen a forum post suggesting this approach.

Given some of the issues, I think I will rethink my approach and have the JsonDotNet source in my project.

-----------------------------------------------------------------------------------------------

Moving along for now...

Is HashSet<T> still an issue on iOS? If I just compile with the UNITY_IPHONE flag set, and at that dll into the project, I still run into issues on iOS. This time it's an actual crash.

Code (csharp):

1.  
2. #0        0x007087b8 in m_Newtonsoft_Json_Utilities_CollectionUtils__c__DisplayClass6__CreateCollectionWrapperb__4_System_Type_System_Collections_Generic_IList_1_object at /xcode/Libraries/JsonDotNet.dll.s:43
3. #1        0x00713cf8 in m_Newtonsoft_Json_Utilities_ReflectionUtils_CreateGeneric_System_Type_System_Collections_Generic_IList_1_System_Type_System_Func_3_System_Type_System_Collections_Generic_IList_1_object_object_object__ at /xcode/Libraries/JsonDotNet.dll.s:43
4. #2        0x00707200 in m_Newtonsoft_Json_Utilities_CollectionUtils_CreateCollectionWrapper_object at /xcode/Libraries/JsonDotNet.dll.s:54
5. #3        0x00707f2c in m_Newtonsoft_Json_Utilities_CollectionUtils_CreateAndPopulateList_System_Type_System_Action_2_System_Collections_IList_bool ()
6. #4        0x006fdef4 in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_CreateAndPopulateList_Newtonsoft_Json_JsonReader_string_Newtonsoft_Json_Serialization_JsonArrayContract at /xcode/Libraries/JsonDotNet.dll.s:43
7. #5        0x006fcd7c in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_CreateList_Newtonsoft_Json_JsonReader_System_Type_Newtonsoft_Json_Serialization_JsonContract_Newtonsoft_Json_Serialization_JsonProperty_object_string at /xcode/Libraries/JsonDotNet.dll.s:43
8. #6        0x006fba64 in m_397 at /xcode/Libraries/JsonDotNet.dll.s:51
9. #7        0x006fb918 in m_396 at /xcode/Libraries/JsonDotNet.dll.s:43
10. #8        0x006fdbe4 in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_PopulateDictionary_Newtonsoft_Json_Utilities_IWrappedDictionary_Newtonsoft_Json_JsonReader_Newtonsoft_Json_Serialization_JsonDictionaryContract_string at /xcode/Libraries/JsonDotNet.dll.s:152
11. #9        0x006fc4b8 in m_399 at /xcode/Libraries/JsonDotNet.dll.s:205
12. #10        0x006fba34 in m_397 at /xcode/Libraries/JsonDotNet.dll.s:46
13. #11        0x006fb858 in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_CreateValueProperty_Newtonsoft_Json_JsonReader_Newtonsoft_Json_Serialization_JsonProperty_object_bool_object at /xcode/Libraries/JsonDotNet.dll.s:54
14. #12        0x006fd40c in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_SetPropertyValue_Newtonsoft_Json_Serialization_JsonProperty_Newtonsoft_Json_JsonReader_object at /xcode/Libraries/JsonDotNet.dll.s:136
15. #13        0x006ffabc in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_PopulateObject_object_Newtonsoft_Json_JsonReader_Newtonsoft_Json_Serialization_JsonObjectContract_string at /xcode/Libraries/JsonDotNet.dll.s:180
16. #14        0x006fe9fc in m_Newtonsoft_Json_Serialization_JsonSerializerInternalReader_CreateAndPopulateObject_Newtonsoft_Json_JsonReader_Newtonsoft_Json_Serialization_JsonObjectContract_string at /xcode/Libraries/JsonDotNet.dll.s:62
17. #15        0x006fc540 in m_399 at /xcode/Libraries/JsonDotNet.dll.s:215
18. #16        0x006fba34 in m_397 at /xcode/Libraries/JsonDotNet.dll.s:46
19. #17        0x006fb918 in m_396 at /xcode/Libraries/JsonDotNet.dll.s:43
20. #18        0x006fb39c in m_390 at /xcode/Libraries/JsonDotNet.dll.s:43
21. #19        0x006dbd94 in m_Newtonsoft_Json_JsonSerializer_DeserializeInternal_Newtonsoft_Json_JsonReader_System_Type at /xcode/Libraries/JsonDotNet.dll.s:43
22. #20        0x006dbd10 in m_Newtonsoft_Json_JsonSerializer_Deserialize_Newtonsoft_Json_JsonReader_System_Type at /xcode/Libraries/JsonDotNet.dll.s:43
23. #21        0x006d9cc0 in m_Newtonsoft_Json_JsonConvert_DeserializeObject_string_System_Type_Newtonsoft_Json_JsonSerializerSettings at /xcode/Libraries/JsonDotNet.dll.s:43
24. #22        0x006d9b5c in m_Newtonsoft_Json_JsonConvert_DeserializeObject_T_string_Newtonsoft_Json_JsonSerializerSettings at /xcode/Libraries/JsonDotNet.dll.s:43
25. #23        0x006d9aac in m_Newtonsoft_Json_JsonConvert_DeserializeObject_T_string at /xcode/Libraries/JsonDotNet.dll.s:43
26.  

It appears to crash when attempting to deserialize an object that contains a HashSet. I temporarily replaced the HashSet with a List<T> (not the same I know), and now I can run without a crash. Odd that this only happens when using a DLL though.

Thanks for your help! Dealing with iOS AOT issues is no fun, but you've been very helpful, and I've learned alot.

Travis

 

Yes, HashSet<T> is still an issue. I'm not exactly sure why it's only an issue on iOS but it should be fixed when I upgrade the whole solution to JSON .NET Verstion 6 which will be after unity 4.5 is released so I can work through all the Windows Phone and Windows Store changes that Unity has been making. It should also be a unified code base as well then instead of having an entirely separate Windows Store version in a subfolder.

The big issue with HashSet<T> is that the AOT compiler can't figure out what to do with it. That is partially due to the Mono version that's being used (and actually that it targets an older version of .NET) because HashSet<T> in .NET 4+ implements an interface that didn't exist in .NET 3.5 and below. I'm not sure how it works on Android and Mac but it does and it makes sense that it works on Windows Store and Windows Phone because it's a different .NET version.

 

What settings in link.xml are required to use this with build stripping? Everything works fine for us with no stripping but when I switch to "strip bytecode" I'm crashing here:

Code (csharp):

1. I/Unity   (32198): NullReferenceException: Object reference not set to an instance of an object
2. I/Unity   (32198):   at System.Reflection.MonoProperty.CreateGetterDelegate (System.Reflection.MethodInfo
3. method) [0x00000] in <filename unknown>:0
4. I/Unity   (32198):   at System.Reflection.MonoProperty.GetValue (System.Object obj, System.Object[] index)
5. [0x00000] in <filename unknown>:0
6. I/Unity   (32198):   at Newtonsoft.Json.Utilities.ReflectionUtils.GetMemberValue (System.Reflection.MemberInfo
7. member, System.Object target) [0x00000] in <filename unknown>:0
8. I/Unity   (32198):   at Newtonsoft.Json.Serialization.ReflectionValueProvider.GetValue (System.Object target)
9. [0x00000] in <filename unknown>:0
10. I/Unity   (32198): Rethrow as JsonSerializationException: Error getting value from 'DebugName' on
11. 'PlayerData'.
12. I/Unity   (32198):   at Newtonsoft.Json.Serialization.ReflectionValueProvider.GetValue (System.Object target)
13. [0x00000] in <filename unknown>:0
14. I/Unity   (32198):   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject
15. (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.Jso

 

edit: I didn't have this exact error, but this post helped me get up and running: http://forum.unity3d.com/threads/13...ompatibility?p=1167573&viewfull=1#post1167573

Code (csharp):

1.  
2. <linker>
3.  
4.        <assembly fullname="System">
5.  
6.             <type fullname="System.ComponentModel.TypeDescriptor" preserve="all"/>
7.  
8.             <type fullname="System.ComponentModel.TypeConverter" preserve="all"/>
9.  
10.             <type fullname="System.ComponentModel.StringConverter" preserve="all"/>
11.  
12.             <type fullname="System.ComponentModel.BooleanConverter" preserve="all"/>
13.  
14.             <type fullname="System.ComponentModel.EnumConverter" preserve="all"/>
15.  
16.             <type fullname="System.ComponentModel.Int64Converter" preserve="all"/>
17.  
18.             <type fullname="System.ComponentModel.Int32Converter" preserve="all"/>
19.  
20.             <type fullname="System.ComponentModel.Int16Converter" preserve="all"/>
21.  
22.             <type fullname="System.ComponentModel.ArrayConverter" preserve="all"/>
23.  
24.             <type fullname="System.ComponentModel.CollectionConverter" preserve="all"/>
25.  
26.             <type fullname="System.ComponentModel.DecimalConverter" preserve="all"/>
27.  
28.             <type fullname="System.ComponentModel.SingleConverter" preserve="all"/>
29.  
30.             <type fullname="System.ComponentModel.DoubleConverter" preserve="all"/>
31.  
32.        </assembly>
33.  
34. </linker>
35.  

 

Thanks tbj22. I was trying to figure this out myself as I'm not familiar with the different stripping levels. I was under the impression that Reflection (with the exception of Emit for dynamic code) worked fine but I hadn't thought about the ComponentModel stuff. I'm going through all of the classes and trying to make sure I know which classes are being used and I'm figuring out what assemblies they typically live in. Out of curiosity...

1. Are you using Assembly stripping or Byte Code stripping and Micro mscorlib?
2. Are you using .NET Full or 2.0 Subset (I'm guessing subset)?

And I'm assuming it's compiling and working with those settings...

 

Api Compatibility = 2.0 Subset
Stripping Level = Strip Bytecode

And yes, things are working fine with those settings. We tried micro mscorlib, but we can't even boot with that. Can't recall what the error was.

To be clear, I only needed the link.xml settings from above when we were trying to link JsonDotNet as a dll. If it's just built into the Assembly-CSharp (or FirstAssembly) we don't need any special link.xml settings.

AOT/Unity on iOS is fraught with peril! Hopefully our team is through the worst of it, but I'm really surprised it doesn't get talked about more!

Travis

 

Bytecode stripping should work, I'm not sure yet about micro mscorlib. It makes sense that including it as a separate DLL would be a problem. That being said, I think I know the problem with Micro mscorlib (maybe) and some of the other bytecode stripping issues which also cause an issue on Android. Another user helped me find something I missed... There are still references to Reflection.Emit even though they aren't being used for iOS / Android, so when the stripping occurs it's blowing some things up.

I'm in the process of stripping those out right now so I'll have a patch that should fix those issues.

 

Ok, I have a release candidate for version 1.3.5 if one of you wants to give it a shot.

 

Thank you. Unfortunately, I've tired this already. It doesn't work for us on Android - I get the same exception I mentioned previously.

Settings:

Stripping level - ByteCode
Api Compatability - .NET 2.0

I'm using the JSON.Net plugin as it is shipped in the asset store, not compiled as a DLL.

Sure, I'd be happy to give it a try if you think it might help with the stripping issue on Android.

 

I sent you a PM earlier. Could you PM me with your Skype?

 

An update should be forthcoming. The changes have been through some testing on Android and I still have some more in-depth testing to do. I'm also awaiting iOS testing though the same changes affected both platforms so I don't foresee any issues. So far so good and it's working with byte code stripping.

 

Version 1.3.5 has been submitted and is awaiting approval. In this update:

1. Stripping is now fully supported for iOS and Android
2. Android supports stripping to micro mscorlib
3. iOS supports stripping down to the bytecode level
4. Corrected a setting on the Polymorphism code sample
5. Fixed missing compiler directive for Windows Phone 8

With several folks needing the Bytecode stripping, I wanted to get this hotfix out. One important thing to note, micro mscorlib works on Android with JSON .NET but not on iOS. For iOS you can strip to the byte code level but cannot use micro mscorlib. This is because some critical functionality (Type.MakeGenericType for one) does not exist in the iOS micro mscorlib.

For those who purchased from Fast Spring, I'll get the new version uploaded within the next couple of hours and send out notification email.

 

Well needed asset! Having done a couple of "ports" of JsonFX, MiniJson and LitJson myself trying to get it working on all platforms, then trying to meet my data requirements, swithing to next one, etc, I am so happy it is finally someone else's headache and I can just use JSON.NET again.

The feature I missed in first 3 minutes after purchase is the proper support for things like VectorX or Quaternion. While doing own JsonConverter is straightforward, the reason for buying JSON.NET for Unity is actually stopping coding serialization bits like Vector3 JsonCoverters.

P.S. For other readers - this is not a negative opinion about the product, just a hint how to make a usefull asset even more useful.

Cheers and keep up the good work!

 

Thanks Tomas! I actually welcome all feedback and opinions as it helps me improve the product. If everyone had 100% positive things to say there would be no incentive for me to change anything.

Headache is right! However, we seem to have the functional issues all ironed out as for as bugs go which is great because it allows me to improve the featureset. For instance, there is an update awaiting approval on the asset store that adds stripping support for iOS and Android (and even micro mscorlib on Android).

Now, there is support for Vectors... I need to test Quanternion to make sure there aren't any issues there without a converter, but Vector2, 3 and 4 all should serialize and deserialize fine. They serialize everything that's public so the JSON gets a little chatty... that's when you would need a converter.

That being said, my next major update is an upgrade to JSON .NET Version 6. This adds all kinds of cool things like async serialization and deserialization and multiple content support for JSON (more than one object in the same byte stream). It also adds support for multidimensional arrays, not just jagged.

On top of the inherent goodies, I also plan to add more support for built-in objects like Vector's, Quanternions, etc. I also plan to create some of my own converters and include them with the package so you don't have to create your own (for instance, stripping down Vector3 to just serializing the x, y and z). And one of the biggest things I want to do is add support for GameObjects which is going to be a challenge because of the overridden equality checks in things like rigidBody.

One of the most challenging is going to be Textures. If you've seen a few posts back we have an example workaround for Texture2D though it's not ideal. Any feedback you have on how you'd like to see the built in objects handled is certainly welcomed.

 

Using v. 1.3.0 - when I try to build to Windows Phone 8, and get this error: Assets/JsonDotNet/Source/Serialization/JsonTypeReflector.cs(348,16): error CS0103: The name `LateBoundReflectionDelegateFactory' does not exist in the current context

Works fine in the editor, and on other platforms. Same error when I try to build the test scene, which also works fine in the editor

 

You should probably update to 1.3.3. Version 1.3.4 has been submitted to the asset store and is waiting approval. If you email me at the email address in the Readme file and send me a copy of your invoice I can send you the update. It was an error I made after adding WinRT support. If you want the quick fix, Open up the file:

JsonDotNet/Source/Utilities/LateBoundReflectionDelegateFactory.cs

And replace the very first line of the file with this one:

Code (csharp):

1.  
2. #if !UNITY_WINRT || UNITY_EDITOR || UNITY_WP8
3.  

I somehow missed the || UNITY_WP8 when I was converting that file.

 

How do I do to it not serialize one of my class fields?

I tried adding [NonSerialized] but it seems trying to serialize it anyway.

 

Try using the [JsonIgnore] attribute.

 

Hello Dustin and thanks for this excellent plug-in.

I've purchased it and integrated it into my project. PC / Mac standalone builds work fine. I've managed to build (but not test) a windows phone version. However, trying to build for Windows Store gives the following error -

Assets\JsonDotNet\Source\WinRT\Serialization\RT_DefaultContractResolver.cs(113,13): error CS0104: 'BindingFlags' is an ambiguous reference between 'System.Reflection.BindingFlags' and 'Newtonsoft.Json.Utilities.BindingFlags'

Thanks,

Jason Stark

 

Hey Jason, thank you for your purchase! I'm still waiting on Unity to approve my submission... I previously stated it was 1.3.4 but it's actually 1.3.5. It contains a fix for the Windows Phone 8 issue. When I added Windows Store support I flubbed a precompiler directive in one of the files (just missed part of it). See below for the quick and easy fix, otherwise you can drop me an email to the email address in the readme.txt file and send me your invoice and I'll hook you up with the latest update:

 

Version 1.3.5 is officially live on the Asset Store.

 

The date is wrong on the website.

 

Whoops, sorry that was a manual database entry typo by me. Thanks for pointing it out, it's been fixed.

 

I was wondering with the JSON we can load game data, and save game data with serialization/deserialization and reading/writing to a file, but if we want to have a feature where the player Deletes all their saved data, what would be the best way to reset all the data back to its default values? Is there an easy way to do so?

EDIT:
The solution was so obvious I've realized now I was overthinking the problem. All I had to do was create a new object of my game data class, then serialize that gameobject, which wipes the serialized data file to the default values.

 

Can you, via your asset, use remove (web based) JSON objects?

 

@Ben - Glad you figured it out. Yeah that's the simplest way and you can save it to file, or database using something like SQLite, or even save the JSON String to PlayerPrefs (although I would suggest using a file for it as player prefs isn't really made for saving large chunks of save data). Also, you might look into using BSON using the JSON .NET library instead of JSON if you don't need your save data to be human readable. It'll save it as binary data instead.

Fluid - I'm not quite sure what you mean. What is it you're wanting to remove?

 

I'm hitting the BindingFlags compilation issue with Windows Store apps. Copying the 4.3.2 WinRTLegacy.dll to the 4.3.4 folder didn't seem to do anything. Does that only work for WinPhone8, not Win8? If there's no Win8 workaround, will Unity still not fix it before 4.5.x? Or 5.x maybe?

FYI: The reason I'd like to upgrade from 4.3.2 is that Sprite.Create is throwing exceptions (same LOC, same texture, sometimes throws), and maybe it's fixed later (there's a similar bug to mine that may have been fixed).

 

Ted -

I just became aware of that issue the other day. Copying the old Legacy DLL doesn't fix the BindingFlags issue for Windows Store Apps. This is because WinRT for Windows Store doesn't have BindingFlags so JSON .NET adds in a fake class internally to use. Unity then added BindingFlags for Windows Store Apps so it doesn't know which to use. There is one file I need to update to fully qualify the namespace to use the JSON .NET version instead of the System.Reflection version provided by WinRT (so it stays backwards compatible). I'll get that fixed today and I'll post a link to download a patch (.unitypackage) here that has the updated file.

 

Dustin, I am sorry, I meant remote..

 

Oh then yes. As long as your web server produces standard JSON. You can use the WWW class to send and receive the JSON to/from the server.

 

Alright, I've created a patch to fix the BindingFlags compilation issue with WinRT and newer versions of Unity:

http://www.parentelement.com/downloads/JSONNETPATCH_1_3_5_2.zip

The zip file contains a .unitypackage with the corrected files. Just import the package and it will replace the correct files.

 

Excellent turnaround on that fix, Dustin!
I'll tell you if it doesn't work.

 

Hi,

First of all, awesome work with the plugin! Thanks so much! Totally worth its money! Amazing job on the AOT and iOS things. Works like a charm!

One question: how do you get referencing to work?

I'm having a hard time deserializing the following correctly:

[
{
"$id" : "infantry",
"linkToOtherUnit" : { "$ref" : "mech" }
},
{
"$id" : "mech",
"linkToOtherUnit" : null
},
{
"$id" : "other",
"linkToOtherUnit" : { "$ref" : "mech" }
}
]

When I deserialize this list of units, the first unit does not contain a link to the mech because the mech has not yet deserialized, but the third unit correctly references the mech because it has.

Clearly, this has to be a two step process, but I've been trying to make this work this entire weekend and cannot figure out how.

The closest I have come to is by reusing a custom Reference Resolver and deserializing the list of units twice, but that doesn't reference the correct instances of units. I've also tried extending the Contract Resolver but failed miserably. I was then going to use the following: http://stackoverflow.com/questions/...3113901?noredirect=1#comment34944625_13113901 but I realized that that uses the most recent version of Json.NET (this one does not include [JsonProperty(ItemConverterType = typeof(JsonRefConverter))]).

Anyway, I've been banging my head all weekend to figure this out. Keep in mind that I also need all the references to be preserved between multiple calls of JsonConvert.Deserialize because other data also might reference them. I.e.

{
"$id" : "small_map",
"unitLocations" : [
{
"location" : "1,1",
"unit" : { "$ref" : "mech" }
}
]
}

So either I have to use the same JsonSerializer, or have a JsonSerializerSettings that is reused between multiple calls.

Can you help me?

Thank you so much!