Search Unity

JSON .NET for Unity

Discussion in 'Assets and Asset Store' started by Dustin-Horne, Sep 13, 2013.

  1. imtrobin

    imtrobin

    Joined:
    Nov 30, 2009
    Posts:
    1,545
    Unity. It works fine on 5.1.3. It conflicts on other platforms, not just windows phone
     
  2. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Ugh, I haven't even had a chance to look yet. I'll be installing 5.2 tonight. I was aware of the Windows Universal / Windows 10 platform issues. I know there are other conflicts as well with extensions I put in. Is it also conflicting on non-windows platforms?
     
  3. imtrobin

    imtrobin

    Joined:
    Nov 30, 2009
    Posts:
    1,545
  4. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Alright, I'll be working on this furiously all weekend. Guess I won't get much football watching in. :)
     
  5. sevensails

    sevensails

    Joined:
    Aug 22, 2013
    Posts:
    483
    Some questions :

    - I'm serializing some Vector2 and Vector3, I only need to save x,y,z but it serializes Normalized, Magnitude. Is possible to avoid all this extra serialization?
    - Is it possible to use JsonConvert.SerializeObject(SaveData) on a Thread?

    My save game routine is taking half a second (the Serialization Process and Saving it To Disk), any suggestion of how to improve it?
     
  6. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Yes, look in the /Extras folder and there is a Vector2Converter and Vector3Converter (and a couple of others). Use those converters when serializing your Vectors and it will only serialize the x, y and z properties respectively.

    It is certainly possible to do your serialization on a thread but you'll have to be careful. If you're serializing in the background, the main thread could potentially change values in some of your classes. You'll need to make sure you take concurrency between threads into account. Also, if you have any properties (get methods) that access some Unity APIs that interact with the main thread, you won't be able to serialize those in a separate thread.

    As for the performance, drop me an email with what the JSON looks like so I can see how big it is. We can take a look at how you're saving it and see if we can figure out where the bottleneck is. It'll take me a little time to get to it as I'm working on Universal app compatibility with Unity 5.2 right now. It goes beyond fixing up the new errors as a final release mode compile in Visual Studio uses .NET Native and is causing some issues.
     
  7. sevensails

    sevensails

    Joined:
    Aug 22, 2013
    Posts:
    483
    You saved my day!

    Using the CustomConverters now my Save File is 60% smaller and it was 50% faster to save!

    Thanks for the support!
     
    Dustin-Horne likes this.
  8. Nerosam

    Nerosam

    Joined:
    Jul 23, 2013
    Posts:
    40
    I just noticed in the latest 5.2.1 p1 there is a release note:-
    (none) - Universal Windows Apps: Removed APIs from WinRTLegacy which were reintroduced by Microsoft in .NET Core 5.0. This should fix errors where same class is implemented both in WinRTLegacy.dll and System.*.dll .

    Going to try this out and see if it fixes the issue.
     
  9. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    That might explain why I'm not seeing all of the same issues everyone else is, because I'm running 5.2.1. It still won't fix the Native build issues but it might clear up some of the ambiguous reference stuff.
     
  10. Nerosam

    Nerosam

    Joined:
    Jul 23, 2013
    Posts:
    40
    Notice I'm talking about the patch release that came out today. I haven't yet tried the general release 5.2.1. I had issues with Patch 5.2.0p.
     
  11. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Ok, I hadn't seen that patch yet. I'll install it this evening and see what has changed.
     
  12. sevensails

    sevensails

    Joined:
    Aug 22, 2013
    Posts:
    483
    Is it working with 5.2.1? I need to generate a Windows Store Build and I didn't updated to Unity 5.2 because I'm afraid of it not working.
     
  13. sevensails

    sevensails

    Joined:
    Aug 22, 2013
    Posts:
    483
    It still does not works!

    Code (csharp):
    1.  
    2. Assets\JsonDotNet\Source\WinRT\Converters\RT_JsonValueConverter.cs(52,52): error CS0433: The type 'IJsonValue' exists in both 'Windows.Foundation.UniversalApiContract, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null, ContentType=WindowsRuntime' and 'Windows, Version=255.255.255.255, Culture=neutral, PublicKeyToken=null, ContentType=WindowsRuntime'
    3.  
     
  14. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Yes I'm aware and working on it. I've actually switched gears and I'm working on a workaround for now. I'm tweaking the newest official JSON .NET 7.x release and getting it precompiled for Windows 10, then you'll need to pull it into your project and set that assembly to only work for Windows 10 Universal (I'll put together some instructions) and I'll make sure that none of my existing source will compile under Windows 10. That will provide a stopgap while I work on my rewrite. It seems to have been a pretty significant change.
     
    Nerosam likes this.
  15. Nerosam

    Nerosam

    Joined:
    Jul 23, 2013
    Posts:
    40
    Its all good man. Keep it up! You're a legend. :)
     
  16. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Thanks, sorry it's taking so long. It was a much bigger deviation than I expected for a dot release.
     
  17. SimteractiveDev

    SimteractiveDev

    Joined:
    Jul 9, 2014
    Posts:
    97
    Hey, I notice ItemConverterType is not included, is there an alternative to it? Trying to serialize a list of enums. Thanks.
     
  18. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    That is correct. I will have it in the future once Unity gets their Mono upgrade in place and I can fully update JSON .NET. For now, you can accomplish it using a custom JsonConverter. I wrote one which another of my customers modified slightly and sent me. I haven't tested it yet but I didn't see anything in the code that looked out of place.

    Usage:

    Code (csharp):
    1.  
    2. public class SomeClass
    3. {
    4.     [JsonConverter(typeof(EnumStringCollectionConverter<MyEnumType>))]
    5.     public List<MyEnumType> ListOfEnums {get; set;}
    6. }
    7.  
    Or if you're just serializing and deserializing the list you can pass the EnumStringCollectionConverter<MyEnumType> in the converters collection.

    Converter is attached.
     

    Attached Files:

    SimteractiveDev likes this.
  19. SimteractiveDev

    SimteractiveDev

    Joined:
    Jul 9, 2014
    Posts:
    97
    That's perfect, thanks very much for all the great work!
     
  20. sevensails

    sevensails

    Joined:
    Aug 22, 2013
    Posts:
    483
    Do you have any estimate of when it will be ready the Unity 5.2 compatible version?
     
  21. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Not an exact one. I'm working on it. It is 5.2 compatible, it's just Windows 10 Universal apps that are having issues.
     
  22. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Nerosam likes this.
  23. Barkers-Crest

    Barkers-Crest

    Joined:
    Jun 19, 2013
    Posts:
    154
    Are you having any luck with the Windows 10 universal issue?
     
  24. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Not yet, I'm still running into issues. If it's blocking you though, send me a PM and I'll get you a workaround. Essentially, I'll give you a version that's precompiled for the editor only and I'll get you a working Windows 10 precompiled assembly. You'll put both assemblies on your project and then use the platform settings in Unity to specify that the second assembly is for Windows 10 Universal and the first is for Editor. That should allow you to build out to Windows 10.
     
  25. Barkers-Crest

    Barkers-Crest

    Joined:
    Jun 19, 2013
    Posts:
    154
    It isn't blocking me to the point where it would be worth putting your time towards a work around. I can wait while you focus your energy on solving this issue. Thanks for replying.
     
    Dustin-Horne likes this.
  26. LiuLink

    LiuLink

    Joined:
    Jul 22, 2015
    Posts:
    3
    Hi Dustin Horne,
    Does your Json.Net plugin working for Webgl also?
    I just started with a Unity project that using WWW class with json to communicate with server. I've been trying with multiple Json plugins for Unity but they seem not to work well with webgl. On Webgl build, whenever I try to deserialize/serialize between json string and my object it somehow stops although it runs on editor.
    List of library that I've tested:
    SimpleJson
    Newtonsoft.Json
    MiniJson
    Could you let me know why? If your plugins can work I think I will purchase it.

    Thanks.
     
  27. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Yes it works with WebGL. I'm not sure why the other packages don't. Supposedly, JsonFX works with WebGL so if you're just needing something simple I would certainly try that first since it's free and it's very similar to how JSON .NET works so you wouldn't really be wasting any time if you needed to convert over. I've not tried JsonFX on WebGL but it's supposed to work with IL2CPP. You're more than welcome to purchase my asset of course, I just want to make sure it's something you really need.
     
  28. LiuLink

    LiuLink

    Joined:
    Jul 22, 2015
    Posts:
    3
    Thanks for your reply, I will make another try with JsonFX.
     
  29. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    You're welcome. JSON .NET does work in WebGL, it just sounded like you're maybe not doing anything super complex and if a free solution works it would be a better fit for you.
     
  30. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    @Barkers Crest
    The fix is pretty large scale because of API changes Microsoft made, but I do have a working workaround now. Please send me a PM with your email address and I can send it over to you for testing. It's also precompiled so should save you some time.
     
  31. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    @Nerosam - Forgot to mention you in the above post as well. PM me with your email address so I can get you that update. I sent you a PM about it as well.
     
    Nerosam likes this.
  32. forzabo

    forzabo

    Joined:
    Aug 28, 2015
    Posts:
    25
    I am pulling my hair out trying to get Deserialization to work with this package in Unity.

    I have a server which is outputting this string as UTF-8:

    [{"id":"1","name":"Mikes_Demo"},{"id":"2","name":"Bos_Demo"},{"id":"3","name":"Some Other Demo"}]

    I have run this output direct from the server to several online JSON Validators, it does validate!

    Hard coded the string deserializes as well.

    But when I try to deserialize it in my code, I get the same error every time:

    JsonReaderException: After parsing a value an unexpected character was encountered: ?. Line 4, position 1.

    Code (CSharp):
    1. string json = w.text;
    2. games = JsonConvert.DeserializeObject <List<gameData>> (json);
    I have scoured the web for answers -- there are things about stripping the first three bytes from the response, which I've tried to no avail, and other dead ends. I am stumped! Can anyone help?
     
  33. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    @forzabo -

    This will happen if your server produces a UTF-8 string that has the BOM marker on it. You should be able to do a String.Trim() since Unity uses .NET 3.5, which *should* remove the BOM, though you may want to handle it differently to account for future upgrade of the .NET profile. Here is another solution from StackOverflow:

    Code (csharp):
    1.  
    2. stringBOMMarkUtf8=Encoding.UTF8.GetString(Encoding.UTF8.GetPreamble());
    3.  
    4. if(json.StartsWith(BOMMarkUtf8))
    5.      json = json.Remove(0,BOMMarkUtf8.Length);
    6.  
    7. json = json.Replace("\0","");
    8.  
    There are lots of ways you can do it, with stream readers, or on the server before you return it, but you get the idea. That BOM has to come off of there in order for it to be processed properly.
     
  34. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    It's also possible that your server is inserting a null byte character somewhere in the string. Could you send me a PM with a URL that generates the JSON? I can inspect all bytes and see what's going on with it.
     
  35. forzabo

    forzabo

    Joined:
    Aug 28, 2015
    Posts:
    25
    Thanks so much for your quick reply Dustin. I don't think the BOM marker is there, though? :(

    When I implement this code:

    Code (CSharp):
    1.             print(w.text);
    2.             string json = w.text;
    3.  
    4.             string BOMMarkUtf8=Encoding.UTF8.GetString(Encoding.UTF8.GetPreamble());
    5.            
    6.             if(json.StartsWith(BOMMarkUtf8))
    7.             {
    8.                 print ("bom");
    9.                 json = json.Remove(0,BOMMarkUtf8.Length);  
    10.             }
    11.             json = json.Replace("\0","");
    12.  
    13.             print(json);  //what's it now?
    14.  
    15.             string test = "[{\"id\":\"1\",\"name\":\"Mikes_Demo\"},{\"id\":\"2\",\"name\":\"Bos_Demo\"},{\"id\":\"3\",\"name\":\"Some Other Demo\"}]";
    16.  
    17.             games = JsonConvert.DeserializeObject <List<gameData>> (json);
    the output is:

    {"id":"1","name":"Mikes_Demo"},{"id":"2","name":"Bos_Demo"},{"id":"3","name":"Some Other Demo"}] // missing leading bracket!

    and the deserialization fails (obviously, as it is no longer correctly formatted)
     
  36. forzabo

    forzabo

    Joined:
    Aug 28, 2015
    Posts:
    25
    hit this:

    http://acmenerdgames.com/api/responder.php
     
  37. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
  38. forzabo

    forzabo

    Joined:
    Aug 28, 2015
    Posts:
    25
    Issue solved, my server responder was tacking on garbage at the end of the response -- an extra ?> to be precise (that "?" should have been a clue to me in the error message)

    Many thanks to Dustin for beyond the call of duty hand holding in PMs while I tracked down this issue.
     
    Dustin-Horne likes this.
  39. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Glad we have it working!
     
  40. riftgarret

    riftgarret

    Joined:
    Mar 13, 2014
    Posts:
    5
    Running into an issue building inside Unity client itself.

    /JsonDotNet/Source/Linq/LinqExtensions.cs(148,20)
    /JsonDotNet/Source/Linq/LinqExtensions.cs(291,21):
    /JsonDotNet/Source/Linq/LinqExtensions.cs(93,53): error CS0121: The call is ambiguous between the following methods or properties:

    `Newtonsoft.Json.Linq.Extensions.AsJEnumerable<Newtonsoft.Json.Linq.JProperty>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JProperty>)'
    and
    `Newtonsoft.Json.Linq.LinqExtensions.AsJEnumerable<Newtonsoft.Json.Linq.JProperty>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JProperty>)'

    any help is appreciated.
     
  41. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    You must have another asset that has the official Newtonsoft.Json.dll in it. Generally other assets will do this for editor scripts. Look through your other assets and find that dll and delete it.
     
  42. riftgarret

    riftgarret

    Joined:
    Mar 13, 2014
    Posts:
    5
    cheers, worked, thanks!
     
  43. riftgarret

    riftgarret

    Joined:
    Mar 13, 2014
    Posts:
    5
    Actually, i have another library that has a dependency on the Newtonsoft.Json.dll and while removing the DLL allows this JSON parser to work, it breaks the library dependency. Any ideas to get around this?
     
  44. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    You'll have to recompile your other library. Is it a Google library? Sometimes people compile these libraries without setting Specific Version = false in the properties, so it then breaks because it's expecting a specific version of the assembly. If it's your own library, that makes it easier because you can load it into visual studio and recompile with that property set to false. If it's another publisher's library, you may have to have them recompile it for you.
     
  45. riftgarret

    riftgarret

    Joined:
    Mar 13, 2014
    Posts:
    5
    The library is AVRO: http://www.us.apache.org/dist/avro/avro-1.7.7/
    the csharp tar should give you what you need. Currently Im using the dll it provides. Though it does have all the source files. I was hoping there was a simple solution rather than trying to dump all the source into the project.
     
  46. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    You're going to have more trouble than that. :/ I dug through and they only have the C / C++ source. No source for the C# stuff. Also, I have a feeling that DLL is ancient. The JSON .NET version they are tied to is 3.5.0 from 2008 which is really old.
     
  47. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    If it's binary serialization you're after, you could look at dropping Avro and going straight with JSON .NET. Instead of JSON you can use the BSON conversion options so you're serializing and deserializing with binary instead of JSON.
     
  48. Meltdown

    Meltdown

    Joined:
    Oct 13, 2010
    Posts:
    5,425
    Hi Dustin,

    If I'm to use your solution, what would you recommend as the best way to store JSON locally on a device? PlayerPrefs?
     
  49. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Hey @Meltdown. It depends on your platform. PlayerPrefs would certainly be a fine place to store it. On some platforms (like Standalone or even mobile) you could also use Application.PersistentDataPath and store that way. In standalone you'd be able to use subfolders, but in mobile you'd be limited to root. In WebGL you're kind of stuck (mostly) using web services and storing that way, though you could interface with javascript and use localStorage to store the data in WebGL.
     
    Last edited: Oct 23, 2015
  50. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    Just a heads up, I have been working on building out a unitypackage with precompiled assemblies instead of source. My next asset update will bundle JSON .NET for Unity as precompiled DLLs but I've found a Unity bug I need to report. When compiling the IL2CPP compatible DLLs with Visual Studio 2015, Roslyn produces IL that the IL2CPP compiler can't handle so it needs built with VS2013.

    This shouldn't be anything you have to worry about unless you're precompiling yourself which is possible by putting the code into a class library project and setting the proper symbols.

    I do have one other issue I have to figure out. The .unitypackage file I'm building has an assembly for Windows Core (Windows 10 Universal). When importing that assembly in Unity on a machine that doesn't have Windows 10 / SDK installed, it results in an exception and the DLL would have to be manually removed. I'm not sure that this will be acceptable so I may see if I can include it as a nested .unitypackage that can be imported as needed. Does anyone have any thoughts or preference on this?