Search Unity

JSON .NET for Unity

Discussion in 'Assets and Asset Store' started by Dustin-Horne, Sep 13, 2013.

  1. layola

    layola

    Joined:
    Aug 6, 2013
    Posts:
    94
    fixed

    public string GetTextWithoutBOM(byte[] bytes)
    {
    MemoryStream memoryStream = new MemoryStream(bytes);
    StreamReader streamReader = new StreamReader(memoryStream, true);

    string result = streamReader.ReadToEnd();



    return result;
    }
     

    Attached Files:

    Last edited: Nov 19, 2014
  2. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    yes or you can open your bson file in notepad++ and change the encoding to UTF8 without BOM and resave it.
     
  3. AndyGFX

    AndyGFX

    Joined:
    Jan 13, 2012
    Posts:
    98
    Changed but :( same result (tested on your TestScene)

    Code (CSharp):
    1.  
    2.  
    3. ...
    4.   [JsonConverter(typeof(JsonDotNet.Extras.CustomConverters.Vector3Converter))]
    5.     public Vector3 test = new Vector3(2, 4, 6);
    6. ...
    7.  
    8. var serialized = JsonConvert.SerializeObject(test);
    9.             LogSerialized(serialized);
    10.  
    11. ...
    12.  
    13. #### Serialized Object: {"x":2.0,"y":4.0,"z":6.0,"normalized":{"x":0.267261237,"y":0.5345225,"z":0.8017837,"normalized":{"x":0.267261267,"y":0.534522533,"z":0.801783741,"magnitude":1.0,"sqrMagnitude":1.00000012},"magnitude":0.99999994,"sqrMagnitude":0.99999994},"magnitude":7.483315,"sqrMagnitude":56.0}
     
  4. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Oh, that won't exactly work... that attribute will only take effect if you're serializing the class that "test" belongs to. Instead, try this:

    Code (csharp):
    1.  
    2. var test = new Vector3(2, 4, 6);
    3.  
    4. var converter = new JsonDotNet.Extras.CustomConverters.Vector3Converter();
    5.  
    6. var serialized = JsonConvert.SerializeObject(test, new JsonConverter[] { converter });
    7.  
     
  5. AndyGFX

    AndyGFX

    Joined:
    Jan 13, 2012
    Posts:
    98
    Sorry, a few seconds ago I found it. Now is V3 serialized only with x,y,z.

    Thx.
     
  6. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Awesome. I'll make the same updates for the other converters and try to get all of my bug fixes put together in a patch release.
     
  7. Xylph

    Xylph

    Joined:
    Dec 11, 2013
    Posts:
    30
    Hi, is it possible to give me an estimate for the fixes?
     
  8. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Hi, I wanted to make sure i am not doing anything wrong with the library.

    I am getting a JSON response from a server and deserializing it to a concrete object using this code:

    Code (CSharp):
    1. response = JsonConvert.DeserializeObject<FBPaymentResponse>(result.Text);
    The issue is, that whatever the input is, this does not throw any exception in case the input JSON does not match the object's interface, for example, this is the actual class:

    Code (CSharp):
    1. public class FBPaymentResponse
    2. {
    3.     public string payment_id;
    4.     public string amount;
    5.     public string currency;
    6.     public string request_id;
    7.     public FBPaymentStatus status;
    8.     public string signed_request;
    9. }
    And this is the JSON response:

    {"error_code":1383010,"error_message":"User canceled the order."}

    My question is - should the JSON deserialization fail in this case? I would assume it should verify that the given type T matches the input string.

    Another option is that the deserializer creates an "empty" object of type T and only attempts to set its fields. In this case it will not fail, but will result in an object instance whose fields will have their default values.

    I'd love to know exactly how it works and whether I am doing something wrong.

    Thanks
    Lior
     
  9. pajamajama

    pajamajama

    Joined:
    Oct 22, 2012
    Posts:
    66
    Hey Dustin,
    I was wondering if you could lend some advice on deserializing this example of json. I'm not sure what data types I should set up inside Unity. The most important bits are the inventory nodes under "hat", "shirt", etc.




    Would these be automatically deserialized as dictionaries called "inventory"? Do they each need to be in their own classes called "hat", "shirt", etc?

    The forum wouldn't accept a .json, so I've attached the example file as a .txt. Thank you very much for your help!
     

    Attached Files:

  10. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Xylph -

    Sorry for the delay as I was out with family for the holiday. I'm working on this again today.
     
  11. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    You're not doing something wrong. The (de)serializer first creates an instance of the object, then enumerates the JSON and tries to set matching fields. The objects don't have to match 1:1. In fact, you can have data in your JSON that doesn't exist in your object and properties in your object that don't exist in your JSON. It does not throw an exception by design.

    The fact that your JSON doesn't match your object at all does indicate that you're not doing something correctly because you don't know what the JSON is that's coming back from the server. If the endpoint that you're using returns different JSON in difference scenarios (for instance a completely different object if there's an error or the payment fails) then that endpoint doesn't have a good implementation. What you'd have to do in this case is deserialize it into a JObject instead (using JObject.Parse) and then check for the existence of the properties you're looking for. Then when you find them you can deserialize from that JObject into the object type you're expecting.
     
  12. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Depends on how you want to handle it. If they have different properties then they need to be their own classes (or structs). In this case it looks like you could have a class to encapsulate all of them so:

    Code (csharp):
    1.  
    2. public class InventoryItem
    3. {
    4.    [JsonProperty(name="inventory")]
    5.     public Dictionary<string, int> Inventory { get; set; }
    6.  
    7.     [JsonProperty(name="isOutOfStock")]
    8.     public bool IsOutOfStock { get; set; }
    9. }
    10.  
    Note the JsonProperty attributes above... if you want your object properties to be Upper Case then you'll use that attribute to tell the serializer to match them to the lower case versions in the Json. If you make your property names lower cased you can omit the JsonProperty attribute. I just added it to show you that you can upper case them.

    Now, for your master object... It would look like this:
    Code (csharp):
    1.  
    2. public class InventoryResponse
    3. {
    4.     [JsonProperty(name="apiVersion")]
    5.      public int ApiVersion { get; set; }
    6.  
    7.      [JsonProperty(name="message")]
    8.      public string Message { get; set; }
    9.  
    10.      [JsonProperty(name="kiosk")]
    11.      public string Kiosk { get; set; }
    12.  
    13.      [JsonProperty(name="hat")]
    14.      public InventoryItem Hat { get; set; }
    15.  
    16.      [JsonProperty(name="shoe")]
    17.      public InventoryItem Shoe { get; set; }
    18.  
    19.      [JsonProperty(name="socks")]
    20.      public InventoryItem Socks { get; set; }
    21.  
    22.      [JsonProperty(name="tee")]
    23.      public InventoryItem Tee { get; set; }
    24. }
    25.  
    You can name the classes whatever you want... but in this case you would deserialize the Json string as InventoryResponse. I think this should work... but it may not because the properties of "inventory" are not actually a collection. They are an object... but you can't have property names with periods in them so you may have to change your JSON response. But give this a try and see if it works for you. If not I can tell you how to change your JSON to make it work with a dictionary.
     
  13. Xylph

    Xylph

    Joined:
    Dec 11, 2013
    Posts:
    30
    Thanks for the update.
     
  14. pajamajama

    pajamajama

    Joined:
    Oct 22, 2012
    Posts:
    66
    Dustin,
    this worked like a charm! Thank you for the awesome asset and support!
     
  15. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    You're very welcome. Let me know if there is anything else you need and reviews are always appreciated. :)
     
  16. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Xylph -

    Do you have a repro project that you could send me for your issue? And what version of Unity are you using? I've been testing this with Unity 4.6 and I'm not able to reproduce the issue. I'm not sure what your original classes and code looks like but I tried your simple sample that you gave me and it is working in the web player without issue. I used the following script and attached it to the Main Camera. I have a Text object (from the new UI) dropped into the scene and assigned to the text field of the script:

    Code (csharp):
    1.  
    2. using Newtonsoft.Json.Linq;
    3. using UnityEngine;
    4. using UnityEngine.UI;
    5.  
    6. public class JObjectTest : MonoBehaviour {
    7.  
    8.     // Use this for initialization
    9.     public Text _text;
    10.  
    11.     void Start ()
    12.     {
    13.         var jo = new JObject();
    14.         jo["Test"] = "foo";
    15.         _text.text = jo["Test"].ToString();
    16.     }
    17.    
    18.     // Update is called once per frame
    19.     void Update () {
    20.    
    21.     }
    22. }
    23.  
     
  17. EmeralLotus

    EmeralLotus

    Joined:
    Aug 10, 2012
    Posts:
    1,462
  18. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
  19. EmeralLotus

    EmeralLotus

    Joined:
    Aug 10, 2012
    Posts:
    1,462
    Apologies -- It wasn't meant to be a spam.
    Edit: It was a question: Has anyone used this really Fast Json package?
    I got excited when I saw the performance comparison and thought it's a good idea to share the link. The reason that I sent the link to this thread and full-inspector is that I own both assets. In my opinion, you and fullinspector are really the expert in Json and serializing and therefore would know better than I do.
     
  20. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I knew you owned mine :). Yeah the performance does look really cool. I think you lose some flexibility but might be worth the tradeoff in some cases. I've personally never heard of fastJson.
     
  21. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Another quick question, we've upgraded from JsonFX and the behaviour we're depending on have changed:

    In some of our code we have deserialized JSON into an object[] and then casting it to other types.
    One particular use is that the JSON contains an array of Dictionary<string, object>.

    Each value can also be a dictionary of its own. Not sure that's a good idea but i'm trying to minimize the changes made for the upgrade.

    With JsonFX we would get those as a Dictionary type alright, but with JSON.NET these are deserialized into a generic JObject type.

    Is there any (easy) way to keep the old behaviour ?
     
  22. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Could you provide a quick code example of how you were deserializing and using it and what the json looks like?
     
  23. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    For example:
    upload_2014-12-3_14-38-51.png

    files used to be serlized into an object[] field.

    Then we would cast each element to a Dictionary<string, object> respectively.
    And again, each value (which is stored as an object reference) would also be cast to a Dictionary<string, object>

    Now i get those as a JObject reference, so I was able to use the ToObject<T> method to cast those back to the original type we used, although i do prefer as little intervention as possible.
     
  24. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Hmm... that doesn't look like a Dictionary... that json is a collection (list or array) of objects. The way you would accomplish it is by using JsonSerializerSettings and setting TypeNameHandling = TypeNameHandling.All but you would need to do that when both serializing and deserializing which would store the full type as a property called $type in the json itself. It may be a little trickier with nested Dictionaries (i.e. you have Dictionary<string, Dictionary<string, object>>). You would probably want to create a custom JsonConverter to use for it.

    That being said, your best bet would be to create a class that matches your data:

    Code (csharp):
    1.  
    2. public class FileList
    3. {
    4.    [JsonProperty(name="files")]
    5.    public List<FileData> Files { get; set; }
    6. }
    7.  
    8. public class FileData
    9. {
    10.     [JsonProperty(name="name")]
    11.     public string Name { get; set; }
    12.  
    13.     [JsonProperty(name="file")]
    14.     public FileItem File { get; set; }
    15. }
    16.  
    17. public class FileItem
    18. {
    19.     [JsonProperty(name="updated_timestamp")]
    20.     public long UpdatedTimestamp { get; set; }
    21.  
    22.     [JsonProperty(name="url")]
    23.     public string Url { get; set; }
    24. }
    25.  
    Then deserialize it as type FileList:

    Code (csharp):
    1.  
    2. var fileList = JsonConvert.DeserializeObject<FileList>(jsonString);
    3.  
     
  25. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Thanks for the detailed answer.

    What I've done for the meantime is this:

    Code (CSharp):
    1.  
    2. // The JSON from the example, serialized as array of dictionaries.
    3. Dictionary<string, object>[] files = GetObject();
    4.  
    5. for (int i = 0; i < files.Length; ++i)
    6. {
    7.     var tempObject = filesObjectArray[i] as Dictionary<string, object>;
    8.  
    9.     tempName = tempObject["name"].ToString();
    10.  
    11.     // Treat the value of the "file" key as JObject and cast to a dictionary.
    12.     var file = (tempObject["file"] as JObject).ToObject<Dictionary<string,object>>();
    13.  
    14.     levelDict[tempName].URL = file["url"].ToString();
    15.     levelDict[tempName].FileTimestamp = file["updated_timestamp"].ToString();
    16. }
    This appears to work fine, not sure what caveats are there to make this fail.

    Also, it looks like you are suggesting to always work with strongly typed objects instead of accessing things from a dictionary, which i guess would be the ultimate solution.
     
  26. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Well, in this case I would work with a strongly typed object to avoid any more boxing / unboxing than necessary. Any time you're dealing with treating a type as "object" or in your case "object[]" it would be good to rethink the approach as any time you have a value type you're going to be boxing and unboxing it.

    I would suggest taking it a step further if you don't need any reference type functionality and replace your file data classes with structs as they are then value types and do not allocate for the GC (except for some unavoidable boxing / unboxing during serialization and deserialization).

    Now, in your case I'd have to think about how that's actually working because it's not really structured like a dictionary... an example of what a dictionary would look like would be:

    Code (csharp):
    1.  
    2. [
    3.   [1, 'Foo'],
    4.   [2, 'Bar']
    5. ]
    6.  
    This would be a Dictionary<int, string>. Now you could also do:

    Code (csharp):
    1.  
    2. {
    3.     'PropA': 1,
    4.     'PropB': 2,
    5.     'PropC': 3
    6. }
    7.  
    And you could deserialize that as a Dictionary<string, int>. In your example it could work as a Dictionary<string, object[]> and "files" would be your Key, and your value would actually be a collection of objects... but not a Dictionary itself as there are no keys... it's just an object array.
     
  27. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Actually, "files" is an element contained in a larger JSON.
    Inside it, it looks like your 2nd example with braces wrapping sets of key-value.

    So we have the first key "name" (whose value is a string) and the 2nd key "file" whose value is in fact another dictionary.

    I guess by using object we can store multiple value types in the same dictionary. maybe that was the original intention, beats me (it predates my time working here).
     
  28. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    BTW is there any other way of reaching out with questions? I think this thread is monitored by other devs and probably may just make it long and harder to find things in it.
     
  29. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Absolutely. My email address is in the Readme.txt file that came with the asset. I also have Skype but I can't log in during the day as it's not allowed on the network here. If you don't have the Readme file anymore just shoot me a private message and I'll send you the info.
     
  30. AlexConnolly

    AlexConnolly

    Joined:
    Dec 23, 2012
    Posts:
    32
    I'm sorry, but why would this be favourable over using the standard JSON.net dll? I can't understand how you can justify charging $20 for something that is open source... hm
     
  31. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    You obviously haven't tried to use the official JSON .NET release cross platform in Unity. In iOS you'll get AOT errors. In Android you'll have AOT and assembly resolution errors. In Web Player you'll blow up because the signature of some base collection classes are different. You won't be able to build for Windows Phone 8 or Windows Store without maintaining separate assemblies in your project because the editor will first build against mono. Also, official release will not work on consoles. While not officially supported because I lack the ability to test on consoles, my version can be made to work on at least Xbox One and most likely PS4 as well.

    In short, the official Newtonsoft version is not compatible with Unity cross platform. James Newton King was given the opportunity to add compatibility and said he wasn't interested. I have invested a lot of time into porting it and making it compatible. And open source or not, its licensing permits me to sell my derived work.
     
  32. Xylph

    Xylph

    Joined:
    Dec 11, 2013
    Posts:
    30
    Sorry for the late reply. I tested it with 4.6b20 before. Right now I'm on 4.6 official release. I'll try to repro it and send it to you as soon as I get back to my project (tomorrow or the next day)
     
  33. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Makes sure you have the latest version of json .net from the asset store as well because the behavior your seeing should have been fixed in a previous release. I have a feeling you may have had an older version.
     
  34. Trexug

    Trexug

    Joined:
    Dec 2, 2013
    Posts:
    88
    Hi

    We got some AOT errors when trying to use JSON.NET for Unity on the Wii U, which is fair, as the store page makes no mention of support for that platform. However, we got it working after messing around with the code a bit. It turns out it is really easy fix. We did one string replacement - replacing all occurrences of
    UNITY_IOS with (UNITY_IOS || UNITY_CAFE)
    and now it works. I hope this can be of help to somebody.

    JSON.NET for Unity version 1.4.0
    Unity3D Pro version 4.3.

    We had a lot of trouble finding a way of serilizing .NET objects on the Wii U as we were unable to get the BinaryFormatter or XmlSerializer to work on this platform. So we are really happy that JSON.NET for Unity did the trick. It is a neat product and, for us, it was well worth the price.
     
  35. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Awesome, glad it works! I'll definite look at adding in Wii u support into the official package.
     
  36. LaughingGull

    LaughingGull

    Joined:
    Jul 8, 2012
    Posts:
    37
    Hi Dustin, has this been tested with unity 5 and webgl export? Thanks!

    Jason
     
  37. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Hey Jason, saw your post in the other WebGL thread as well but I'll post here for completeness. WebGL is my #1 priority at the moment and while there is no ETA I am working on it as quickly as possible. I am actually going to see if I can get engaged directly with a Unity developer to step through and see what pieces IL2CPP is missing or if there are modifications I can make to JSON .NET to make it function. I'll certainly keep everyone updated.
     
    Skjalg likes this.
  38. Xylph

    Xylph

    Joined:
    Dec 11, 2013
    Posts:
    30
    I just noticed I'm not using the latest version lol. I tested it directly in my game and everything works as expected now. Thanks!

    Also, great work with this plugin. It's so easy to serialize objects "my way". ;)
     
  39. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Awesome, glad to hear it!
     
  40. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    This is great product, by far the best JSON helper for Unity.

    But to be honest, I am still using JSONObject because your JSON.Net doesn't support micro mscorlib on iOS. Is this something your working on, or would even be possible? Have you looked into it and offer any insight why it doesn't work on mscorlib? I may just change it myself potentially.
     
  41. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    I had looked into it at one point and determined it wasn't possible but I can't recall exactly why. I believe there were some necessary interfaces and types missing from the IOS version of micro mscorlib. Do you happen to have a stack trace of the exception you get when you try to use micro mscorlib?
     
  42. Victor_Kallai

    Victor_Kallai

    Joined:
    Mar 5, 2014
    Posts:
    123
    Hello Dustin,
    I'm new to Json .NET and I have a special request, I want to know if this plugin can help me with this situation.

    Say I have a class Foo:
    class Foo{
    public string name { get; set; };
    public Vector3 position { get; set; };
    public int age { get; set; };
    }

    and an object foo = new Foo(name = "John", position = new Vector(0,0,0), age = 31};
    and a string json bar = "{"name":"George", "age":"23"}";

    I want to deserialize bar and update/overwrite foo, so that foo will be:
    Foo(name = "George", position = new Vector(0,0,0), age = 23};

    How can I achieve this? Thanks!
     
  43. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Is there a particular reason you want to deserialize over the top of an existing object instead of creating a new one? It can be done but depends on your scenario. The easiest way would be to use a JsonConverter and pass the existing object in which will work fine as long as the existing object is a class (which Foo is).
     
  44. Victor_Kallai

    Victor_Kallai

    Joined:
    Mar 5, 2014
    Posts:
    123
    In order to store persistent data in my game (currency, user progression, etc) I have a custom class, which on exit I serialize, encrypt and store in PlayerPrefs. But when I want to update the game and add a new field for persistence, the deserialize method fails, because the stored object has a different signature. Obviously, I want the users to keep their current stats, and my games to be backwards compatible.
    Can you give me a simple example on how to use JsonConverter to achieve this?
     
  45. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    No I don't. I just haven't tried given the warning you put on the info.
     
  46. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    What you're looking for is JsonConvert.PopulateObject. I *think* it is included in my port but I need to verify. You can test it to see if the method is there.

    Code (csharp):
    1.  
    2.  
    3. var foo = new Foo(name = "John", position = new Vector(0,0,0), age = 31};
    4. var bar = "{\"name\":\"George\", \"age\":23}";
    5.  
    6. JsonConvert.PopulateObject(bar, foo);
    7.  
    My port is a couple versions back so it's possible this method is not yet included. After the release of Unity 5 I'm going to attempt to update my port to JSON .NET version 6 which will include more functionality and async support but I need to wait until WebGL is solidified because there are some things (such as threading) that you can't do with WebGL so I need to make sure I can support it.
     
    Victor_Kallai likes this.
  47. Victor_Kallai

    Victor_Kallai

    Joined:
    Mar 5, 2014
    Posts:
    123
    Dustin-Horne likes this.
  48. mog-mog-mog

    mog-mog-mog

    Joined:
    Feb 12, 2014
    Posts:
    266
    Does it work on IOS with API compatibility level to ".Net 2.0" ? I cannot set it to .Net 2.0 subset because of parse dll error.
     
  49. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Yes it will work with both subset and 2.0 full. If you run into any issues drop me an email.
     
  50. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    It sounds like the Unity team is making some IL2CPP fixes to make JSON .NET work. We should see them in an upcoming beta. I'll still have to do some heavy testing with WebGL but it looks like we're going to be able to support 64-bit iOS out of the gate. I'll keep everyone updated.