Serialization [0.6.3-preview] Package Performance Issues

Is there a way currently to read in a json string and deserialize to a data struct containing a NativeString?
I was hoping the new JsonSerialization utility was the solution, but it seems that NativeString is only currently deserialized from its specific structure: (length, Byte30, etc) rather than from also a string using the implicit conversion.
EDIT: It now deserializes as shown below; however, there is a rather large performance impact utilizing this. Thread title changed to better represent the current problem.

Example:
Struct:

Code (CSharp):

1. public struct DataStruct : IComponentData {
2.     public NativeString32 Value;
3. }

Json:

Code (JavaScript):

1. {
2.     "Value" : "Test"
3. }

 

This actually worked, but would love confirmation if this is how we should be handling json translation from string to NativeString.

Code (CSharp):

1. TypeConversion.Register<SerializedStringView, NativeString32>(view => new NativeString32(view.ToString()));

 

After some experimenting, the Serialization package for json is not ready quite yet. The results are showing 100 measurements of 100 iterations. This is purely reading the same json line. These values are HOPEFULLY either a debug test is running in the background of the serializer, or I am using it wrong.



The code used to test these two is below:

Code (CSharp):

1.     public class JsonTests {
2.         private const string Basic = @"{""Value"":""JarJar""}";
3.  
4.         public struct BasicData {
5.             public string Value;
6.         }
7.  
8.         [Test, Performance]
9.         public void TestJsonUtility() {
10.             Measure.Method(() => {
11.                     var data = JsonUtility.FromJson<BasicData>(Basic);
12.                 })
13.                 .Definition("JsonParsing_JsonUtility")
14.                 .WarmupCount(1)
15.                 .MeasurementCount(100)
16.                 .IterationsPerMeasurement(100)
17.                 .Run();
18.             PerformanceTest.Active.CalculateStatisticalValues();
19.         }
20.  
21.  
22.         [Test, Performance]
23.         public void TestJsonSerializer() {
24.  
25.             Measure.Method(() => {
26.                     var data = JsonSerialization.DeserializeFromString<BasicData>(Basic);
27.                 })
28.                 .Definition("JsonParsing_Serializer")
29.                 .WarmupCount(1)
30.                 .MeasurementCount(100)
31.                 .IterationsPerMeasurement(100)
32.                 .Run();
33.             PerformanceTest.Active.CalculateStatisticalValues();
34.         }
35.     }

 

So far, it has only been the deserialization where performance has gone totally awry. Which incidentally is what I need more heavily . This actually seems backwards from what I expected, but still not to this degree.

25 iterations per measurement. 100 measurements:

 

Unrelated, sorry, but I was wondering what you are using for your benchmarking?

 

The performance testing package (using Unity.PerformanceTesting; )which is included as a dependency for ECS 0.2.0

 

@Soaryn, have you experimented with other serializers, e.g. Odin, https://github.com/neuecc/MessagePack-CSharp, etc?

However, are you sure you really need the string values inside of your burst compiled jobs, or would it be sufficient to use whatever the fastest kind of serializer available and then run the job with blittable data?

 

So I have NativeStrings being my end target data container inside my structures which is why I was looking into the new serializer. JsonUtility doesn't handle customs very well. NativeStrings are fine to use in Bursted jobs so long as I don't go back into managed string land

I'd prefer not to have a 3rd party library added for this if I can at all avoid it as that adds another dependency to this which is already 1 too many. The Json I am receiving is not of my construction unfortunately. The data to placed as an entity component is not necessarily the full json though. I am currently just wanting to serialize a provided json to a custom data struct.

 

I still have yet to get the Serialization package to work as performantly as it would suggest. As a thought, I decided to test to see if batching the json into a single large string would improve the performance, and even then it takes a minimum of ~1.97ms (1/8th of a frame) to deserialize a single struct that contains a single field. This does improve the more you add; however, this is not ideal if receiving say 1 message per second and you need to be reactive to it rather than buffering messages.

Test cases are jsonString, with batch size. 1 iteration per measurement, 100 measurements.

Code (CSharp):

1. [TestCase(@"{""Value"":""Barry""}", 1)]
2. [TestCase(@"{""Value"":""Barry""}", 10)]
3. [TestCase(@"{""Value"":""Barry""}", 100)]
4. [Test, Performance]
5. public void TestJsonBatched_Serialize(string json, int count = 10) {
6.     var jsonBatch = $"{{\"batch\":[{string.Join(",", Enumerable.Repeat(json, count))}]}}";
7.     Debug.Log(jsonBatch);
8.     Measure.Method(() => {
9.             var batch = JsonSerialization.DeserializeFromString<BasicDataBatch>(jsonBatch);
10.         })
11.         .Definition("JsonParsing_Serializer_DeserializeBatch")
12.         .WarmupCount(1)
13.         .MeasurementCount(100)
14.         .IterationsPerMeasurement(1)
15.         .Run();
16.     PerformanceTest.Active.CalculateStatisticalValues();
17. }

Regardless of the use case, this deserialization method is far from performance by default

Any Unity staff able to look into why performance of this package is awry on deserialization? Serializing I would think would be the slower aspect, but somehow it is night and day difference to the deserialization :\

 

Same test as before with the JsonUtility used instead for comparison.

 

Just wanted to say good job and keep at it! Hopefully you get some eyes on this soon from someone at Unity.

 

I can; however, it is purely using the scripts above. I don't mind proving a package later tonight however

 

Hey @martinpa no worries! I was curious if there has been any progress in tracking down what might be causing the long interval of time when deserializing?

 

Very excited for low level support. The only reason I am using Newtonsoft JSON over JSONUtility is because it lets you write your own json converters. Our usecase: We have a database asset in our project that references all the Database elements in the project (Both are ScriptableObjects). Every element has an ID that never changes so you can globally lookup a database element from anywhere given an int ID. We wrote a Json converter for type DatabaseElement that just writes out that elements ID when serialising (0 for null) and when deserialising it does a database lookup for that element. There are other ways to achieve the same thing though this is just so pleasant to work with. You just serialise a class that has references to a bunch of SerialisableObjects and when you deserialise it the references to those SerialisableObjects are automatically populated.