Job system for general purpose / managed use-cases

As I understand it, the job system is currently meant to go hand-in-hand with the burst compiler to produce highly optimized code that's only achievable given certain constraints (no allocations, no reference types, etc.)

In a large codebase I work on, we use our own custom job system to schedule tasks such as serializing and deserializing JSON. It would be nice to take advantage of Unity's job system if only for its scheduling (our threaded job system has no knowledge of Unity's job system worker threads and their job priorities, etc.) but the currently exposed Unity job system APIs do not provide a way to achieve what we do with our own job system, despite the use case not necessarily needing the benefits gained by the restrictions that prevent the use case.

One could make an argument for exposing overloads of Unity's JSON serialization APIs to work with native containers, but we'd still need to create a string at some point so that'd end up allocating more memory than with our current solution.

We also use gzip compression on the serialized JSON, and since the platforms we target have hard memory limits, we perform the gzip compression/decompression in chunks as the JSON is being serialized using streaming APIs - which is not currently possible using Unity's JSON serialization APIs (we use JSON.Net right now).

Will such a scenario ever be supported using Unity's job system, or will we have to keep using our custom job system for operations that directly involve strings and other reference types?

 

Well first of all you should really switch to protobuf for this, and then design your data so you use integers instead of strings for data sent over the wire. Your choice of serialization and data design is your larger issue.

As to concurrency, depends on what your networking looks like really. Try to use the thread context that it uses for your serialization if you can. And keep it simple. Your best bet is smart use of tools like BlockingCollection and ConcurrentQueue, with maybe a long lived thread using say Task.Run. Most hand crafted stuff here that I see is typically way more complex then it needs to be, and also not as efficient as just correct use of existing concurrent collections.

 

To clarify, I'm not talking about realtime networking (which I wouldn't use protobuf for at all, I'd use hand-created structs). Some of the data is used with a pre-existing database we've had in place for several years, some of it is used elsewhere (converting to a format suitable for modification in a text editor, for example). it's quite out of the scope of what I'm talking about.

We already have solutions in place (and have for a long time), it would be nice to switch to Unity's job manager in order for the job scheduling to be smarter. Nothing except Unity's job system knows about Unity's other scheduled jobs and how it orders them. If Unity is capable (or can become capable) of prioritizing jobs such that it doesn't do non-critical work during its graphics jobs, for example, then I would like to be able to take advantage of that to maximize Unity's performance.

 

I don't think you realize the overhead of json and how inefficient it is to serialize/deserialze. In terms of memory use, GC, and speed. And FYI structs/protobuf are completely orthogonal.

Basically if you are doing enough serialization where scheduling matters, your serialization is the problem. Scheduling better will be white noise compared to fixing the underlying issue.

 

I'm not sure what Unity has in mind for IO-bound jobs or allocation-heavy jobs like deserialization. But I actually went ahead and rewrote an IO-bound job to use the Job system. It went OK. For the most party you just have to copy data in/out of NativeArrays. Compared to the speed of de/serialization, it may be no big deal.

If you need to perform allocation on a separate thread you can use NativeList (take care to read the entire thread).

For strings you can use something like this:

Code (CSharp):

1.  
2.     public struct HeapString : IDisposable
3.     {
4.         private readonly IntPtr _ptr;
5.  
6.         public HeapString(string s) { _ptr = Marshal.StringToHGlobalAuto(s); }
7.  
8.         public string DeMarshalled {
9.             get {
10.                 return (_ptr == IntPtr.Zero) ? null : Marshal.PtrToStringAuto(_ptr);
11.             }
12.         }
13.  
14.         public void Dispose() {
15.             if (_ptr != IntPtr.Zero) {
16.                 Marshal.FreeHGlobal(_ptr);
17.             }
18.         }
19.  
20.         public static HeapString Null { get { return default(HeapString); } }
21.     }

If you need the job to hold a HeapString (or any value type really) as output, you make a NativeArray of size 1. (Explanation here)

It's all kind of goofy, but you can make it work. If you have only a few routines which need to be run in a separate thread, but which aren't well-suited to the Jobs system, this sort of jerryrigging is the right choice.

 

I disagree heavily that deserialization is something that needs to run fast. It might be the case that the deserialization has no impact on his product, provided it happens off the main thread.

 

I know exactly how long my serialization takes and what resources it uses... I can provide other examples of how we use threads that don't work out of the box with unity's current job system, but at the end of the day they all boil down to "operating on reference types or generating managed objects as a central part of their design, under the constraints of hard memory limits."

The question is about the potential of using the job system, likely without the burst compiler - maybe with a slightly different set of APIs - to work in the above situations with the understanding that the best performance when using algorithms to transform data is to use the current native data structures and APIs.

Protobuf is just a general purpose serializer. I meant that if I were working on realtime networking for a specific game, I would rather hand-write the way it sends data rather than using a more general "one size fits most" solution, because I would know exactly what sort of optimizations I can apply to the transmitted data if necessary.