C# job system?

In the announcement post the C# job system was named as one of the main features for the Unity 2017 cycle.
From what has been shown so far my game is a perfect fit for it, so I'm curious if this is still on schedule? (Assuming the next release after this is going to be 2018.1)

 

Have you seen the 2017 Keynote ?

"None of this is shipping any time soon"

 

Is the job system unity's version of threading?

 

Of multi-threading yes, but they are improving the compilator too.

 

Whole thing interests the cashew nuts out of me. I would enjoy more information about the whole thing in text whenever it's feasible to comment on it (can't hear vids).

 

Yeah, that's what I meant (multi-threading).

 

Available since yesterday, this video might interest you:

 

Yep! Some good stuff coming pretty soon after all
Summary for hippocoder and others who can't watch video: C# job system that was shown end of last year planned for 2017.3 later this year/early next year; job compiler shown at this Unite not this year and no ETA yet. There's also a new math library coming.

 

Finally. I've noticed my own code and framework choices are trending in the direction of ECS and dependency injection. While getting the C# job system this year is good, is there a timeline for the ECS framework after it? Around the same time or early next year?

 

It wasn't clear, but I think he said they already have a working ECS project that they will make available soon (he talks about it somewhere in the questions segment).

Basically I think the ECS project comes before the final jobs system

 

Since Unit 2017.1 is out and you can develop with .NET 4.6, why don't just use TPL for any multi-threading jobs?

 

I assumed the Job system would give us access to some or all of the Unity API during jobs. The biggest limitation with threads, even in prior versions of Unity, is the challenge surrounding access to data hidden behind Unity APIs (transforms being a big one, but Time and others are an issue as well).

 

There are likely performance and access considerations for the C++-side of the engine that TPL's "general solution" might not work well with. And on top of that having it structured in a specific way likely helps simplify the optimizing compiler down the road. You're probably free to use TPL for non-simulation code in Unity or for data-only processing (especially if you're dealing with large datasets from a server to drive a UI) once the mono upgrade is complete.

 

I'm not seeing this appear on any of the beta timelines unless I'm missing something. I certainly can't see it listed for 2017.3 - however that is still stated to be in planning.

I did make a whole heap of notes and attempted to capture most of the code scripts from the Unite talk - great BTW. Perhaps I need to dump this somewhere so others can read it....

 

haha, I did the exact same thing in an attempt to start building a game framework that will be ready for the ECS and jobs system. Basically, I'd say it boils down to:

• The components you put on gameobjects must be data-only and must not rely on inheritance (since they'll have to become structs)
• You have systems that operate on parallel tightly-packed arrays of these components (data-oriented programming style)

 

Considering from past experience, think of UI or Input, how much time Unity Technologies spends on integrating new systems in their product after their first announcement, I'd be surprised to see the C# Job- or ECS land in a 2017.x release. I'm all for surprises though!

 

well... you never know. In the talk, Joachim says:

• The job system API should land in 2017.3 or 2018.1. This is only the "API" part, not the "Compiler" part. I guess this means this code won't actually be fully multithreaded, but at least it'll allow us to start working with it.
• The ECS is supposedly already ready as a prototype that will be released to the community as an experimental package
• The job system compiler tech will be for later in 2018

So it really seems like we'll start getting access to all this good stuff in 2017.3 or 2018.1, and then we'll just have to wait for the magical performance boost when the new compiler tech is ready. This really doesn't bother me too much because if I start a new game project in late 2017 / early 2018, it most likely won't be ready for release before the compiler tech arrives

 

It certainly would be very nice to get it early, so fingers are crossed

 

By the way, @CarlosRincon

Are you aware of any plans for releasing the ECS prototype? I would definitely like to start working with it as soon as possible, even if it changes completely at some point in the future. Just like the Playables API which ended up changing massively a bunch of times before it got officially released, it's still very useful to have something that you can start experimenting with and giving feedback on.

The reason why I specifically want the ECS to be released soon (as opposed to the actual Job System) is that it really determines the entire structure of your gameplay code, so it's almost unthinkable to start a new project without it if you plan on using it later. At least with a prototype, converting your project to the final release would probably be more realistic. It would allow us to start working right now

It could be released as an "in-progress" thing on Unity's github, just like the HDRenderPipeline, the old Input prototype, the PostProcess stack v2, etc.....

 

Where is there more information on the new ECS system?

 

To the best of my knowledge, the C# Job System Unite presentation is the only place where this is talked about

 

Unless I lost the plot, isn't Unity already Entity Component System...? I mean isn't this how you work right now? Unity even mentioned here: https://en.wikipedia.org/wiki/Entity–component–system

So what is "new" ECS?

Sounds like we probably would not want to adopt this model for anything except large scale numbers like boids or something...?

Docs please Unity! no need to release early, but we would all appreciate a nice wip google doc complete with typos (we unenrdsand)

 

Regardless of the "high-performance data-oriented" aspect of it, I really feel like handling all logic through "systems" and all data through "components" is the most well-structured game code architecture paradigm I can think of right now. For me, it's the ideal solution to solving the problem of creating links between different components of an entity (and links between different entities) without having to write tons of boilerplate code, or turning everything into a big spaghetti, or having to do "GetComponent()" all the time.

I'll admit it's a bit hard to explain why exactly I think it's better. It's just so.... elegant. The thing that convinced me of the greatness of this sort of approach to an ECS is the Overwatch Gameplay Architecture And Netcode talk from GDC 2017. Unfortunately it's for GDCVault members only, but it's very worth it

 

OK well in my case I want to get things done, and that sounds pretty inefficient to author, for designers and artists. I know Unity dislikes using gameobjects as data blobs but I do, it's much easier for a tiny team to use them as data blobs.

In the case of the new ECS, I see it as more of a tool for the job thing so I appreciate where you are coming from, but don't think I have the time to engineer an entire game around it because I would need to do a fair bit of support engineering and many more tools.

I can see why you want more info, it does sound a bit daunting.

 

Here is some info from Unite Austin 2017. At the end of the tech talk (linked below), they provide some dates...

• C# Job System in 2018.1
• Early preview of new Component system in 2018.1
• New compiler tech later in 2018

The C# Job System / Compiler Tech talk starts at 1hr 26m

 

Just gimmie already.

Question though: at what point does using jobs etc become more efficient than the classic workflows? I'm guessing it's fast even at scales of 1.


@Joachim_Ante nice one

 

https://create.unity3d.com/jobsystem

Not sure whether this will just send an email if it's officially available, or if you get your hands earlier on it by signing up there

 

I am so happy about that RTS game presentation. I cannot wait to get my hands on all this.

Also, the physics are getting jobified too : https://twitter.com/melvmay/status/915302715433111552
If you look at the replies of the tweet, you'll see Anthony Yakovlev saying 3D physics will also get more saturated multithreading once PhysX 3.4 is integrated. Exciting times!

...on the other hand, I kinda don't feel like starting any new project until 2018.1 is available

 

Thanks, didn't even know that link was a thing.
It's actually the feature I'm looking forward the most since I saw the previous Unite Europe Keynote.
I've been basically annoying everyone about it on the IRC channel.

Was a bit disappointed when I checked out the new 2017.3 Beta and there was nothing on it in it, since they first said the first phase of it was planned for 2017.3.

 

I found this presentation from Unite Korea. There are some more details about the c# job system. Not sure how accurate it really is though.

http://event.eventservice.co.kr/unite17/unity_mobile/pdf/0517_Track1_6.pdf

I also found these code snippets:
https://gist.github.com/valyard/e96060ee8e5a9d52d43901c7c6ad4288

 

Nice find! It's the Rotator example from Unite Europe, but having all the source code is pretty cool

 

That code feels retro yet familiar...

 

I find it amusing they say "Old school MonoBehaviour design".

 

What worried me from the presentation is this:

Async/Tasks | C# Job System | Native multithreading​

Unity API no | no | no

This still means the job system wont give access to the API, just transform, mesh and textures. That's strange in my opinion, since most of those components could already be accessible by a simple abstraction applied on top of them.

What is the use for job system if I don't have access to let say, camera API or animation API?

 

Playables API (animation) is supposed to be jobifiable though

 

The new jobs stuff is a huge change to workflow and design. Joachim basically says "... abandon OOP if you want to go fast." and he provided a nice look into how it is currently expected to work in Austin. It is real, and it is very different. It's going to take some time for adoption.

The company that did the nice fancy game demo (forgot their name) showed how it was applied practically but there were huge and wildly different approaches to their systems and solutions that went very, very low level. If you're the average user at this time, you'll probably not be using it for several years until there is a wider reception, better documentation, examples, tutorials, other assets using it, etc.

If you're a more advanced user there is practical application now for effectively spreading workload across the cpu in a much more efficient manner. The way it is done feels more like old cpp approaches than anything else and really makes you design differently than you would with OOP. There are a lot of caveats and things that make your life easy which you will be surrendering when using this new stuff.

It's far from a 'go fast' button and requires that users are ready to dig deep to improve performance.

 

If the Unity API is not accessible in the jobs (Only Transforms, Meshes and Textures - which is pretty cool itself!), I feel like they should have selected their words more carefully in the keynote. They said they are instantiating and destroying hundres/thousands of objects (The arrows) and that it was running so smoothly thanks to the new system. I guess this isn't 'true' instantiation and object destruction as we know it?

 

Yeah that sounded weird to me. They must've used some kind of pooling for this

 

For the demo shown in Austin there was no pooling, but iirc they used the gpu instancing and drawmesh stuff to render everything. They also had some bizzare tricks like putting animations in textures, using double quaternions instead of transforms, and other weird things.

It still is largely going to boil down to people being good programmers to make their stuff go fast. Everyone really needs to understand that. Joachim gave some examples of small changes to structure that resulted in some pretty big gains, but rarely does anyone ever bother to dig deep enough in to get those gains.

 

So, yeah, I don't think they were instantiating arrows. Just populating a list of vectors that represents arrow positions, another that represents velocities, using the velocities to update positions and using the positions to draw GPU instances arrows. Honestly, fairly straightforward stuff even without jobs. We will just have to wait for some more information. Lots of confusion and mystery surrounding all of this at this early stage!

 

The only part of this that I've really struggled with is how to use Graphics.DrawProcedural for drawing objects that have proper shading and shadowcasting (an equivalent of the standard shader) in deferred rendering. The best I could do so far was unlit. I hope unity releases some kind of example of this