Job System + Minecraft

Hey folks,

So I am one of the many developers working on a minecraft engine and trying to decide if jobs will be right for it. I already have a complex multi threaded engine within Unity and changing it to ECS and Jobs is no small feat, its a ton of work.

Before I start on it though I wanted to gauge if people who are already familiar with it think there will be much benefit for me. Currently I setup a few threads and keep them always running/sleeping to save the overhead of making new ones, which to me is a worry of what the job system does is make new threads although I would hope it also keeps threads open and just fills the with jobs.

Anyway this system is pretty efficient for me, the biggest bottle necks are ones job can't solve yet anyway, loading mesh data into the physics system and loading data into the mesh itself.

Based on this, does it sound worth it for me to invest a lot of time converting to the job system? I can't think of any quick tests I can do to really compare the results and give me more info to base this on.

Cheers!
Jason

Current threaded workflow:

Create a chunk on the main thread, add it to a list of chunks needing block data
Once its picked up by the generation thread, its filled with data and put into the lighting list
Same story, generate light and add to calculate thread
Calculate the mesh data, add to an update thread
Build the mesh/colliders split over a few frames to minimise FPS drop

The reason I split like this is priority/the player looking in a new direction. So the initially populated lists will become low priority if the player does a 180, so as each section is finished it will skip chunks in too low a priority if there are ones needing generated sooner than them. So in addition to the threads above, I have a thread constantly scanning in front of the player and updating chunk priorities based on distance and field of view.

As you can see, the jobbed version would have to be a Complete rewrite of this one

 

I think that whether you benefit from jobs or not depends largely on your data access patterns across these threads. One of the primary advantages from the jobified ECS paradigm is that data is stored and accessed in linear, cache-friendly ways.

 

That is one of the gains I think it will give me is that it will force me to that linear cache friendly structure which I certainly don't have currently! Guessing it will also reduce my context switching.

This is difficult though since overall even if this works amazingly better than what I have, all it means is my chunks load quicker and they load decently fast already :/

 

Is your multi-threading working well today?
Easy enough to maintain, troubleshoot, and background new pieces of code?
If so, the value of converting would be less.

It made sense for me, as I wasn't multithreading and needed a way to background my greedy voxel mesher.

The learning curve is not trivial.
It took me a couple of weeks hacking various hybrid ECS patterns before I settled on a way forward.
The resulting components, component tags and systems make for a clean architecture.

Specifically for the job system, you'll need to convert to Unity's Native* data types.
This can require some refactoring if you have >14k data, strings, or nested data like a dictionary of dictionaries.

Job chaining might be of interest to you- tho I've found it simpler to just add/remove component tags.

 

Yeah it seems to be doing the job just fine, its slow in the editor and you see gaps as it loading chunks but silky smooth in a build and loads up chunks just fine. FPS is my real issue and the main bottle necks are typically loading the data into meshes.

I gave up with greedy meshing in the end, I had a rough first pass done but it exponentially increased my calculation time which didn't feel worth the trade off since rendering was solid. Modern GPU's eat verts and not targetting mobile. If I switched to jobs and a burst compiler etc I imagine the calculation of greedy would become much less noticeable so thats an advantage I didn't consider But again, a trade off I am not currently worried about.

I am considering using the native data types without jobs since Slice is something I have wanted for a while, will save me significant calculation times alone.

I am leaning more towards not implementing jobs I think. I have yet to think of a strong benefit. For your situation andywatts it makes perfect sense though! If you had no threading before its certainly worth getting in there and I imagine you saw huge gains in calculation speeds as I did when I first stuck threading in. Its the comparable difference I worry about

 

you can try taking your most CPU-intensive functions and try to jobify (with Burst) it, then profile.
if you get the promised massive speedup, it can be worth to port it.

Burst should be a huge part of the 100x speedup they showed at Unite.

plus, eventually they will add jobified API for assigning mesh vertices, and I don't think you can use them with custom threading solutions

 

Oh yeah when they add the API for assigning mesh vertices then it will certainly be worth it! A bit risky for my project to wait on that though :/ I am guessing a year is a sensible round figure before we see that and optimistically it could be done in half a year which is still a long time to wait

 

I really don't get your logic here.
- if you don't convert your code, you won't have the, hopefully faster, setVertices() neither now or later and if you put more and more systems on your own solution it will become more and more hard to jobify it
- if you start to convert your systems now, you won't have the faster setVertices() now, but later, when they do it, you can easily integrate it into the jobified system you have

So I see no reason to not to move onto the job system, except if you're very close to release and your systems are enough for the work.

You really should not wait for anything, but you have to estimate your own development cycle and also estimate what Unity has done already and probably will do in the same time-frame.

 

Uuumm you just directly contradicted yourself and answered your own question (of sorts)

 

There is also the glaringly obvious point that Unity themselves say Jobs/ECS should probably not be used for a production project unless essential for your game

 

ECS is a preview package, insert disclaimer, etc etc...

Jobs can be used without ECS and are released with 2018.1 without the "Experimental" namespace, so I assume they are stable enough to be used in prod. It is a low level system though.

 

Ahh thought it was both! That is something at least I remember some part of what I was looking into for what I needed was still labelled as experimental

 

Errr... no. I did not. The 'not waiting' and the 'evaluate if you need some feature before it potentially done' are not contradictory terms. Since we don't know your plans, your development roadmap, we are just guessing. This is the reason I wrote whatever I wrote. It is up to you, but merely the non-existent setVertices() isn't enough to drop a potential move to job system, since with your own you will lack this feature forever, meanwhile if you make your convert to the job system, you will have this feature at some point in the future. Which is better than never.

But, obviously, it's your job to make this decision, I only can shed some light on the basic logic part.

 

I'd recommend taking another look at greedy meshing, and also at compressing UV's (i.e, pack 2 floats into 1 etc). Reducing the size will scale performance for setVertices etc nearly linearly.

 

@LurkingNinjaDev Errr actually you did twice The first contradiction was saying you didn't understand my logic and then repeating my logic at the bottom, the second was saying you see no reason and then giving the reason I quoted as well.

 

Never looked into UV packing although it sounds lossy, got any references to tuts etc handy that you can recommend?

Ah interesting I didn't consider greedy meshing as saving time loading vertices, that is a good point. The algorithm I found/tested was a sort of post process operation rather than in-lined so in my head greedy meshing was something I would revisit and generate as I build the first set of verts to save having a second expensive pass. Typically this is probably not possible but in a minecraft world I figured there must be some decent tricks to accomplish that.

Anyway, at the time the test of it was overall really slow (close to doubled my overall generation time per chunk) so didn't feel worth it and on top of it I had to rethink my already a little complex shaders to support it

 

Yeah it'll be lossy for sure. But my chunks are 16^3 - and with the way I'm doing the UV's, they'll always be between 0.0 and 16.0. My textures are 256x256 in a tex2DArray, so the pixel values of the UV maps end up being between 0 and 4096 (whole numbers). That's 12 bits of data. I did the naive thing (x * 4096 + y) on my uv channels, and in the shader I do a trunc / multiplication to get it back out, and it just worked so I've left it there. I do the same thing with my 4 tex2DArray indices - I doubt I'll ever load a few thousand textures so it won't run into lossy problems. So I'm storing 6 values into 1 List<Vector3> uv.

For greedy meshing, there's some tutorials around that can help you on it. The primary thing for me was to store which sides to make from a 2d slice of a chunk in a buffer (only 16 * 16 values), and start iterating that. When you find a block side to make, just check the neighbouring sides in the buffer. If you can enlarge to a square (2x2, 3x3, 4x4) - do that. Otherwise try enlarging to a rectangle. Repeat until you can't enlarge it. Then make the quad and mark them made in the buffer. Repeat until you checked every flag in the buffer.

For UV's with greedy meshing - it works very easily if you're going the tex2DArray route, which I'd highly recommend. You can just scale the UV's with the quad size and it works. The tex2dArray's also fix any texture bleeding in lower mipmaps. Doesn't work for dx9 / older openGL though.

You may end up having T-junction issues with the greedy meshing - they're hard to solve correctly. I'm currently winging it by enlarging the quads by a tiny fraction so you can't peek pixels between 2 quads. It results in some z-fighting for a millimeter if you look very closely, but I haven't heard complaints yet.

This is a good resource to read through to get a general idea (including comments):
https://0fps.net/2012/06/30/meshing-in-a-minecraft-game/

Edit: some grammar, it's still a mess though.

 

Your info looks great thanks, I am in my 9-5 so only glancing over it for now.

I am indeed using texture arrays, it makes such a huge difference. At first I was tying to use .DDS files but they didn't give enough control (within Unity) to do things like only use the first 4 mips in my atlas instead of 7 etc etc. Currently we are running with textures 128 by 128 but have been considering 256 for a while. Hard to say how many textures we will need though so sticking with 128 for now until the project comes together a bit more. I hire 2 artists currently and the only coder so I suspect they will flood me with textures over the duration of the project

 

Ah yeah I see what you are going for with the compression now I am reading it in the right mindset, I don't think it will work for me though since my chunks are currently 32,128,32. I found it much easier to make a huge chunk in terms of data and then split them into submeshes. So 128 is my max world height and it will contain slices of chunks in the size 32,16,32. I found 16,16,16 to need too many draw calls and by using sub meshes it let me do some custom static batching a bit easier.

Unity static batching works great but damn does it take a while to calculate and chews your memory. If your not familiar with it internally it makes all possible combinations of your meshes (so far as I understand it) in a static form so its was much more efficient for me to make one big static mesh draw in the distance and I swap it out once the player starts breaking it up