A Voxel Game prototype in Unity ECS

So I decided to take another stab at making a minecraft-style voxel game in ECS:



Results were pretty nice. Full source is available from my repo. It's pretty barebones at the moment and the code is not great but I plan to keep poking it at. I'm happy to answer any questions about implementation if anyone is interested, otherwise hopefully someone finds it useful.

 

This is so dope!

 

Thank you for sharing!

 

This is awesome, man. Thanks for putting this together! I'm sure this will be an incredible learning resource for many (myself included).

 

This is great! Thank you so much for sharing it.

If I've understood your code correctly, you're harnessing ECS to calculate and store the vertices, triangle indices, and UVs, and then you use that information to build a chunk's actual mesh and apply it to a regular GameObject. Did you choose this approach for any particular reason, or is it simply because ECS isn't ready yet to properly deal with meshes, normal recalculation, etc.?

Like I said, thanks a lot for making your code available.

 

Yup, you got it exactly right. I originally tried using hybrid renderer for rendering chunks and it was truly horribly slow. I'll give it another shot once 2020.1b3 is out and we have access to hybrid renderer v2, but based on the roadmap video they just released it sounds like game objects will still be faster for this particular purpose.

Even using game objects the rendering is pretty slow, and uploading mesh data is *really* slow. Once I switch over to 2020 I'll take a look at combining meshes and using the new MeshData api to upload mesh changes in jobs.

 

I don't want to be rude just want to ask question that would clarify for me how I should use ECS. How would you implement blocks that would actually do something? For example furnaces or any other type of machine which would need it's own system to work? Because from what I can see blocks in your game are not entities at all an such furnace wouldn't have any system because it's not an entity.

 

Yes I specifically avoided the entity-per-block approach. An entity in Unity is 2 ints, that's a *lot* of wasted memory for massive worlds of blocks where the vast majority of blocks do nothing special. I chose to represent a 16x16x16 "chunk" of blocks as a dynamic buffer of ushort, where the ushort is the block type.

I'm still thinking about how to handle blocks with behaviour - what minecraft calls "tile entities". My initial thought is to just spawn a behaviour entity for any block in the buffer that has behaviour when the chunk is created and go from there. That will be annoying to keep its existence tied to an index in the block buffer but it feels like a reasonable naive solution to start with. I'm a ways away from worrying about that though.

 

I tested it some time ago, and it crashed ECS

 

I think it's important to realise that grouping all data together isn't necessarily the best case. For stuff like furnaces, it might make sense to rather have a separate entity that can take care of the processing side, with it just making sure it pushes the necessary data to the chunk so that it can be rendered correctly.

 

If I would want to, I would use blocks as entities for active chunks.
Unless having own algorithm for collisions checks, (I did for example one without colliders, for blocks selecting, using flattened array for blocks in a chunk; and different one with octree).

But ultimately, data shouldn't be stored in entities. Entities would be used just for mapping. But when saving, is just index position in chunk, ID of the block and alternatively other properties of blocks, if applicable.And

And in fact, you can render voxel world, without having any block as entity, or GO at all.

 

Thank you so god damn much this helped me make my marching cubes ecs implementation. though i cant get the hybrid renderer to work for some reason so i just used game objects

 

Yes hybrid renderer can be awkward in my experience. I am not sure, if it become any better since last halve/year ago? Anyone to comment on that is more than welcome. But heh, avoid game objects for rendering, if you want most of DOTS benefits
Unless you don't have that many of them, or they are static.
Either way, there is graphics mesh rendering instantiated API, which may be for your interest.

 

It's been quite a while, but from my prototype testing using plain old GameObjects with meshes was absolutely the best option for the way I was doing it. Each voxelchunk has a completely unique mesh, and even a simple scene would have thousands of these voxelchunks. You can't use gpu instancing in this case because of the unique meshes and the hybrid renderer just wasn't optimized for this specific scenario - at least the last time I tested it.

 

Yeah I get you where you come from.
I had very recently similar trouble / challenge on my mind, wanting to generate hex tile base planets.
I think I share my short experiance, if you don't mind sneaking into your thread, as the challenges I believe are very similar.

GameObjects in my case were out of scope, as I try to reduce use of them to bear minimum.

This is +2k tiles hex planet example, with various of tiles "types" and heights.



But I stress tested even with 160k.

Spoiler

And 160k.

Spoiler

Now if you consider individual tiles (12 pentagons and rest hexagons), if you look carefully, each of them are slightly different.



So I knew, I can not use classical mesh and instantiation for that purpose.
But I wanted ECS to control them.

And must for me, to be able control each individual tile color and height.
So I end up using DrawMeshInstanced and ShaderGraph. However,
DrawMeshInstancedIndirect would be much better, but I am not ready for it yet.
As the result, I use only 2 meshes, 3D pentagon and 3D hexagon, with cut off bottom.
Then use shaders to adjust vertices and I can dynamically control look of the planet.
Shader Geometry would be even better, but me writing manually shaders, is beyond my expertise.

 

I guess the most troublesome part about calculating the voxel mesh then pass it to DrawInstanceIndirect is to sort the chunk...?

 

I am not sure, if I understand you correctly, but you can grab chunks to render, based on camera position and rotation.

 

Im trying to figure out the most affective thing to store the voxel points in. huge dynamic buffers freeze my pc when the voxel chunk is created.

 

How big is your 'huge'.
How do you create them?

 

In an experiment i store the voxel data as 16x16x16 chunks with max 3 bytes per voxel (12kb per chunk). I use bitmask and byte-enum for some data.

 

It would be nice if Unity makes some kind of contest for programmers, teams of programmers(with prizes preferably in cash ) say for [Marching cubes, voxels, Planet renderer, huge worlds support, mesh destruction etc] dots - assets which at the end might become standard Unity packages

 

I'm now running 64x64x64 dynamic buffers of floats for voxel data. They are indeed rather big; I have plans for making the voxel size and buffer size smaller, I just haven't gotten around to optimizing that part yet. I'm not having any issues. I use [InternalBufferCapacity(0)] to set the within chunk capacity of the buffer to zero. There is no way I am fitting even a tiny compressed voxel data array into a chunk. When I create the component, I copy an entire 1D array all at once. I don't iterate and fill each voxel one at a time.

I'm curious what happens when your PC freezes. I think it takes something like 200 ms to generate the voxel data from the noise algorithms. If you generate a bunch of data at once, that could surely slow down your computer. I save the data, so when I'm testing, I'm mostly just loading from a file. It isn't optimized either, but it is much, much faster. Actual voxel mesh generation from loading data to finishing only takes about 40 ms. Even still, doing 100s of them in one frame would make everything very slow, so its nice to spread out the voxel chunk generation.

 

Never built a voxel engine, or played with DOTS much, but here is my two cents:

Would the heavy processing of voxel chunks be better for your frame rate as a separate world? The idea is to create a 'request' entity passed into the voxel world, which would in turn send back a 'response' entity with the proper data when finished. This way while you're doing 200ms work, your game world is still responsive.

From what I've read, voxel chunk mesh information per 'block' is based on the current block type geometry and the block type geometry of the surrounding 26 neighbors. My initial thought would be to buffer the relevant edge blocks from neighboring chunks, run a 3 x 3 x 3 kernel test against each block, and use a cached geometry hash map with keys as the result of the kernel test. Theoretically a 64^3 target chunk blocks + 64^2 x 6 neighboring chunks' edge data shouldn't be too bad for a burst'd parallel job. Parallel process a chunk by dividing the chunk cube such that each 'sub-cube' is the responsibility of a single thread. A sub-cube thread uses readonly chunk data and writes geometry values to their own containers (verts, tris, normals, UVs, TextureArray indices, etc). The sub-cubes' geometry containers are merged at the end once all sub-cube threads complete.

 

I just do some for loops to make a 3d grid and add points. Im made a function that can flatten a 3d array to a 1d one but makes no spaces between desired indexes. Im trying to aim for 255 x 16 x 16 but just for testing purposes i have been using 85 x 16 x 16.

 

i have been thing about storing the terrain points in groups of 4s and have the points in those groups share the same voxel type(for example dirt) to lower the amount of memory the buffer stores. (25% in my case). Tomorrow I'm going to work on chunk loading around the player like Minecraft. Ill post if it goes well

 

every thing besides the assigning all the tri, vert, and uv data to the mesh, are multi threaded.

if i get what you said right you mean to march every cube in a separate job instead of marching all the cubes in each chunk in a job. currently im going through each chunk entity and marching all the cubes in a for loop. but yur saying i should perform the marching of each cube in separate jobs?

 

do you mean coroutines(had to look that up)? Im going to check those out

 

Minecraft has 16 vertical sections of 16x16x16 voxels in one chunk. It also faster for generate meshes, you don't need to recreate the entire chunk, only one 16x16x16 section.

 

Im not recreating chunks. rn im working on the generation of them. after the chunk is generated, it runs fine

 

is there a way for me to spread the generation of chunks over more than 1 frame in a job?

 

The point is, you don't need generate whole chunk at once. You can only generate visible chunks. And you don't need to touch chunks, filled with same blocks. For example air blocks above the ground. This way, you save tons of calculations.
You don't need generate chunks underground, if not going be seen yet. At least, until they come into view.

There is multiple ways. Either split data into multiple Native Arrays. Or using IJob.. Deffered and there are few more options.

 

Every time voxel changes, you need regenerate the mesh.

 

rn I'm just generating chunks in a area around the player on spawn, this cause the time needed to enter playmode take much longer. I already have the necessary code for generating only necessary chunks from before i converted to ecs, i just haven't implemented because of the freezing when a bunch of chunks are generated in one frame.


After i get the chunks to generate faster i will check out only updating certain terrain points when the chunk is edited but right now chunks are not being generated. the voxel map is given noise values, then i run marching cubes over them.


I may have found what is slowing it down though. when going over sarkan's code i saw his gridutils. I made something similar but i didn't know the proper formula so i put it in functions that had for loops which are probably costly considering I'm going over thousands of terrain points. Im currently changing all that stuff so hopefully it runs faster.

also what would Ijob...Deffered do ?

I heard that if you spread things like procedurally generated things over more than one frame, that it helps get rid of performance spikes

 

When editing you only need to march the cube/ Whatever people do for blocks, in a certain radius. if i edit a point, i only need to re update any cube in the grid that has that point as a corner

 

I don't know exactly what you mean, how exactly does you represent your blocks? I have 16x16x16 voxel chunks that are created in a single mesh. I hope not that you have a mesh for every block. that would be 65,536 meshes for a chunk of 16 * 16 * 256, so it would be no wonder if you get problems with several chunks.

 

@PBN_Pure
In fact IJobParallelForDeffered may not neccessery be what you want. It is more for unknown size of collections, like hasmap, where you add elements to it.

See Few post down, for better explenations
https://forum.unity.com/threads/when-should-you-use-ijobchunk.982212/#post-6381012

Can you profile your jobs and see, what actually makes your bottle neck? I will suspect the way, how you write to the mesh. Do you have any GC?

 

Just got entering playmode to go from like a minute or 2 to 6 seconds. I finally found actual formulas for position to index, before i used a for loop with the index to pos and checked if it matched but that was pretty slow because of how frequent it was called

 

I do use a single mesh for a chunk. i fixed the problem by using a better method for getting a index from a position. ill start working on chunk generation around the player and see if i have any problems

 

thanks for all the suggestions. The biggest thing slowing it down was my method of getting a index from a position. before i was using a for loop but now i finally found a formula

 

In my case i have a priority component for chunks, calculate it for chunks that need an update by camera frustum and player distance. Then Sort and process an max amount of chunks per frame.

 

That actually sounds like a good idea. if you dont mind me asking, how did you find the chunks in view. I have started optimizing the hell out of my terrain. got it to go from 1500 ms to 700 ms in 3 lines of code. I never knew that for loops in for loops were less performant than one for loop of the same total length. I have got it down to 100ms. I either want to load only the ones in view or find a coroutine like approach so there are less spikes.

 

I used the API of the HybridRendererV2 for the Frustum Culling.
If I remember correctly, what my last numbers were, i can generate about 30-40 16x16x16 Chunks per frame with 30-40fps in editor (fill with data and mesh generation).
II do have some problems with the HybridRendererV2 with lots of chunks.
(in the beginning i had cubes as placeholders)https://forum.unity.com/threads/hybrid-renderer-v2-0-4-0.847546/page-8#post-6223029

 

How did you use Hybrid Renderer? I don't know how stuff was in v1 but in v2 you have to use managed components that are slow - no burst, no jobs, with CG, or RenderMesh component that's shared and doesn't fit for chunks with unique meshes. I stress-tested neither of these but both ways seem not good for a voxel engine.
So I'm thinking about switching to your model "render with GOs, manipulate with ECS". Do you think it is still actual?

 

Yes, it was not good. I wasn't doing anything unusual and it was about as slow and awkward as you'd expect.

I haven't messed with it in a while, but yes I assume using mesh renderers for rendering is still the way to go. We didn't have hybrid components back then - or I didn't know about them at least - so it should be even easier to do now.