Tile based RTS terrain - best way to render?

Hello. A few weeks ago I started to work on 3D rts game, like "Age of empires". This is my first huge game project on Unity, and I'm a little bit confused about choosing best way to render my terrain.

To cut a long story short, I have a tile grid-map, about 256x256 size. Every tile represent 2m*2m surface, but the values may change over time. After researching the topic, I think that I have 3 options to render my terrain properly:

1. Per tile rendering. So - this is very simple approach. Every tile is unity GameObject with simple tessellated geometry and relative texture. Just use the Camera culling and render all stuff. Fairy simple, but the core drawback is: drawcall number may be very high.
I know that Unity has static batching option - so this may helps, but I'm not quite sure how to config/use this properly.
2. Render the whole map by 1 drawcall. Map = one big mesh. But in this approach we have problem with textures - this one big mesh should use one material.
Even if I pass with each vertex the texture_index attribute for texture array, the "Standard unity shader" can not complain it. So obviously I need my custom shader. This is not big problem for me, cause I have experience with OpenGL + glsl. But, when I start to use unity, I really hopes that the engine will get the rendering work for me. I can write shader for general map rendering, but I don't want to write such things as Light + Shadows for this. The Standard shader works well, and have light+shadows from the box. But it looks that it is impossible to render tile-terrain by Standard shader.
3. The last way is build-in unity terrain engine. But it looks that it is well suited for RPGs or other kind of terrains. Basically this terrain for player walk over it. And as I see, this terrain have no real option for "procedural generating" it from scratch. (We can generate elevations, but the textures is a big deal).

So, what you thinkg about it? What is the best way to render 3D tile terrain?
As I think, the way number 1 is works well for fast beginning, but may lead to huge fps problems in future.
The way number 2, requries me to learn the whole unity shader programming, that takes a lot of my time. While I'm studying - the project will stand still =(
The final result should look like Warcraft 3 / Empire Earth games. Ideas?

 

Yeah, don't do that. You should read this regarding drawcalls:
https://docs.unity3d.com/Manual/DrawCallBatching.html

The standard shader isn't a good fit for anything with complex or special requirements. I suggest you take a look at Megasplat or Microsplat by @jbooth.

It's probably possible, but the terrain engine is one of the weakest features of Unity, if you don't absolutely need it, I'd stay away from it and go with meshes.

While it can be fun to mess with shaders, I agree that this is a huge time sink and not beneficial for you actually getting your game playable. Like I said above mega-/micro splat might work for you. I don't use them myself because I made my own custom shader with Shaderforge, since I have a few extra special requirements for my terrain. It does take a lot of time to experiment with that for sure.

Can you make art assets of that quality yourself? If not, that's your biggest problem imho.


Regarding culling, LODs, and polycount... I have no hard data on this, but using very big high resolution meshes for the terrain I had very little problems with performance, whereas the Unity terrain component spent a considerable amount of time on calculations that didn't seem to benefit the frame rate at all. Just make sure that you stay within the per mesh limit for vertices (something 64k-ish I think) with your individual terrain tiles and make sure you keep them roughly square, or else you might end up wasting drawcalls because the auto-split for overly large terrain meshes tends to cut them into long stripes.


How are you gonna solve the "outside of the map" area? That's the thing I haven't figured out myself yet.

 

You can also batch a number of tiles such as a block of 16x16 or 8x8 tiles which can be converted to one mesh. You would have to do this by script after the sub-tile is created. This way you have lesser drawcalls and the advantages of culling and LOD. If your 2mx2m tiles have LODs as well then you can combine each LOD level for all those tiles in the sub-tile.

I don't know how your terrain would look like but that comes in mind as an alternative option to what you have described.

There are more options but those would be specific to what you want to achieve, which poly count we speak of and so on. For instance you would like to reduce drawcalls for tiles which are further away. On the other hand if you only look more top down on your map than flat, then this wouldn't be an issue.

 

Thanks a lot for your reply and suggestions! I have checked this assets, and it's looks great. I never used paid assets before, but it looks like that the time has come. )

I digging a little into unity shaders for now, and temporary decide to write simple shader by my own. As I see, it's relatively easy to code something based on unity's "surface shader". It seems that it gives all lightning & shadow calculations from the box, while I still can performe multi-texture splatting.

My current idea is:
0) Generate tile map, and all needed vertex data first. For every tile texture appoint its "power". For example if power of GRASS_TILE > power of DIRT_TILE this means that when GRASS_TILE will be adjacent to DIRT_TILE, the GRASS_TILE will render its part on the DIRT_TILE location.
1) For every adjacent tiles pre-calculate their relations in terms of power. Assign splat maps to the tiles - so the every tile knows the 4 indexes of splat maps for left, right, top and bottom neighbours. Now it comes shader works...
2) pack all available terrain textures into texture array, and pass it to shader.
3) pack 1-dimentional splat textures into texture array, and pass it to shader
4) for every "tile" i will pass the following data to shader:
POSITION, NORMAL, UV (respective to tile point, so left-bottom corner is 0,0 and the right-upper corner is 1,1);
MAIN_TEXTURE_INDEX - index in terrain_textures_array,
LEFT_NEIGHBOUR_TEXTURE_INDEX + LEFT_SPLAT_MAP_TEXTURE_INDEX
the same for RIGHT, TOP and BOTTOM neighbours.
So, i need to pass 9 indexes to my shader. Looks like it is possible by abusing the uv2, uv3, uv4 mesh params.

And in shader I will perform 9 texture fetches: 1 fetch for self tile texture, 4 fetches for splat maps and 4 fetches for neighbours textures. The final result will be blend toghether in fragment shader.

This approach gives nice flexibilty. And it is even possible to switch texture type of a tile in runtime by doing small VBO streaming (sorry for GL terms, don't know the right HLSL word for this yet).
The only thing I'm afraid of is 9 texture fetches per pixel. Is it too many?

If I fail with this approach I will use megasplat.

 

Yeah, I also think about it. But the core problem for grouing tiles in this way, is still texture. When I group 16x16 tiles into one mesh, I also need to provide some texture for this mesh (obviously pre-generated). This way gives a great chance to make really complex blending, but is hard to realize. I need to use Render textures to generate big-textures for this 16x16 groups, and it is a little bit confusing to me now (setup orto-camera, draw tiles, blend it in some way (how?)). As you can see, I'm not very good with Unity's shaders for now, so this method is not good for me for now...

 

I also think about two simple optimizations for my 9 texture_fetches approach.
1. Is it possible in shader, to NOT doing texture fetch, if I read the 0 value from splat map? For example, I read the 0 value for left neighbour in splat map. This is obviously means, that the pixel that I will read from the left_neighbour_texture will be fully transparent, and the reading is useless. How can I cancel texture fetch for this case?
2. Most tiles have the neighbours of their own type. The number of "connector" tiles in the whole map are not so big. The most common case is GRASS that adjacent to GRASS from all sides. For such case, I could have the "special splat index" meaning, that there is no splat map for respective neighbour at all. Is it possible to NOT doing even SPLAT map texture fetch at all (and don't do the following texture fetch as well), if the vertex splat index is for example -1 (not valid index for texture_array).
Thanks a lot!

 

I'm fairly certain I have more and it's no issue for high end desktop hardware. If you are targeting mobile it might be different, I don't know current mobile specs. But to me the whole thing sounds a bit like premature optimization.

Branching in shaders has its own cost, you might shoot yourself in the foot with that. Don't worry about the performance for now. If Warcraft 3 is your visual target then I don't think shaders will be an issue. Fillrate/overdraw/drawcalls/pathfinding are more likely to bottleneck you. And making the animated art assets will be a ton of work.

 

The issue you will run into with this approach is that your indexes will be barycentrically interpolated across the face in the vertex to fragment structure. So if you have a triangle with indexes (1, 7, 12) these will end up values between 1 and 12 depending on which pixel you are on.

 

Oh god! I've just realize that Unity shader's have not the OpenGLes qualifier word "flat". Looks like that I will need additional job to correctly round this index values. Thanks for your comment!

 

May be you are right. But I prefer to do things right start from the beginning. Or at least enough flexible for future changes.

I know. The only profiling can give an answer what is better. I've just read about conditiona branching in shaders, and looks like that if you are NOT in "uniform predictable" branching (in OpenGL terms), the branching has really huge cost.