[TerrainEngine] Voxel Terrain : Smooth,Cubic,2D,Hexagonal Infinite Procedural Terrain

TerrainEngine is a framework to create infinite (or not) procedural volumetric terrain, planets, buildings, asteroids ... it's voxels..
​

Spoiler: Features

Features : (Language : C#, Unity 5)

• Voxel Terrain : Algorithms

1. 1. Marching Cubes : A smooth and fast terrain algorithm
   2. Terrain Octree based LOD
   3. Hexagonal : An unique terrain hexagonal
   4. Cubic : A simple cubic terrain like minecraft or cubeworld
   5. 2D Tiles : (Terraria Like)
   6. Surface Nets : Like marching cubes, but less detailed
   7. TerrainEngine contains a robust core that allow to implement any algorithm easily.
      TerrainEngine core is made for real-time terrain deformation and it can handle a lot of players/objects deforming the terrain without any FPS drop

• Multi-threading :

1. Action Framework : Dispatch actions over multiple frame for keep a constant FPS
2. Multi-threading Framework : Dispatch actions over all your CPU core to improve performance
3. Thread-safe PoolManager : Pooling system that will store and reuse all objects to improve performance. (Objects and GameObjects).This system contains also an Automatic Smart memory deallocation. Objects will be deleted automatically if needed.

​

• Terrain Systems : (Processors)
  1. Grass : Grass system that will generate plants into the terrain. This system can generate all type of plants : Ground,Climbing,Weeds,Underwater, Aquatic. Plants can be added to the ground or to the top (caves) of terrain. It's possible to use a texture and a mesh as grass, so there is no limit to the possibility. Batching is multi-threaded.
  2. GameObjects : This processor will add Gameobjects to the terrain. It's used for Trees/Rocks/Particles/etc... It contains a multi-threaded LOD system that can switch over different objects depending of the camera distance. It contains also a function to automaticaly generate an LOD plane/crossplane from any object for use it when camera is at a certain distance. LOD objects can be batched to reduce drawcalls.
  3. Details : This system is used for add objects to the terrain that can be combined. For example for small rocks,plants,roots etc. This processor combine objects to reduce the Drawcalls so it allow to have a lot of details to the terrain with a small fps impact. Batching is multi-threaded.
  4. GameObjects Scripts : A simple and easy scripting system is provided for gameobjects, it's possible to handle when terrain is modified and play effects like if the voxel is removed under a tree, the tree fall.
  5. Construction Kit : A system for create building easily. It allow users to create multiple voxel form and place them to the terrain as object. This system interract with fluids so it's possible to create bridge, barges etc.. . http://forum.unity3d.com/threads/terrainengine-free-official-construction-kit-7dtd-like.204013/
  6. Voxels Physics : Flying voxels falls. It's possible to build and destroy everything
  7. Unity Terrain Converter : Convert your Unity Terrain to voxel terrain. Keep Grass/Details/Trees

• Terrain Utilities :

1. Raycasting : It's possible to use raycast directly over voxels. It's thread-safe so it's possible to use voxel raycasting into multiple thread for increase your game performance.
2. Occlusion Culling : This system disable invisible chunks. It useful when the camera is in cave, all mountains/invisible terrain are disabled so performance is increased.
3. Dynamic Terrain Combiner : This system combine chunks using the same material to reduce the drawcalls and improve performance. It works in tandem with occlusion culling.
4. PathFinding : A simple and fast PathFinding system that will return a list of points to traversal to go through a point A to a point B. This system can manage if the path is for Ground,Fly,Aquatic,etc units.
5. Optimized Zlib : Custom Zlib framework for compression. Used for save/load terrains.
6. Optimized LibNoise : 40% more performant libnoise library than all existing solution for procedural worlds.

​

• Fluids :

1. 1. Real-time fluids : Add water or lava to your game, fluids will flow in realtime without any fps impact.
   2. Fluids Effects : With a simple script it's possible to handle when a fluid flow and add effects like particles, sounds etc.
   3. Fluids are Multi-threaded
   4. You can define a lot of params like evaporation, visquosity, etc
   5. Fluids combiner : A combiner that will batch all fluids mesh, only few drawcalls for the entire terrain.
   6. Waterfalls : Add waterfalls and rivers to your game.

• Procedural : (Infinite Terrain)

1. 1. Custom LibNoise Framework : A modified LibNoise framework that can manage noises and biomes
   2. HeightMap Processor : A complete system to generate heightmap from noises. It contains a complete GUI and inspector for create and manages noises and biomes.
   3. Biomes : It's possible to create biomes in differents scenes and blend them together. For each biome you can add objects (grass,trees,rocks,etc) and define when and where they can be added to the terrain : depending of the altitude, depth, noise, underground, etc..
      Everything is drag and drop based, you can create a world in few minutes with a lot of details like climbing plants, Trees with LOD, waterfalls,etc...
   4. Procedural Caves
   5. Procedural Volcanoes
   6. Procedural Floating Islands
   7. Procedural Rivers and Waterfalls
   8. Procedural Canyons
   9. Procedural Fantasy world

• Shaders :

1. 1. Tri-Planar (PBR): TerrainEngine use an advanced distance based tri-planar shader that allow to have a lot of details.

• Multi-player :

1. 1. Server : TerrainEngine architecture is client/server based. When you play in local, the game start an internal server and communicate with it.
      The server generate and send terrains to players.
   2. This internal server can be used for create a multi-player server easily :
      TerrainWorldServer Server = new TerrainWorldServer();
      Server.Start();
   3. This internal server also can be exported to an external project, the complete generation system doesn't use any unity classes so it can be compiled directly into another project.
   4. TerrainEngine architecture is ultra generic, it's possible to use all systems like PathFinding or Raycasting server or client side without any modification
   5. A Simple network server (TCP) is provided as example. Terrain deformation are sync
   6. The server can interract with GameObject Scripting system. It's possible to active or disable a script directly from the server. (Example : A player hit a tree, you can active the TreeFall script for all players around)

• Demos Scenes :

1. 1. 20+ scenes : The solution contains more than 20 scenes and examples game using all Type of Terrain and game type. All youtube video are made from demos scenes.

TerrainEngine License information
fastmarchingcubes@gmail.com
​

All sources are included
​

Videos :

​

TerrainEngine + ConstructionKit V8 :
http://www.youtube.com/watch?v=51JNyjBcDMo

Realtime Occlusion Culling :
http://www.youtube.com/watch?v=twpR1xz7IkI

World Exploration :
http://www.youtube.com/watch?v=3yPFuORw5Vg

The minimum PC system requirements :
-Dual core CPU
-At least 2 GB main system memory
-Graphics card with at least 512 MB of VRAM and support for DirectX 10

Recommended system requirements :
-Quad core CPU
-4 GB memory
-GeForce 600 series or Radeon 7000 series

Changelogs :

Spoiler: Changelogs

TerrainEngine V10.2 : (Changelog)​

Server/Client Generation :

• - New Generation System : Blocks are now loaded one by one by all available threads. Memory/LoadTime optimization
• -Load Time optimization (8%) : Generators : Grass/Details/Trees loading fastest. Objects are stored in temp list and applied when generator is done. Avoid long Lock() call
• -Terrain Load Order optimized : Now game ready in 1 second (3 seconds in V10 beta)
• - Fixed random loading bugs (freeze, holes introduced in V10)

GrassProcessor :

• -New Fading system : Using FadeTime, no more distance based fading
• -Materials array/Properties now stored inside the processor and reused. Memory/GC Optimization
• -Mesh Generation time optimized

DetailProcessor : (Plants,Rocks,etc)

• - New fading system : Using FadeTime, no more distance based fading (SpeedTree like)
• - New per material shadow option. It's now possible to Set CastShadow/ReceiveShadow for each material. FPS optimization
• - Materials array/properties now stored inside the processor and reused. Memory/GC Optimization
• - Mesh now use Color to simulate winds. Plants wind is more realistic

- Mesh Generation time optimized
View attachment 158648

GameObjectProcessor : (Trees,etc)

• - New GameObject data storage. Each object now use 6 bytes in memory (9 bytes before). Objects are stored at different resolution than terrain

GUI :

• - New GUI for Terrain LOD management
• - New GUI for Debug Console : (DebugGUIProcessor)
• Logs : Draw current state of Server/Client
• Pool : Draw all objects in memory for each threads
• Stats : Draw DataTable with all function called for each thread. Record Total Time,Average,Execution count,ThreadId
• Profiler : For each frame, record all actions executed
• TimeLine : In Game TimeLine, record actions Start/End Time
• New classes :
• -ProfilerGUI : Profiler.AddObject(Name); // Add an action to the current frame
• -TimeLineGUI : TimeLine.AddLine(); Line.AddObject(Name,StartTime,EndTime);
• -DataTableGUI : Class that contains orderable columns and data. DataTable.AddColum<Type>(Name);
• DataTable.AddValue(ColumnName,object value);

FluidProcessor :
-New fluid GameObject management. No more unused gameobjects in memory. LoadTime optimization.

PhysicsObject : (Voxel physics)
-Fixed incorrect splatmap applied to the Mesh

LibNoise Optimizations ( 3% )

New PBR triplanar shaders (specular,metalic with noise)
Fixed random Colliders bug. (player fall)
Fixed memory leak on TerrainChunkGenerateAndBuildGroup
Fixed Terrain Save/Load

TerrainEngine V10 Features :

• -Terrain LOD ( Drawcalls / Vertices / Memory optimizations)
• -Terrain SplatMap ( Drawcalls optimization, smooth voxels transition )
• -Improved LibNoise library ( Generation time : 36-40% faster )
• -New Noise functions / GUI
• -Grass/Details resolutions (Drawcalls optimization : 50% less drawcalls)
• -Grass/Details/GameObjects Memory optimization ( 30-50% )
• -Voxels Physics ( Voxel fall when disconnected from ground )
• -Terrain Operators ( Fastest terrain modification, less bandwidth used on network)
• -More multi-threaded tasks ( Terrain Generation 30% faster for small terrain size, 80% faster for big terrain size)
• -New Menu and Tutorials ( Create game in few seconds )
• -Unity 5.2. Improved Triplanar shaders
• -Garbage optimizations ( no more classes, 0 garbage spike/lag )
• -Unity Terrain Converter ( import Unity Terrain,Grass,Details,Trees,config)
• -Trees LOD fading/batching optimizations (40% faster)
• -In Game Profiler
• -Fastest Fluids generation ( 30% faster )
• -New procedural canyons examples
• -TerrainData memory optimizations ( 90% for imported Terrain, 20-30 % for procedural terrain )

TerrainEngine V9 :

• 
  
  • Multi-Player TCP server : World fully generated/managed on server with complete fluid and scripting management
  • TerrainEngine now play a client and run an internal server.
  • Asset Streaming : only needed assets are loaded from disk or website.
  • Biome Combiner : You can now build each biome on different scenes, and import all of them in one scenes without any modification and the % of each one.
  • Biome Switcher : It's now possible to handle when object enter to a biome and, play events/ change weather / Modify Camera effects / and more
  • UndoRedo Manager : Generic UndoRedo manager working with all systems (Voxel,ConstructionKit,Plants,Gameobjects,Grass)
  • Particles/Sounds Processor : Simple and fast system for spawn particles or play sound when terrain is modified.
  • PathFinding : Flying, Aquatic,Ground units demo scenes.
  • Loading Time optimization ( Terrain ready to play in only few seconds )
  • Generic Scripting system works now with the network without any modification. You can call a script to any trees (like tree fall), all players will see it fall.
  • Smart Rocks/Plants/Grass Height generation. You can switch between random or noise system for generate height. The noise system will create really natural pack of plants or grass, with big or small bush.
  • Flammable Plants/Grass. Fully customizable fires
  • Custom Mesh for grass. Groundcover
  • Grass/Plants/Trees now follow terrain alignment.
  • Automatic Smart memory deallocation
  • Dynamic fluid combiner ( Draw calls optimization )
  • Noise Voxel type placement depending of altitude and depth
  • And more

TerrainEngine V8

• 
  
  • Construction Kit
  • Water / PathFinding interaction with Construction Kit
  • New Fast Loading system. Few seconds only for game ready to play.
  • Procedural Waterfall + Rivers
  • Procedural Floating Island + Waterfall + Vegetation
  • Terrain Dynamic Batching. A new smart system combining chunks to reduce drawcalls and working in tandem with realtime occlusion culling.
    Drawcalls reduction : 40 to 70%, Fps multiplied by 2 to 3.
  • Procedural Caves : Fully customizable
  • Procedural Volcanoes : Lava flow
  • Multiple fluids support and new options

• TerrainEngine V7
  • Smooth Voxel Transition.
  • Climbing Plants/Grass/Details/GameObjects option(Lianas,stalactite,etc..)
  • Automatic Trees Colliders Generation (Capsule Collider for bark)
  • New Snow Biome.
  • Occlusion Optimization (4ms for all terrain occlusion calculation)
  • New Frustum culling system (1ms for all terrain)
  • New Terrain algorithm : Surface nets. You can now switch between Marching Cubes (more details) And Surfaces nets (Less vertices/tri)
  • New Memory Optimization : 1go in Ultra mode V7 against 1.8go before in V6.
  • New multi-threaded smart pooling system.
  • New Triplanar Shader, distance based, ultra detailed.
  • New Generic scripting system, screens/video coming soon. (Shatter,plants/details physics, Ice+Fire -> Water + Temperature -> Ice, and more).

TerrainEngine License information
fastmarchingcubes@gmail.com
Or pm.
Not available on Asset Store.

Any comments are welcome.​

 

Example Scenes

V9 Exploration :

Smooth Colored :

​

 

All comments are welcome eh?

Well everybody, there was a discussion about this before here: http://forum.unity3d.com/threads/14...te-Terrain-Infinite-Object-Fast-and-Optimized

His price point was 3,000 euros. So while this is outrageously awesome, it pretty much takes the 'indie' out of 'indie game'.

 

Hi Psyckosama.
This subject is already well developed on several pages of my other post :
http://forum.unity3d.com/threads/15...arching-Cubes-Robust-Architecture-FPS-Control

 

Maybe you can post some videos that show drawcalls and other statistics ?

 

Never said Indie meant poor quality, though much of the time it does mean extremely limited budget. Most start ups companies and one man shows simply can't afford that large an investment.

Now, I'm not saying you're terrain engine is poor, anything from it. I'll be honest, it's an amazing piece of work.

Honestly I'd like to suggest that you juggle the price a bit to maximize your profitability by balancing the individual unit price with volume of sale. Maybe even take a page out of CloakWork's book and have multiple versions.

Shroud Studio for example, comes in two versions. One is the free version that while still extremely powerful, somewhat reduces the functionality and is contractually unavailable to companies that gross more than 100,000 USD a year who can afford to pay the full price. The second is the Professional version that costs $599. Unlike the Free version it doesn't have the same technical and contractual limitations.

Fluvio also does similar with multiple tiers aiming for multiple price points.

I'd like to humbly suggest you do something similar. Sell the full version at a higher cost, and then have a reduced price/reduced capability version for the Asset Store with a similar gross income restriction. It'll serve as a low price point introduction for start ups, a demo for those who are interested in using it but would like to try before they buy, and really, as a marketing tool that will bring attention to you and your creations.

But I've said my piece, and best of luck to you. This is an amazing engine and I think you have the potential to make it one of those extensions that most people just use by default. Believe me, volume of sale can make up for a world of ill.

 

New Pooling System + Garbage Collector Optimization : Infinite Terrain Size.
You can now have extremely big Terrain Size without any Garbage Freeze/Lag.
The Pooling System is completely generic and can be used for your game, You can all types of object, the memory usage is constant after the first loading.

Procedual GameObject LOD system :
- ++3000 trees (LOD still contain colliders for distant interaction like grenades) (4%)
- One Rock on ground every 3 voxels (33%)
- One grass per voxels (100%)
45 Fps :

Some screens : (Memory 550Mo)

http://imageshack.us/photo/my-images/826/terrainengine2013041919.png/

 

Does this work on Android? does it work on the OUYA?

I am looking for a extremely optimized Terrain solution for the OUYA console and PC, Mac and Linux. I would prefer to go over the details of my game via PM if possible.

 

Yes. I've made video on touchpad.

 

WIP : Marching Cubes Fluids :

 

Booya

 

So this is 3K???? That's twice the charge for unity?

 

@john essy : No. You can contact me for more information

 

dyox...regardless of the price, that is one heck of a framework you have there. Kudos to you for driving that forward, constantly.

About the water. Do you handle flowing water, or is that up to the programmer, and your implementation is mostly just showing the water volume?

 

Thank you. Yes water flows. You can put water on top of a mountain, it will sink to the bottom. Water can remove the plants in its path and uproot trees.
it is still under development. 60%

 

I know this is only somewhat related to your project...but what your thoughts on how feasable a custom tree + lod system instead of Unity's tree system? Any thoughts about performance, and how it compares?

 

Hum. It's very easy to do. The LOD is procedural. At game start , the game take screens of Tree mesh.
For the Perf, UnityTerrain Tree LOD is more faster , but it's question of ms.

 

Thanks, Dyox. I'm mainly asking because using Prefabs with LODs allows for rotations and more control over LOD groups (obviously). Since you are using this method, I was curious if it was really that big of a performance penalty. It sounds like it's not! I think I'll do some testing.

 

Ho , i'm not using LOD groups. It's multithreaded system.
Distance/Camera Movement check on Other Thread. Billboarding system on LateUpdate()

 

Hey, Dyox - Very nice!

Would you mind shedding some light on how you made this awesome terrain system?

Put it on a blog or something I would read it

I am personally thinking that I might be buying your terrain system in the near future.

- Ben

 

Just a suggestions: I'd pay good money for a more up-to-date vegetation system, separate from the built in one

 

@HeadClot88 : I will not explain my work. Only doc for the terrainEngine. It will take too much times to explain all features.

@hc_lvngstn : Vegetation system need terrain, it's working with voxel position/volume. Marking separate system, it's remake another complete package.

New video : Marching Cubes Trailer :

 

This is starting to look very good.
Any eta on the executable demo? Would like to see how it runs for me.

 

@ronan : Thx. I'm working on Demo, with GUI for all terrain options. In 5-7 Days i will release the First Demo with 3 Scenes. Desert,Tropical,Forest and menu for modify all options like Tree Density, water level, grass density, Rocks, etc...

Some new Screens with Tropical Scene :
++ UnderWater Effect
++ Generic Script for Handle Current VoxelType.
** WIP Water

 

You mentioned fps (with without video capture) a few time but not the machine specs, on what hardware do you run this? (cpu vidcard mem)?

 

Also, is all this DX11 compatible or not tested there?

 

The engine is tested on 4 Computer. (50 -> 180fps)

Computer 1 (Low): (50fps constant even when loading)
CPU : Core2Duo Intel, 2cpu, 2ghz (Load Time 15sec) X86
GPU : Geforce 6600GT (512 mo, 400 Mhz)
RAM : 2Go (DDR 2 667)

Computer 4 (Max) : (180 fps constant even when loading)
CPU : AMD FX-8120 8cpu (Load Time : 3s) X86
GPU : Radeon HD 680 GTX (4go, 1ghz)
RAM : 3Go (DDR 3 1333)

Touchpad IconiaTab : (30 fps even when loading) (Grass system reduce the FPS, LOD system doesn't work)
CPU : Tegra 4, 1ghz
RAM : 1go

Project works on All Unity Version v2 -> v4

It's DX11 compatible.

 

Interesting, so perf is ok on low-midrange and scales up nicely with core numbers.
Nice that it's DX11 compat!

 

Yes, the multithreading system use all Computer capacity. On all screen/video the terrain use maximum 1go of ram (it's my limit with all computer) and as you can see, the range of view is very very correct for game.
I've just take some new screens : Ram usage 990mo and fps is around 50-60. (i'm on my low computer)
The digging system is starting to recude the FPS due to Grass/Trees/Rocks on the ground. I'm working on it actualy.

 

Great progress, as usual! Neat stuff!

 

Does terrain size at same quality increase linearly with size? (2X size on each side = 4X memory used?)
So 4GO = 2X 16GO = 4X on each side? Will it scale up to that? If so that sounds like it could give some really nice results even quite far.

 

Yes, each block take X memory. it's linear (except for mesh memory that can have different size). So if you're on Block with many vertices it will take more memory.

 

Ho, and I forget to mention. All screens are on TerrainScale X1. So as you can see it's far , but you can scale the terrain. There will are more space on each grass but i'm working on Grass Resolution so it will be possible to have X grass per voxel than one (actualy one voxel of grass = 12 planes)

 

I've made new video of Systems :

 

Some new systems :

-New LOD system with : Billboard,Procedural Texture, Procedural Plane/CrossPlane
-New Triplanar Shader with : Noise, Scalable Texture/Noise texture/ Normal / Parrallax / Light Power / Normal Power / Global Color / Noise Detail
-New GUI with Terrain logs, Pool System Informations.
- Marching cubes Optimization :
16*16*16 Chunks Generation Time : 8.10ms before : 1.0ms now. (Mesh Simplification , 1800vertices before, 505 after)
-Memory Optimization : 5mo/block before , 2mo/block now

 

Can you post some video of the water flowing?

 

@MikaelTroc : Fluid fow is still under dev. But i've made video for you

 

Some new Fluid Progress :

-Alignement Fixed between 2 Chunks :
-Fluid Flow algo optimizations : (16*16*16) chunks
-- 5.8 to 6.0ms Before
-- 2.3 to 2.7ms Now. (Data Caching, 2 pass, one fore displacement, second for data modifications)

-Fluid Mesh Generation optimizations :
- 6.0 to 8.0ms Before
- 3.5 to 5.0ms Now: ( Chunk Borders vertex reuse)

++ Added Infinite Water Block type. It's possible now to make infinite cascades, rivers and the ocean flow more faster on plains.
It's possible to set infinite water block in top of mountains, and it will generate cascade.

++ Added Evaporation system.
Water can disapear after some times , if the volume is under X, or if around blocks have some Type. (Like sand absorbs water, The lava evaporates water, etc).


WIP :
Generic Water Event scripting.
It will be possible to add particles when water block move, have block around. Change the Water Block in other Block if for example water is close to Lava , that make Stone. Or add foam on the waves.

 

Revision : 4013.
-Forest scene with fluids
-Optimized LOD system
-Optimized Fluid Flow generation
-Memory optimization
-FPS increased

 

Looks really good. Are you still planning to release the demo ?

 

How are you shading the water? Custom or existing solution? Does it look as good when the water flows? Would it look waterfall like if i suddently removed a side of the land? And if the answer is yes to all these questions, are you planning to sell those shaders standalone? I'd be interested in that outside of the voxel world.

 

@MikaelTroc : Thank. Yes , i will provide demo soon. I don"t have any date for the moment.

@ronan : I'm using the unity water4 material. It look good when water flows yes. Yes , and you can script the water fall event and add particles.

Some new screens of the Terrain Height function :

Code (csharp):

1.  
2. Terrain.GetHeightFromWorldPosition(float x, float StartY, float z, byte BlockType);
3.  

You can get the Height at any point of the world, with the block type you want. For example , the grass system detect the height of ground type Grass. It's 3D detection, so you can get the height of undercave or flying island for example. So it's possible to have different height at same X/Z point
You don"t need to use any raycast, and the function is optimized and can be called an unlimited time in any update. It can be very usefull for any IA,GameObject placement,etc...


And the grass system GUI :
You can select the number of plane per voxel , the scale min/max, the chance. All is procedural. And soon, it will be possible to add noise function to the grass placement. So it will be possible to change grass type in function of current Biome Type.

 

TerrainEngine : 4015 :
-New Grass Shader
-New Ground Shader
-New Grass Type : WEED
-Fluid Flow 95%
-20 new plants
-20 new grass/weeds
-5 new trees
-Memory optimizations
-Grass optimizations

 

New Video : Rev 4015

 

Is this some kind of Multiplayer Ready? That would be amazing if it came with its own small server software Especially for the price.

I would LOVE to buy this. I can see so many project possibilities with it.

 

@FatalNickle : For the moment no. But i'm working with company for an easy implementation. Like Voxelfarm, i will provide complete Server with map generation/storage/ Database management and TCP network.

TerrainEngine : Rev 4016 :
-New Generic and Easy to use Biome Generator

 

TerrainEngine : Rev 4017
-Advanced Biome Generator with Preset, Noise Selection/Blend + Shema
-New Editor GUI

 

TerrainEngine : Rev 4018
- New Tree Node GUI. It's possible to select witch Tree will be spawn at Any BiomeType/Height/GroundType
- New NoiseLib : Noise.GetValue return now all Noise Used (Mountains,Desert,etc..) And it's possible to change any terrain details depend of noise used.

 

Where do those plant/tree assets come from? I like them (or if they're made by you, any chance they'll appear on the store?)

 

All assets on my engine are free from internet. On google : "Free 3D models or Free 3D Plants, etc..".
'Turbosquid' or something like that.