[Official] New Terrain System

oh, that is really bad news.

 

Thanks. It is a rendering solution only as it picks up the prototypes and distribution from the terrain.
Due to the abstraction unity's terrain uses to store and build grass the amount of data which is needed to populate a 600x600m terrain is about 2–8MB. Storing per instance data instead (localToWorldMatrix) would need about 300MB.

It supports grass, smaller foliage like fern or nettles and small objects you would paint using the built in vertexLit shader like small stones or any other kind of greeble.

 

Yea, I see. For my system it's an all in one, both painting and rendering with custom shaders.

However your approach might be better for huge quantities, instances take a lot of space. The bad part of my system is the fact that at the moment it only uses instances making it bad for huge quantities. It would work kinda of for terrains that are about 4x4km in size but it would not be a solution for anything larger than that.

I have an API for runtime appending of instances but I also need some other logic to generate the foliage on the fly... I've thought at detail maps, spawning volumes with rules, horizon zero dawn style spawning etc... but implementing any of those will take a ton of time. I'll have see what solution might be the best for my case. Maybe the integration of some 3rd part API that already does that would be the most beneficial. Since with tools like gena I'm sure the market does not need any extra procedural spawning tools, but tools that render all that data in an optimal manner...

 

@larsbertram1 Can you explain the workflow already for your new asset?

 

you add grass, flowers and foliage to the terrain just the way you are used to: either painting it by hand or using tools like terrain composer, map magic or gaia.
then you attach the grass manager script and hit play. done.
of course you may do some tweaks afterwards

 

the advanced terrain grass package has been released:
https://www.assetstore.unity3d.com/en/#!/content/100014

 

What use-cases, features, workflows do you have or would like to see?
- Tessellation shader
- Compatible with custom trees and SpeedTree's tree
- UE4's like brush gizmo
- Accept custom shaders for grass without adding them into the terrain shader body
***- Road system based on textures and not based on actual geometry (if it's possible)
- More sample files based on unity pbr shaders (tree,grass,rock)
- Better brushing values. I always use "1" value as brush opacity to paint slowly. Higher values are unusable (too fast)

 

https://forum.unity.com/threads/nested-prefabs-lets-remind-unity-team-that-we-really-need-it.499330

#WeNeedNewTerrainSystem ?

 

#WeNeedNewTerrainSystem

 

I would just like to point out that the hashtag idea, while cute, is actually quite annoying in practice. I can understand wanting to annoy the actual staff into making improvements, but given that normal users make up 99% of the activity on the support forums, and some of us are set up to receive notifications on threads like these every time a new post is made, this seems a less than ideal way of actually doing that.

 

In response to the new thread at:

That's actually a pretty clear answer. It's not a priority and there won't be releases any time soon. And I'm guessing they'll feel even less inclined to put a lot of effort into a new terrain system once TerraFirma and Vegetation Studio are out. Plus, from what I've seen in the last years, I'm quite sure that no amount of forum lobbying is going to convince Unity to dedicate a larger team of developers to this issue. So, don't count on Unity, if you need better terrain you'll have to bite the bullet and get all those terrain tools from the asset store.

 

#WeNeedNewTerrainSystem
[FONT=Roboto, sans-serif]Fight for our engine, Fight for Unity. [/FONT]

 

Looks really nice, but one of the biggest complaints with existing Unity solutions like grass is the poor performance. Does your grass solution perform better than Unity grass or just look better?

 

Throwing in my new foliage system based on Unreal's one:
https://www.assetstore.unity3d.com/en/#!/content/99233

For people interested on both performance and awesome look! Also check my forum post for more details on it's features.

 

@Assembler-Maze Thank you very much for your efforts. I've seen post of yours where it became appearant that you're working on it for a very long time already, so THANK YOU very much.
A have a bit of a side-question: Do you think Unity could do something with their terrain core engine that might even help your asset, or do you think we'll have it like it is still for a VERY long time, and we should accept it as it is?
Could there be any enhancements by Unity that even your asset would profit from?

 

it performs better on the cpu – as it does not produce any spikes anymore.
it may however take some more time on the gpu.
but atg comes with a lot of techniques to speed up gpu performance so in case you make use of these gpu performance most likely will catch up.

 

There could be some very specialized optimizations (like merge-instancing) that I would benefit from but I think that it might be a little bit of overkill. With the 'Instanced Indirect' more and GPU caching I don't know if I would have a lot more to do in order to optimize the system. Ah yes, there would be GPU culling, that would take even more pressure off the CPU. And there would be some micro-optimizations I guess, but it is quite fine as-is right now. And ofc, moving it to C++ . The C# update loop is about 4x slower than it would be in C++.

Doing it 'right' would imply the Unity team take a radical approach and take it right into their engine, C++ core.

And for the other question, there will be improvements in my system, bugfixes, performance micro-optimizations etc..., but I will never modify the core.

 

You mean you should take your system to C++? Sorry, I didn't quite understand the C++ part and who should do it. And where.

 

The unity core team should do it

 

This isn't a place to idly chat or promote your own assets, though I get why. It's still an official feedback thread and should be left for people giving actual feedback from use. It may be too late to be usable by Unity but you never know...

I'm not concerned because Unity staff really should be giving regular updates on this to keep us rabble contained and they're not...

 

I've given all my 10 votes here:
https://feedback.unity3d.com/suggestions/dedicated-unity-blog-about-terrain-core-update

 

I gave 7. Let's make a critical mass!!

 

Finally, some news from the Unity Terrain team.
An image of their current work has leaked:

 

Already around 350 votes.
Keep it coming, guys (and girls!).

 

I will post my blog post here since I really should've looked for an official topic like this to share my thoughts anyway.

As the creator of Terrain Former, a manual sculpting tool for Unity that's purpose is to build upon the frustrating built-in Unity terrain tools, I know a moderate amount about Unity's terrain system's shortcomings. In this post I want to talk about them all in hopes of getting them addressed in the future versions of Unity.

Use a More Suitable Base Data Type for Heightmaps/Alphamps
Unity's terrain data asset uses of a 32-bit floating point precision number for heightmaps and alphamaps, while only uses values ranging form 0.0 to 1.0, which means it's only using 25% all available values. Of the values used there are noth of 1 billion values used, which is overkill - not only that, but it would make a lot more sense to encode these values as an unsigned 16-bit integer value type instead for fixed precision, and to use half as much memory. A floating point number would contains lots of tiny fractional values close to 0 which are unnoticable for a game to make matters slightly worse again.

There could be an option to set which data type is used including uint16, custom 24-bit encoding and uint32. With a unsigned 16-bit integer there are 65535 unique values, meaning with a 100m (328 ft) tall terrain there would have a 1.56mm (0.06 or 1/16th inches) height step between each values which more most circumstances will be more than enough. There would be ~655 unique height values per meter in this case.

Alphamaps actually are 3D-dimensional, with the depth being equal to how many textures are on the terrain at once. Once again it uses the same rang (0.0 to 1.0), but memory usage increases in three dimensions instead of two, which is already a very lot. Also I wonder if you even need much more than 8 bits (256 shades) per alpahmap layer, so much less than the 1-billion non-evenly distributed values currently used.

Scalable To Meet All Needs
Terrain Former can work across multiple terrains at once in a grid pattern (a terrain grid). The only reason this feature exists is because sometimes you want large terrains with a total resolution over 4097. This feature took me months of full-time work to implement due to the massive amount of refactoring and complexity this brings when the plugin was initially designed to handle one terrain at once. This also adds complexity to users as they have to deal with terrain grid setups, and if they change there minds it's hard to add/remove terrains to the grid, or move to only one terrain.

Compression Options for Heightmaps/Alphamaps
Unity currently applies no compression to heightmaps and alphamaps. By default Unity should losslessly compress both maps with an optin to compress them lossy for devices where storage and/or memory usage is more restricted.

Allocation-Free APIs
Calling TerrainData's GetHeights method allocates 32 bits of GC memory per sample, which can easily weight in at >200 MB even with just a regular sized terrain. There is no way to re-use the same array when gathering all terrain samples across a terrain grid. An allocation-free API would also benifit other uses such as terrain deformations or any other realtime terrain modification.

GPU-based Sclupting and Painting
Terrain Former is now multithreaded, but even then it's fairly heavily optimized C# code that wouldn't be close to matching equivalent C++ code, let alone the level of what a GPU-based implementation could bring.

The Little Things, Together
Having every little thing refactored, streamlined, made more robust, optimized and put all together into one single tool would truly be amazing. Imagine Terrain Former with more features, but also integrated directly into Unity leading to an even better, more optimized experience. I'll try to list everything else that doesn't need to be discussed in detail (yet):

• Everything already present in Terrain Former. Including far more powerful tools that can be used on large areas of the terrain (Unity's maximum brush size is only 100 heightmap in size!) and speed improvements.
• Speed improvements. Updating a heightmap internally takes a realtively long time, it surely could be made quicker.
• Vegetation placement/distrubution.
• Built-in instancable Sub-Surface Scattering shader for vegetation.
• Instancing support in general for vegetation (already confirmed by Unity in some Unite talk)
• Terrain's physics material should differ per texture, for example sand is slippery and less bouncy than cement.

Bonus: Voxel-based Terrains
Voxel-based terrains would be the ultimate, allowing for concave cliffs, caves and everything in-between. The downside would be they are more complex to implement across the board, but I'd say it'll easily be worth it.

 

TO be frank, they have so much emphasis on optimizing unity now, that I can wait for terrain, because it seems that's a prerequisite before touching terrain.

 

The Unity terrain system made me switch to Unreal. Unless You spend tons of money on the assets in the store, You won't get even remotely decent looking and performing terrain (and still assets make it only slightly better).

I think one of the core features to add (probably mentioned here somewhere ) is World Composition like tool to create a really big worlds. The demands for more and more opend world games will only rise and with the current system You either use World Streamer which is great solution but extremly annoying to use or go with other engines.


Also at this point i think we are soooo deep into asset store stuff for terrains i think what Unity should do is to perhaps make changes to the terrain api to make it easier to use for the asset creators who can utilize them better, also focus on upgrades and optimizations to the terrain core. I am a realist/pesimist in life and i don't think we will see any sagnificant changes to the terrain system.

 

We're at the same point as @Rapapastor as I've also written here:
https://feedback.unity3d.com/suggestions/dedicated-unity-blog-about-terrain-core-update
I would need a statement from Unity or else I see no other way than releasing with UE while I prefer Unity in some ways.
I'm talking about dense vegetation speed and LOD limits, not so much about the paint tools.
What I'd need would be a clear statement. Else I really see no other way than not using Unity for release.

 

yeah i don't even remember when was the first time a Unity person told me they were working on a better terrain system... i remember it was Unity 2.6. Then on several Unite and events and forum and blog and mailing list... so... after all this years, i'm craving that release because it must be f awesome!

 

Wacom pressure support for painting operation and for placing operations

 

I've heard rumors that somebody of Unity will post something here or in a blog in the near future.

 

no news is good news right?

 

Define "Near Future"

 

500 units of time, at least.

 

They forgot to specify traveling speed. So time may differ from theirs.

 

EEhhh bummer

 

What's funny is that certain things are better than any assets with their terrain. Let me enumerate a few things
1) Their terrain, meaning the land "building" per se, rendering is horrible. Almost anyone I know of, convert their terrains to 3d mesh and use them that way. It's ridiculous that it's faster to render a 3d mesh with millions of triangles than an "optimized" terrain system. I never knew of any system that worked better in meshes for large terrains until i met unity.

2) Tree creator and unity billboard grass is faster than anything i've seen on the asset store. Nothing is faster than whatever Unity has done internally with the billboard of the terrain objects. That's a positive. I can render grass hyper fast compared to anything on the asset store.

3) CPU spikes are horrible. Yeah, I can render grass hyper fast and then... every 1.5 seconds it lags, cpu spikes... goes back to normal, repeat. If they fixed the cpu spikes for grass it would be awesome.

no that it matters. I mean, we've only had this conversation a few dozen times and provided feedback another dozen.

 

Vegetation Studio is faster.

 

For the most part the asset store has solved a lot of the related issues with terrain. The big one left is the terrain itself. And I think that needs to be re imagined from the ground up.

Height maps are dated, they need to go. But I don't think voxels are the answer. None of the popular approaches out there now actually do what a lot of newer games want to do.

You really need an approach that separates the basic shape/volume definition from surface definition. Why is because for 3d shapes a voxel like api is just easy to work with in code. But you don't want the limitations of a voxel surface. So both are still dependent yet separate.

So with something like the above you could define the surface first using height maps or the same procedural approaches used now, then fill in the voxels (sparse). Or go the other way. Or at runtime add/remove voxels which would cause the surface in that area to regenerate.

 

Yep although it is actually sort of part of Unity's terrain system work.

This is because Unity decided to work on instancing and foliage rendering first to upgrade the terrain. They released these functions early based on that, with the expectation that programmers or asset authors would start using these features right away until Unity manages to get around to doing their own vegetation systems plus it works beyond terrain, so it was a no brainer.

Not taking anything away from the hard work of Veg Studio's author - just a bit of a trivia, and people can see work is being done - just not where they expected it first.

A lot of people totally mistakenly think of terrain as being this all-inclusive lump of object that magically has everything built in. Rivers, grass, trees, rocks, texture blending, snow, water, everything.

That doesn't exist, it's lots of separate solutions and the rendering lots of objects problem is easily the biggest deciding factor if a terrain looks good or not. Right now, with something like microsplat + Veg Studio, you basically have better terrain than at least half of AAA games out there right now, today. Only art and art direction is missing.

The terrain could have a whole bunch of next-gen meshing features but it's important to note that a lot of AAA games don't come with those features either!

I definitely want terrain upgrades but I think doing the vegetation part first makes huge sense.

 

you know, at this point i don't mind if i have to author the terrain in 3d s/w. My only problem is the shading (have to deal with texture array splatting manually) part and foliage/terrain decal scattering and instancing.

 

And I would argue that implementing a new "Terrain System" as one lump sum like this would be a tragic mistake. With every implementation of any of these systems, there are huge tradeoff's your making which are going to favor one kind of game over another. Unity adding efficient instancing and DrawMeshIndirect was exactly the type of thing I want out of a 'new terrain system' - raw power that can be used to produce a wide range of solutions for all kinds of different games, not specific solutions that will be great for some types but not others.

 

No. It is not faster to render vegetation Studio than unity billboards grass. It’s actually about 1/2 the speed. I was beta testing vegetation studio and the author arrived to the conclusion that there was no benefit whatsoever on that case. Vegetation studio is thought for 3D meshes, not billboarded grass.

 

+10 of my votes!

In my fantasy world, I'd like Unity's terrain to do the following:

• Fully deferred shading option
• Custom ATOC foliage and grass as an option
• GPU-accelerated (maybe via async compute) LOD levels
• Tessellation as an option
• Weather effects as an option
• Handle large (4km+) terrains automatically with regards to performance/optimizations
• Customizable shader quality levels for foliage
• More threads, less main thread spikes

Some of these are fantasy ideas that I know Unity will never do, but I can dream. After looking at the recent stuff Unity have been pushing, it's more likely we'll get some sort of new XR movie-making cloud-enabled machine-learning gizmo that runs on a web-connected toaster than anything actually useful in production games.

 

Unity hasn't even made a change to Unity Terrain for over 8 years... and you expect them to add a "list" of improvements? Heck, I would be satisfied at this point if all they did was open the code and put a comment on it. I've lost hope on them. To give you some perspective. Years ago we were promised a much improved terrain system, then it didn't come but it was promised for unity 3. So I bought Unity 3 license, pro, big bucks. Several 3.x came... nothing changed. It was promised for Unity 4. So I bought Unity 4 too!... never came. I was at Unite a few years ago and asked Joaquim (the architect) about the terrain. He told me they were working on it with some new hires. Unity 5 came. I bought it too hoping this time some new terrain system would finally come. I mean, Joaquim had personally told me. It wasn't only the (i lost count) of feedback requests on the forum from Unity over the years about what we wanted on the next terrain update. So, now we are on Unity 2017.x, the terrain on the roadmap is not even on research area. I mean, if you wanted someone to be heartbroken about your product promises... they've mastered to perfection. I've been on their private betatester list for years, getting all those shirts and all. Lost track of how many times i promoted Unity and bug reported S*** that is fixed throughout all unity versions because of my reports. Unity 2017 is the first time I didn't buy unity. I feel scammed because of this. I don't begin one single new project on Unity out of self respect. I'm not saying Unity sucks. I'm saying they promised stuff for years that they never did neither gave an explanation. They let me down big time.

This thread has been dead for years (began in 2014), same than any Unity terrain votes on their feedback system, and same than the feedback/request system that existed before this one (2009?), and same than the threads before this new forum came to existence (2010?). It's prehistoric bs at this point.

 

Yeah I'm sick of that stuff. I don't want to see yet another nuanced fiddletronic zbrush UI for what essentially amounts to a 2 click job or a function call.

 

Which mean next time someone talking about terrain engine in unity we should response with "Nah, you just seeing/hearing things, it's just a myth"

 

Wow, didn't know that they promised so many times. Bad for them. Unity can be a very nice and comfortable environment to create big open-wide areas. And all that we have just an AR and VR crap. I hope we finally get our nested prefabs and prefab editor at the end of 2018 as they PROMISED.

 

Did they promise? Please quote as trying to get Unity staff to promise something would immediately raise several warning flags, a klaxon, some sirens, a medley of flashing lights and even a vague sad-sounding clown hooter in the distance.

Basically if it's in the roadmap under a specific Unity release number it's a promise, and anything else is just "vague WIP maybe".

 

You welcome. Click, there are timecodes. But as I see they just "trying" to deliver that system. So, as always, it will be a long story.