Assets Very Large Mesh Terrain Exporter

Update:

the Open World Terrain asset is now available on the Asset Store!

https://assetstore.unity.com/packages/tools/terrain/open-world-terrain-148431





ORIGINAL POST:

I've recently been thinking about making a game that requires a very large terrain, much bigger than the one supported by Unity and one which would need to be streamed in. I still wanted distant mountains to be visible though, so a simple streaming approach wasn't going to work. I investigated a few solutions on the asset store but nothing seemed to fit at the time.

What I ended up doing was creating a tool which can take a Unity terrain and export it into a set of progressively less details LOD "segments". For example, for a terrain of heightmap resolution 512x512px, it would export 21 128x128 vertex meshes to represent the terrain; where 1 mesh would represent the full 512x512 terrain, another 4 would represent the 256x256 quadrants of that terrain, and 16 would represent the 128x128 (or 1:1) sections of the terrain. Along with accompanying splatmaps for the 16 "real" segments and textures for the 5 lower LOD segments.

I've also created a runtime which loads these terrain segments with progressively higher detail depending on their proximity to the camera and the option to export "terrain entities" (such as trees, houses, NPCs, whatever) and only load them at specified LOD levels.

The final hurdle was terrain details such as grass and foliage which I implemented by using instance mesh renderers and compute shaders to dynamically place grass and other details as the camera moves around the terrain. They're also frustum culled to improve performance. It also allows for details to only appear on certain terrain textures and populate less and/or scale based on texture intensity on the splatmap (see pic1 and pic2)

The benefit of all this is that I can have large unity terrains, each within their own scene (so that editing the terrains isnt a pain), and export them to a format where I can render large mountains in the distance without the memory and performance cost of having many unity terrains or terrain meshes of all the same size. (see pic3 - this terrain is created from 4 2048 terrains and runs pretty well on my smartphone)

I'm posting this up on here to gauge whether anyone other than me would be interested in such a tool, and if so, I'll spend some time simplifying/cleaning the process and editor window (see pic4) and upload it to the asset store.

Hopefully I made sense here, but I can clarify if anyone has any questions.
I've attached a few images to hopefully make things clearer.










To summarise, here's the features that I currently have working and would be available in the tool.

• Export terrain as mesh segments and dynamically load in terrain segments.
  
• Export game objects such as buildings, trees, people and specify at which LOD levels they should be loaded/unloaded.
• Grass and other details such as small rocks and bushes can be rendered efficiently using instanced mesh rendering an frustum culling and compute shaders are used to dynamically place the meshes onto the terrain.
• Exports terrain texture splatmaps separately to meshes and supports more than 4 textures on a terrain.
• Support for terrain texture normal maps.
• Allows for very very large terrains with good draw distances.
• Can use Unity's new terrain neighbours by default or can be specified in the editor window (they don't have the be in the same scene for the terrains to "connect" correctly)
• Can be used as a simple terrain to mesh conversion tool for low poly background scenery at the very least.

Let me know your thoughts
Thanks

 

Very interesting! It would not be bad to be able to buy a ready-made system instead of doing it yourself.

But I have somewhat different goals. I need to have a terrain about 10x10 km in size with a detail of 4 vertices per meter near the camera. That is, the distance between adjacent vertices is 0.25 m.

This means that the height map for such a terrain should be about 40,000 x 40,000 pixels. Naturally, the standard Unity terrain is not suitable for this. I plan to implement a system similar to yours. Quad-tree LOD. But in my case, the data on the heights of vertices are loaded into LODs from binary files, and not stored in memory. For the entire gigantic terrain, only a few dozen meshes are used that are stored in the pool of objects. From the point of view of productivity, it is better to move the same object from one place to another than to create a new object each time and to delete the old one.

It is also necessary to load the colliders only for the closest tiles near the camera. For more distant LODs the collider is not needed.

So far, my implementation is at the idea stage and I have not conducted tests. Perhaps your project may interest me.

How do you load landscape segments? Do you use a mesh pool using the same segments, moving them along with the camera? Or do you create on the stage at once all the plates of the landscape of all the Lods with the necessary heights of the vertices for each, and then make only the necessary LODs visible?

Does your system allow for higher landscape detail than standard Unity-landscape, while maintaining the large size of the world? For example, one terrain 512x512 divided into 4 segments 512x512, each of which is divided into another 4 segments 512x512, etc.

 

Hey Lesnikus5,

It sounds like we're thinking along the same lines. It basically works exactly how you described except that It's currently loading and unloading the segments as standard Unity prefabs.

The main reason for this versus modifying existing meshes in a pool (which I agree, has many benefits - loading the meshes is done on a separate thread to combat some of the downsides) is that I can further optimise the mesh segments ahead of time - reducing redundant vertices, adding navmeshes, combining large low LOD cliff/rock meshes with the terrain into one mesh + texture (WIP), etc - I basically looked at how Skyrim was doing it, and from what I can tell they are taking this approach but it wouldn't be too difficult to try the binary terrain segment and pooling option, i'm tempted to investigate the benefits...
Because they're prefabs, it also means that I only add collisions to the lowest LOD terrain segments as you mentioned.

I only load the segments that are currently needed to reduce the memory used at any one time, rather than have them all exist but only visible when required. Because everything is destroyed and created, I don't have to move anything along with the camera. Higher LOD meshes are destroyed and replaced by 4 lower LOD meshes as the camera gets closer, continuously until they reach the base LOD. And then the same in reverse as the camera gets further from the quads.

As for you last question; since the exporter works on standard Unity terrains (It's not a separate terrain editor or anything like that - it simply converts a standard Unity terrain into mesh segments), exporting the terrain at a higher resolution is possible but doesn't really give you anything since the mesh wont magically have more detail than the original terrain - it will just interpolate between the data at the terrain's resolution.

What you certainly can do is create a 1024x1024px resolution terrain and export that at 1/4 the size which might be what you're after?
In that case, you would have 1x terrain of size 512x512 at resolution 512x512 and 4x terrains of size 256x256 at resolution 512x512. Children of those would be of size 128x128 at resolution 512x512 but they wouldn't have enough detail to warrant that many vertices as there is only 256x256 sample points for that quad (I'm not 100% if i'm making sense here or if I've answered your point correctly )

I should also add, that the sytsem I'm developing could probably handle a 40,000x40,000 terrain. You would basically need 100 scenes each containing a 4k terrain, but in theory you would only need to have 128x128 vertex meshes for almost all of those 100 4k terrains until the camera got closer. It would take some time to export but it's certainly possible with what I've got so far and should run pretty well. The biggest I've tested so far is a 16,000x16,000 terrain.

 

Is there any news? Are you still planning to release it to the Asset Store?

 

Hi, sorry for the lack of updates recently. Yeah, I'm still planning on releasing it on the asset store.
I've been putting in the final few touches (bug fixes, cleaning up the ui, documentation, meaningful error messages, etc) but it should hopefully be up on the store this week or early next week.

 

Just a quick update for anyone interested. The package is now in submission. I'm not sure how long that usually takes but hopefully everything is fine and I'll be able to have it on the asset store soon.

In the meantime, I've been creating videos and image assets for the store and figured I'd post them here as they give a bit more insight into what it is the tool actually does.

I'm currently working on creating a video tutorial based on the footage here:

 

I wonder how you avoided the appearance of artifacts (holes) on the border between two different LODs? I know two options: "skirt" and the adjustment of the position of the vertices on the edge of LOD with higher detail. Which one did you use?

 

I eventually went with the skirt approach.

I initially implemented an approach where I would modify the vertices at the edges at runtime since I was worried that using a skirt wouldn't look good.

I had originally planned to support both but after trying it out (and looking at how games like Skyrim and Horizon:ZD did it) I decided to switch to using only skirts as it reduces the complexity at runtime when loading a new segment since the other method requires each segment to be aware of it's neighbours whenever the quadtree splits. It also helps keep the code simple since each segment it's it's own entity that doesn't need to care about the rest of the terrain.

 

the Open World Terrain asset is now available on the Asset Store!

https://assetstore.unity.com/packages/tools/terrain/open-world-terrain-148431

 

Nice ! Can this work with Vegetation Studio Pro ? Or is it planned to ?

 

So far I've only used Gaia's procedural terrain generation and foliage placement which works really well and there's a only a little bit of setting up to do to make them work perfectly. From watching Vegetation Studio Pro tutorials (I don't own it myself yet) it looks as though the same should be true for that too.

Vegetation Studio Pro uses the same method (instanced mesh rendering) to render details as the Open World Terrain does, so it should be pretty simple to integrate it into the open world terrain tool. The overall goal is to make the tool more and more compatible with these terrain generation tools, so I'll definitely be taking a look at vegetation studio in the future, to answer you question.

 

Thank you for your very quick reply; if you can integrate VSPro then it will be an instant purchase for me; as there's no comparable solution like yours out there; just some hacks with SECTR and WorldStreamer. And I think if you make it you'll have many many (many) users from VSPro that would be interested in your solution.

 

Thanks for the input I'll take a look into Vegetation Studio Pro just to make sure, but since Vegetation Studio Pro's Asset Store page states that it works on mesh terrains (and OWT is a mesh terrain) it should just work.

 

It looks Great! Can this work with lightmap when running at different LOD levels?

 

Sorry for the late reply to this.You

can bake lightmapping onto the meshes but it doesn't do that automatically. You'd have to add all the terrain chunks into a scene and bake the lightmapping into the segments that way. But it's definitely possible.

The terrain LODs are simply textured meshes, so you can pretty much do whatever you want with them after the terrain export process

 

Hi, I have used all sorts of streaming solutions over the years, I work in the industry and also made my own solution, but a pain to maintain, so I bought others on the store, and I been fairly happy with them.

but can you tell me what is different from yours, compared to say World Streamer, Sector, and even Map Magic. Terra. and others. You had said you couldn't find what you where after, so just wondering, before I invest, what is different about yours .

Does yours handle floating point issues? what about for server side, which is typically the issue, do you plan to team up with Photon, or even add to Unity networking side of things. I get many DM's over the years, and this is one of the more common issues I see people asking to fix. now some of the others do handle floating point, but have not teamed up with anyone, or even added it for Unity, to give users a good starting point.

anyways, those are some of my questions. Thanks for any information.

glad to see some more people making these type of solutions I believe they are wanted/needed.