Large Terrain Problem - how to recreate this in Unity?

This shot is from Google Earth - 13,000 m high looking across the English Channel.

Can I do this in real world scale in Unity 2018.3? I'm not making a game but a movie. I'm not planning to look at detailed trees - a textured terrain would be fine. But I'd like to be able to move around this area and look at aeroplanes from various angles and see this sort of image in the background.

I've experimented with large planes 200,000 m square but run into all sort of weird culling problems which I cannot solve.

If you could point me in the direction of someone who has done this with some tutorials that would be great.




Fewes has done something like this in Unity but I can't find any tutorials on this sort of thing

Please let me know if I can provide further info

Thanks

 

If you need a tutorial just to start you off with it, you really should not do it. You should do smaller things and learn how to do stuff.

There are many techniques, you can use impostors to fake the stuff at the distance (create low res meshes), you can use simulated stuff if the player can't actually travel there. Or you can use Worldstreamer or Sectr to load/unload pieces of terrain when you need it. And probably you will need to combine these techniques. Plus it's good if you familiarize yourself with the floating origin technique as well.

 

@Lurking-Ninja thanks for your reply.

I didn't realise how much smoke and mirrors was involved and thought it might be straightforward and that someone might have made their techniques available. Apparently not.

I've done some reading in the past few hours and see now that it is definitely not straightforward. I can't have infinite clipping planes cos that causes floating point problems, most streamers are for onground walking/runners not 13 km up in the air where you can see hundreds of kms at once, looks like LODs/imposters are going to be part of the solution.

Oh well.

Thanks for the floating origin tip.

Cheers

 

I hope it is OK for me to put my notes here. If not please let me know mods.

So it appears there is no one stop tutorial for solving the "Continuous World Problem" so I am going to put my findings here in case others may benefit. My goal is get something working and put the scripts here for the Unity community (if time and my skills permits). My particular interest is in using GIS/DEM data to sim real world terrains.

We start with this talk from Unite 2013 where Robert Oates talks about GIS Terrain & Unity

My takeaways from this talk:

Solving the Continuous World Problem is not easy (who knew?) and the solution is a work in progress.

Problem: Memory Limits
Finite memory, lots of content
Height maps
Render meshes for terrain and other objects
Collision structures
Terrain textures
GameObjects - agents, props
Scripts on GameObjects

Problem: CPU/GPU Limits
Finite CPU, lots of systems
AI/ Behaviour - increases complexity by #objects^2
Terrain object collision every frame
Object object collision every frame
Animation
Scripts
Culling and Rendering

Problem: Numerical precision limits
Transforms use IEEE 754 float
Have gaps between the 2^32 values on a 2^129 domain
Least error near 0
Most error near +/- float.MaxValue
Numerical instability far from origin
Visible symptoms at 60 fps
@ 100,000m float errors are 0.0078m componded up the transform stack at 60 fps
at 20-40,000 it will look really bad

Solution: Streaming terrain in any direction
fundamental ideas - Stream terrain around frustum
moving tiles
terrain built at runtime not serialised in scene, dynamically constructed
database used to persist tiles and objects and stuff
queried at runtime

recycle terrain objects
keep frustum in centre/ keep tiles near origin
persist/retrieve objects
stream imagery and heightmaps
numerically stable
constant CPU/memory load

Method:
generate a n^2 terrain mosiac
store for later use
center on origin

each tile has lat/lon independent of world space
SW corner is origin with lat/lon

each tile gets imagery splat texture from disk, cache, remote server
each tile gets heightmap data
if tile moves clear imagery

Database Persistance
lat/lon coordinate
orientation?
query database by lat/lon bounds

Implementation (2013)
"good computer" 17 X 17 terrain tiles
10km X 10 km loaded at once

Optimisation techniques (2013)
reduce number of tiles loaded in memory at once
distribute tile loading across multiple frames using coroutines
recuse size of tiles
reduce fidelity of heightmaps or elevation data

 

Unity's solving this at the same time as you are, for what it's worth: terrain data streaming, ECS (see megacity demo) and so on. If you were going for this, I would be looking at just seeing where ECS takes you as you'll be able to move huge chunks of data around without it taking any time to do so.

 

Thanks for commenting @hippocoder

"Unity's solving this at the same time as you are" lol - well I guess we'll see who gets there quicker!

ECS is on my list of things to learn about - several people in my googling have mentioned it.

Will check out megacity - cheers!

<edit>Megacity Demo

 

Here are some scripts to illustrate the basic concept of "streaming terrain chunks".

We have 9 terrain tiles with random noise heightmaps and as the camera moves the tiles behind us are recycled and moved to in front of us.

I've positioned the camera to highlight how the chunking works. It is the same 9 tiles over and over again.



The UpdateTerrainPositionsAndNeighbors() method is the core function here. Note that SetNeighbors is called every time the UpdateTerrainPositionsAndNeighbors() method is called - this makes sense because a tile may have new neighbours when it is recycled.

The main Update method has two objectives - keep the player on the central tile and keep the player from moving beyond the outer bounds of the 9 tiles.

Instructions
Create Terrain in Scene
Create Terrain Layer, add grass texture
Attach Extended Flycam script to Camera
Attach InfiniteTerrain.cs to Terrain object, drag Camera to Player Object slot in script, drag New Terrain Layer to Layer slot in script

References & Credits
https://github.com/Meowmaritus/Unity_SpaceGame/blob/master/Assets/Scripts/InfiniteTerrain.cs
https://www.reddit.com/r/Unity3D/comments/2q4ghg/procedural_terrain_generation/
http://wiki.unity3d.com/index.php/FlyCam_Extended
https://forum.unity.com/threads/terrain-layers-api-can-you-tell-me-the-starting-point.606019/
Texture from Terrain Toolkit
https://assetstore.unity.com/packages/tools/terrain/terrain-toolkit-2017-83490

 

Getting real world elevation data/ height maps into Unity - Show me the data

Note: This information is presented as a summary of stuff I could find in a couple of hours – it is not an exhaustive survey of all resources and providers. If you know of alternatives please let me know.

Earth is not a perfect sphere, mapping real-world elevations is not a simple exercise.

A location on earth’s surface can be referenced with a lat/lon coordinate. This Wikipedia page has a great explanations of lat/lon decimals degrees. A 8 decimal place coordinate would map to a mm – so for our purposes a 4 decimal place lat/lon coordinate will probably suffice.

Several mapping systems/standards/projections exist - WGS84, Google Mercantor, etc. It is important to know what projection standard the dataset is using because it may impact how the data you acquire will display in Unity.

There are many real world elevation data providers. We’ll look here at NASA/USGS, Microsoft Bing and terrain.party.

It is worth noting that depending on your use case higher resolution elevation data might be available for your area of interest – maybe a LIDAR survey. For example ESRI has a nice post about higher resolution (1m in some cases) heightmap data of for particular locations.

It appears that the Shuttle Radar Topography Mission SRTM in 2000 is the baseline for publicly available real world terrain data/ heightmaps. NASA flew the Shuttle over the earth in 2000 and bounced/measured non-lethal radar beams to get elevation. The first public release was at 90m resolution (STRM 90) and a later release of the same data was at 30m resolution (aka STRM 30, SRTM Version 3.0, SRTM Plus). Bing Maps has 10m resolution for the US but has a 1024 sampling limit (which I have not yet understood with regard to Unity tiles)

Not all of the earth was mapped in the STRM 2000 mission, the polar regions, for example were not in the release. Derek Watkins has nice visualisations of the data here and here. The STRM data is also noisy and has voids – do not expect perfection. The accuracy is also questioned by some GIS practitioners. Nonetheless it is an incredible achievement and a great resource for Unity developers wishing to get the real world elevation into a Unity project.

A number of providers including terrain.party and CGIAR have merged other elevation datasets with STRM to address the problems of voids and accuracy. This is obviously great news but could potentially lead to issues when you are asked “Where is this particular datum from?”. Personally I'd be using the CGIAR dataset if I need something canonical. You can read their FAQ here. Be sure to understand the licensing agreements and any costs associated with using the data provider. Also ensure that you get your to understand the licensing agreements with regard to commercial/non-commercial use of derived products.

The 14,278 .hgt files for planet Earth are available as from NASA EarthData (registration required).

Each .hgt file represents a 1X1 arc second degree snapshot of the underlying terrain. At the equator this equates to approximately 100x100 kilometres per tile. As we approach the poles the real world dimensions of the tile change significantly.

The 16bit RAW file is a 2d array with the centre of each pixel representing the averaged elevation over a 30/90m window of a real world lat/lon location.

Here's a pic from CGIAR illustrating how the SRTM pixels should be interpreted.


Source: http://srtm.csi.cgiar.org/faq/

 

These videos show SRTM data for Earth in Unity 5 from Technische Universität Darmstadt. Apparently 122GB of data - "Calculating all terrains from SRTM and Blue Marble raw data takes round about 7 days"

 

Just adding these 2 notes for future reference.

http://ninetwopro.com/ obviously not Unity but an open source (and textured) world terrain dataset. Not sure if can convert from MBL format to something that can be imported into Unity

NinjaFreede's Infinity Zoom in Unity. The technique is very interesting: "It's all fake. 5 cameras are set to show one layer each (galaxy, stars, planets, ships) with synced rotation looking at the same target. Each layer "scene" is below 1000 units so no floating point issues. When you mouse wheel, the cameras move in and out in order at different speeds." via Reddit

 

Oddly enough im trying to figure this out also, I just want to make a virtual Ireland, not using satellite imagery but apply rules for texturing different heights, then use easy roads with the aid of OSM data to create roads, im stuck at rock bottom however and haven't made any progress as of yet

 

I found this tutorial if its any good to you
https://gamedevacademy.org/how-to-h...ating-distant-regions-to-improve-performance/

 

And you might find this of interest

 

hey @iichiversii thanks for this information. I've put this on hold/ moved on/ given up and am trying simpler stuff in 2D animation.

When I started this thread I mistakenly thought I could simply generate a big mesh and then run around in Unity. That is not the case - like I say in the posts above there is a lot of smoke and mirrors/ kludges/ workarounds/ creative coding etc to get this effect. And these aren't limits/ problems with the Unity Engine but rather problems like floating point math and our current CPU/GPU architectures or the unsightly terrain topologies using NASA data.

I ended up purchasing a real world terrain package from the Asset store but that had issues with placing items in real world locations and keeping accurate location at different scales of views. The developer was very helpful but again this was a limit of external third party terrain data and not a problem with Unity or the package.

If I may give you some advice (and please excuse me if you are an experienced developer and know this already) - start small. Like really small. And then cut that goal into 3 easy pieces and tackle each piece separately. Solve that one problem and move on to the next. Personally I like to script/ automate my workflow so I know I can repeat it - and the cool thing I just did was not just a product of late nights and cold beer.

Here are a couple of links that might help:
OSM projects on UnityList https://unitylist.com/browse?search=OSM
Openworld workflow https://forum.unity.com/threads/realistic-openworld-workflow.593788/

this looks new and might give you some ideas https://github.com/Microsoft/MapsSDK-Unity

best of luck and Cheers!

 

That's the type of creative solution I love.

 

Hi,

I came across this thread and just thought I'd mention that I'm working on CityGen3D, which is a real world terrain generator that auto heightmaps and textures terrains in just a few button clicks all within Unity Editor.

It can also generate roads and spawn other prefabs automatically based on OSM data, as well as having a tree and detail placement system.

More info at www.citygen3d.com and on work-in-progress thread here:
https://forum.unity.com/threads/bet...cedural-city-generation-from-map-data.514677/

It's currenty in Beta so not all the issues mentioned in this thread have been solved yet (re world streaming, etc), but it's already quite a powerful tool.

Hopefully on Asset Store later this year but Beta already available via website.

 

You can try my GitHub repo