TerraLand 3 - Streaming Huge Real-World & Custom Terrains For Open-World Environments

Thank you, I copy too the smooth file from the first page and it pops these errors afte I over write the file

Attached Files:

 

Hi, that's exactly as @ForceVFX said. You can also simply rename the class name and filename so that the conflict is removed.

The new version "3.1" has been out since yesterday in your account on our website. If you don't have an account, register from here: http://terraunity.com/my-account/ and send us UAS Invoice No. so that we setup your account to get the latest package downloads. The AssetStore version is submitted too and waiting for approval.

In 3.1, we have removed all possible conflicts with Standard Assets and included all needed assets within the package. Here is the list of main changes in the update:

• Fixed Geocoding not working anymore and changed the source from Google to Bing
• Included Standard Assets related files within the project so no longer 3rd party asset import is needed
• Fixed project launchers pop up frequently
• Renamed PauseMenu to PauseMenuTT due to conflicts
• Fixed SmoothFollow script top avoid automatic overwriting
• Updated "TerraLand Terrain" component with minor bug fixes and improvements
• Works out of the box with .NET 4
• Initial testing with HDRP & LWRP is approved

 

In ver. 3.1 when you are building the scene, if you receive this error:

Code (CSharp):

1. ArgumentException: The Assembly Mono.Web is referenced by System.Web ('Assets/TerraLand/TerraLand Core/Plugins/System.Web.dll'). But the dll is not allowed to be included or could not be found.

Simply find the System.Web.dll in TerraLand => TerraLand Core => Plugins folder and delete it. The reference to this dll is not needed anymore and will be removed in the next version; so you can have a clean build with full TL3 assets in place.

 

At minute 2:58 in the tutorial video, a white terrain grid rectangle appears, then the terrain appears. However, on my Mac I see only a brief flash of white in the scene, but no terrain appears. I've tried with Google and Bing maps, small 4km size. Neither Static World nor Dynamic World shows a terrain. The issue occurred before and after upgrading to Unity 2018.2.13 and Terra Land 3.1. I do see "Terrain 1-1" added in the hierarchy. What am I missing?

PS: This problem is also occurring on a second Mac which had never had Unity installed previously. I updated the 2 .config files in about 7 folders. (But only in the "2.0" subfolders, should I do the other versions of mono as well?)

 

First of all, choosing different mapping sources in Interactive Map section is only for previewing the area and does not affect terrains' elevation/imagery data source as it is receiving its data from ESRI only:

https://forum.unity.com/threads/rel...world-environments.532304/page-2#post-3583949

This is where TL initializes terrain tiles and then applies heightmap and imagery on them, if for any reason you see them disappear or there are no terrain objects in the hierarchy existing after this stage, it may be possible that some 3rd party assets force them to destroy. Create a terrain from Unity menu and see if there are no general problems with terrains in your scenes!

Static World mode generates terrains in the editor and they will be placed in the scene hierarchy as objects but in Dynamic World mode, no terrains will be generated in the editor but a cached server will be created in a place you define for later runtime streaming of those tiles; we have explained the progress of these 2 modes in the same video you pointed out.

Does it mean that the terrains are there and they got their heightmap/imagery properly?

Unfortunately Macs are behaving differently in file management and some other areas where these occasions may occur and being caused by general malfunctions between Unity & Mac.

No, you just have to replace them under 2.0 subfolders and that's it. Also note that this step is only for having more asynchronous calls to the servers while obtaining data and has nothing to do with terrain appearance issues you have.

 

Yes, that is what happens, but I then get a warning message "Texture wrap mode must be set to repeat". This appears if I edit the terrain texture. I believe the reason may be that the texture image is smaller than the terrain size -- 1024 squared for the image texture out of TL and the terrain size is 1500 squared.

I'm not certain but I think that warning message is part of the problem in adding other terrain textures. If I clear the message by tiling then I can have other textures. Of course, as you note, with tiling/wrap of the TL texture then it looks wrong.

 

Yes, as already stated the warning is only related to the image wrap mode and has nothing to do with its scale and size. This is the same for every texture you want to add to the terrain system but as TL applies textures through the codes, it is possible to have it tiling clamped on terrain. If you want to change it to be tileable automatically, just open up the script "TerrainGenerator.cs", look for a line where it has the word "clamp" and change it to repeat to later have access to normalmap slot and metallic/smoothness parameters but if you don't want to bother changing codes, do this manually as before.

Also in Unity 2018.3, the terrain now uses per pixel normal rendering and there is no need for any global normalmaps on terrain, see below:



This is the same when you have a low-poly model but having the normalmap baked on surface so in terrain settings you can crank up the Pixel Error to save a lot of polygons and gain performance.

Also it now has the instanced rendering which sends a lot of process directly to the GPU and reduces a lot of drawcalls.

 

I just wanted to show an example of my TerraLand 'mobile' Terrain map..this is the 'Low End' 2.0 model Shader l ended up using. I Baked light in maya and encoded into alpha channels. All Terrain LODs and props/buildings are using just two unlit shaders, so this is best looking/quickest shader I've come up with out expensive normal maps/lighting..

I'll post both shaders, if any one wants 'em...baking a slight normal in the light maps helped a lot.

This is why I need close and far blends.

Landing and taking off.

p-

 

When I build (Android / Unity 2018.2.13f1) I get the following, seemingly related, error:

Code (CSharp):

1. ArgumentException: The Assembly Mono.WebBrowser is referenced by System.Windows.Forms ('Assets/TerraLand/TerraLand Core/Plugins/System.Windows.Forms.dll'). But the dll is not allowed to be included or could not be found.

I tried deleting this file, but considering the error that evokes, I guess this reference still is needed...

Code (CSharp):

1. Assets/TerraLand/TerraLand Core/Scripts/BaseCmn/BaseCmn.cs(48,74): error CS0234: The type or namespace name `Windows' does not exist in the namespace `System'. Are you missing an assembly reference?

Do you have suggestions for this? Many thanks!

 

@ForceVFX Thanks for sharing, would be nice to see the shaders; a lot of people are looking for performant terrains on mobiles.

@daniploeger I recommend deleting all TerraLand related folders in project after generating scenes when building, this avoids any extra assets to be included in the build for mobiles. Optionally you can duplicate the original project for build but without TL stuff and you won't need to be worried about these kinds of editor scripts errors while building.

 

Thanks, that works! Just to make sure (for anybody else who runs into this problem): delete all TerraLand related folders, except for Terraland>Terraland Core>Downloads

 

Thanks for reminding.

 

Does this means that Terraland is working on 2018.3.x with no issues? Thanks!

 

I am trying to import TerraLand 3 into a 2018.3.0b8 project, but am getting the following errors. Also interested in knowing if it's possible to get it working.

Code (CSharp):

1. Assembly 'Library/ScriptAssemblies/Assembly-CSharp.dll' will not be loaded due to errors:
2. Reference has errors 'System.Web.Services'.
3.  
4. Assembly 'Library/ScriptAssemblies/Assembly-CSharp-Editor.dll' will not be loaded due to errors:
5. Reference has errors 'Assembly-CSharp'.
6.  
7. Assembly 'Assets/TerraLand/TerraLand Core/Plugins/System.Web.Services.dll' will not be loaded due to errors:
8. Reference has errors 'System.Web'.
9.  
10. Assembly 'Assets/TerraLand/TerraLand Core/Plugins/System.Web.dll' will not be loaded due to errors:
11. Unable to resolve reference 'Mono.Web'. Is the assembly missing or incompatible with the current platform?
12.  

and then there are a lot of warnings like this as well:

Code (CSharp):

1. Assets\TerraLand\TerraLand Runtime API\WorldExplorer\Offline World (Experimental)\RuntimeOfflineWorld.cs(430,22): warning CS0618: 'TerrainData.splatPrototypes' is obsolete: 'Please use the terrainLayers API instead.'
2.  

 

@Alcyone @sharkapps TerraLand is compatible with 2018.3.x but you have to do these 2 following steps to remove the errors you may receive in the console:

First, go to Settings => Player and set the "Api Compatibility Level" to ".NET 4.x"




And then in the Project tab go to TerraLand => TerraLand Core => Plugins and delete the "System.Web.dll" from the project.




So all errors and warnings will go away and you can use TL without any issues. Please note that currently these steps are manual because we do not support compatibility for Beta versions and these versions are subject to change frequently.

These warnings won't affect any functionality in TL yet but this means that unfortunately Terrain API has changed a lot since 18.3 and we have to port all these codes to be compatible specifically for 18.3 & later. Currently they work and supported in Unity but once they are officially dropped and 18.3 is out of Beta we will release a specific version compatible with 18.3.x and later.

 

Thank you for the response. Following your instructions, I was able to get the Tournament demo working and a couple of the Offline Streaming demos as well, but I ran into a few issues:

1. WaypointCircuit.cs Line 257 was hitting iIndexOutOfRangeException. I changed line 255 to prevent -1 from being passed in as index

Code (CSharp):

1. 253            if (items.arraySize > 0)
2. 254            {
3. 255                for (int i = 0; i < items.arraySize; i++)
4. 256                {
5. 257                    var item = items.GetArrayElementAtIndex(i);

2. Tournament/Scripts/GameManagerTT.cs:120 IndexOutOfRangeException. I changed line 617 to fix

Code (CSharp):

1. 617            if(trackIndex >= audioTracks.Length)
2. 618                trackIndex = 0;

The last issue is an issue with the terrain textures not appearing. When I ran the TerraLand Runtime Bare Earth scene the first time, everything looked fine -- the terrains and textures appeared as expected. Since restarting Unity, however, the terrain textures do not appear (and there are no messages in the console). Have you encountered this issue on 2017.3?

Best regards,
Tony

 

Quick question, working on a new island map, noticed that the map display window shows a "zoom level" like in World Composer...Does it reflect the image output? If I zoom to 16 in the window, I don't think I get 16 in the output.

just ran the same coordinates in both at 16K (TL3 & WC / zoom at 16), and WC has better density per square meter, when both are sized exactly the same, and viewed side by side.


(I just wanted some small islands, man did I over estimate the size)

Spoiler: Examples/Some Captures: Same Place, same Size TC3/WC

TC3 8K (spent a couple of hours, clipping out the sea, to create a shore mask.)


WC 8K


Why it matters? its just a small part of this map:


which I then split into (2) 2k Splat/Textures and a 9k Mesh..
Enhanced Height map of 256 to 4K in World Creator, to bake 2k normal maps:

below, thats only 9k tris!!..super decimated..all the details are from that new (reduced to) 1k normal map, pretty much a "must have", for all sat height maps now.



of course, Up Close, (using my mesh splat/distance blend shader,) where the low density gets noticeable.

WC uses bing as a map provider, is that it?

Can I zoom in or out? to get a higher texel density per square meter? and how would I do that?, world composer has a zoom setting, and as I adjust, output resolution and grids change accordingly. Anything like that?

p-

 

Thank you so much for the fixes, will include them in the next release Both issues were already tracked but was drowned in the pipeline.

In this case the only reason I can think of is that you have enabled the "Elevation Only" checkbox in the "RntimeOffline" script which means the generated terrains will not have satellite images/textures on them and only heightmaps are applied on terrain tiles. Let me know if it's something else.

 

Please note that the Zoom Level in Interactive Map window of TL is for visual references only which is used for area definition and does not change the final resolution of the downloading satellite image(s).

So choosing different mapping sources in Interactive Map section is only for previewing the area and does not affect terrains' elevation/imagery data source as TerraLand obtains its data from ESRI only.

More on this: https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-7#post-3164935

Yes, it was there from the beginning but with another terminology than the zoom level. In TL you can have arbitrary tile resolution for both Heightmap & Imagery data through their dedicated sections as following:

• Heightmap Downloader:

• Satellite Image Downloader:

The option to give each tile its user-defined resolution is available because ESRI servers output results through its rendering algorithms instead of mosaic based manipulation of tiles like Bing/Google and most mapping providers support and are static. TL is the only asset with this feature and can handle dynamic resolutions for the tiles:

https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-4#post-2813587

Also in TerraLand there's a section called "Engine Resolution Presets" which you can use pre-defined presets for the UI resolution settings and the Manual mode to go on and set desired resolutions for yourself:

Also different mapping sources provide different resolutions of data throughout the Earth surface and it is possible that any of them has better resolution than the others but with the toolset available in TerraLand you can also use custom elevation/imagery/splat data from other sources and get them used through processors in TL:

https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-4#post-2826055

P.S. well done for your project, pretty nice workflow you have there.

 

Thanks for the suggestion. The "Elevation Only" checkbox is unchecked, so it must be something else. The textures don't appear to be added to the terrains (see video).

Spoiler: Video

Best regards,
Tony

 

@sharkapps As I didn't test the streaming scene in 18.3 yet, I guess that this is due to the new API in 18.3 and the new physical terrain layers in project instead of internal alphamaps in terrain shader.

I recommend using TL in Unity versions prior to 18.3 to avoid these issues until it is stable and we update the source to be compatible. Thanks for understanding.

 

Understood. I happen to be experimenting with 2018.3 at the moment, so I am trying to cobble together whatever works. I can work around for now -- my attempt to convert all the SplatPrototypes to TerrainLayers resulted in the textures appearing, but the same textures are repeated across all the terrains.

Spoiler: Close but no cigar

So I did something wrong, but it is not important for now and I am perfectly happy to wait for the update.

Best regards,
Tony

 

@sharkapps I didn't see the new API myself yet but I guess you then have to give it proper size and offset as it used to have in splatprototypes. The bad thing is that we have to go through hundreds of code lines and change them to be compatible with the new API...

At least in 18.3 the Static World Mode is running without any issues for now.

 

@TerraUnity I didn't find much of an API myself when I looked around. I just tried modifying all of the deprecating SplatProtofype code to use TerrainLayer instead and then adding to terrainData.terrainLayers. Since I am not familiar with TerraLand code, though, I am sure I did some part of the conversion wrong.

I am curious if you are able to access building and street data through TerraUnity to allow the possibility to generate meshes from that data. Is the data available?

 

@sharkapps Yes, the most involving changes are coming from TerrainLayers, just checked TL with 18.3 b9 and I confirm that texturing is broken even in the Static World mode, we have to wait for the official release and then start porting codes.

We have done this since 2011 but never released it publicly but it is planned for our upcoming product TerraWorld, little info is available here: https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-9#post-3497378

We will release TerraCity as a component in the new product but won't be there in the first version. It is planned to have the roads and streets stuff first as a release and then the buildings. More info will be available after the initial release of TerraWorld.

 

Been sending emails to info at terraunity dot com with my UAS Invoice # so I can download 3.1. Have not gotten any answer. Can you check that out? Thank you!

 

@crashTX We did not receive any emails from you as we process and answer all emails! The last email received using search criteria for your username was from 15th July if that's you!

TerraLand 3.1 is already in your account to download in both AssetStore and our website's panels, so either of these give you access to the latest package. If you still need help in other subjects and can not send us email, send me a PM.

 

Hi, I'm trying to use this with Unity 2018.3, and am getting a lot of errors:

Read only asset Packages/com.unity.render-pipelines.high-definition/Runtime/RenderPipelineResources/Material/NewLayer 1.terrainlayer has no meta file.

I'm using the latest version from the asset store. Any ideas? Thanks

 

@Darthlatte

https://forum.unity.com/threads/rel...world-environments.532304/page-4#post-3860908

https://forum.unity.com/threads/rel...world-environments.532304/page-4#post-3876001

https://forum.unity.com/threads/rel...world-environments.532304/page-4#post-3888718

But as always you can create terrains in scene with TL, optionally remove all TL related stuff from project if done and finally open/upgrade your project in any Unity version you wish to.

 

TerraLand 3 is on 50% sale, will end soon

 

Who's coming to Unite India? We'll be there...

 

In a previous post there was talk about how to place objects on a scene using their real world lat and long. I noticed the formulas given do not take into account the curvature of the earth so work on the distance between two points as being a straight line (ie flat earth). Do you know what the size of error this produces on say a 3km map (assuming 2 points topleft and bottom right) when compared to a non-flat earth projection?

 

Terra team - have you see any members of the community attempting to use this alongside the SpatialOS system (their Unity GDK) as the solution for enabling cached world streaming?

SpatialOS is a highly scalable server/networking option for unity, but it doesn't handle terrain out of the box. I'm unsure how your plugin could be utilized to save out terrain pieces, and then brought into the SpatialOS world... perhaps if any other community members have utilized it in this way, some light could be shed, or even some code?

 

How do I download the google images for the terrains and not the bing images? In the interactive map going back and forth thru google and bing shows different images but terra land is downloading the bing images only. So where in the options can I download the google images only?

Also can terraland download 16bit raw files instead of 8bit raw files.

 

Why I am I getting this type of terrain whats going on here?

 

Problem with asc files.
I cannot generate asc files from the dowloeader it always gives me this message.
Raw and tiffs are fine.
Also when I use the highest setting for any type of files(asc, raw and tiff) the dowloader get stucked...
Please Help!

 

Hi
I would like to substitute a small part of the dynamic terrain with a local data high resolution terrain or even better a static mesh(4km), so I will use only the dynamic terrain in the distance. The player will not be able to go outside the local 4 km area. Is this possible? how do I hide the part of the dynamic terrain that will be substituted by the local one?

 

I am using Terraland 3 on Unity 2018.2.13 on Mac OS 10.13.6. The terrain generates but no image is applied. The control panel says 1 available terrain, 1 satellite image. The terrain is in southern California. How can I get an image to display on the now white terrain?

 

Not sure if I'm missing something or what, but neither reading through this thread or examining the code gives me a good idea of how to make custom world loading work.

I have two specific things I'm trying to do, and it seems like TerraLand 3 provides good solutions to each of them individually, but can't do both together:

1. Create a custom world (not Earth) for players to play in
2. Load world data from a Web server rather than having it packaged with the game

Everything seems to be built on the implicit assumption that you can either load Earth data from a GIS server or custom data from the disc, but what I'm working on is a multiplayer game involving exploration, so having all the world data on the disc would enable cheating.

There are two things I need: a specification of the protocol TerraLand 3 uses to download world data from a web server, so I can build one that will work, and a way to point the online system at the server. Where can I find them?

 

Never tried to measure the error between points and I'm not sure if there is any errors at all, because it all comes down to the projection and maybe the projection in Unity is not spherical and it's flat!

However, measuring the error (if available) is easy, just create your terrain and put geo-referenced object in the scene using the formula in the previously available script - same as in this video - and see if it offsets or not? But don't think for a 3km map the error is large at all.

The best bet is to select a region close to poles and see if there is spherical projection errors in the formula or not.

 

Just heard about SpatialOS recently but did not have a chance to test its gdk yet. The current streaming system in TL is kind of a cached GIS server rather than a multiplayer style streamer but we are planning to develop a new solution to stream all assets on demand just like assetbundles from external sources even without pre-processing in engine.

 

https://forum.unity.com/threads/rel...world-environments.532304/page-4#post-3805531

Heightmaps in TL was always 16bit from the beginning.

 

https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-5#post-2878638

 

That's related to your machine's mime type settings related to a ertain software and its read/right permissions. So if other formats are ok, asc files should work too.

Highest setting on Engine Resolution Presets you mean? The higher presets have nothing to do with save elevation data format and by "stuck" do you mean it freezes forever or for some time? Please note that setting higher resolutions for data requires more processing and so it may be possible that the plugin freezes for some time until it finishes processing requests.

 

That setup is a custom one which is not supported built-in. But you can put the high res mesh in the scene in the editor and have empty tiles at center where the mesh is and leave the other surrounding background tiles as they are. So you will have a mixture of both your own mesh terrain and streaming ones at background.

The opposite of this behavior is implemented in this video:

 

We're running into a strange issue with offline streaming where there is a row of "blank" white textures before it adds new tiles to the streaming world.



If I move backward, the newly generated tiles disappear and when I move forward there is another blank row of textures before a new terrain row in generated.

 

If this issue is a permanent one, please read the following to resolve it:

That's a known issue on some Macs which TL Downloader can not finish its satellite image downloading operation due to .Net incompatibility with Mac IO functions in latest Unity versions.

To resolve the issue, please go to TerraLand = > TerraLand Core => Editor folder in project and open up the "TerrainGenerator.cs" script in your IDE, find line 1176 and uncomment the line with this code:

FinalizeTerrainImagery();

This should solve the problem and satellite images will be generated in your project upon finished downloading images from server. Please report back it this solves your issue and we will update the code in the next version.

Side Note: Whenever there are issues in downloading satellite images in TL (e.g. stuck at downloading all images and/or progress interrupted), you can press the button "FORCE FINISH" that will appear after pressing the GENERATE buttons so that it will force finish the downloading of images and will apply any successful downloaded images and generates them.

 

The best bet to get started about streaming in TL is to experiment with the provided sample scenes in the package, there are 3 separate sample scenes for offline streaming in TerraLand.

Also make sure to watch closely the provided video tutorials on our YouTube channel.

For generating a cached streaming server from own custom data files in TerraLand, watch this tutorial here

which generates streaming terrains out of World Machine's heightmap and splatmap renders in runtime.


Some general information regarding the streaming system in TerraLand:

The current streaming system in TL is pretty naive and firstly it was meant to act as a cache GIS server such as all mapping servers out there. It does a good job at this right now and you can put gigabytes or more heightmap/imagery data in those servers and TL can load them and stream them at runtime and generate terrain tiles from scratch dynamically even though they were never imported into Unity at all which is the system's greatest advantage.

The system access server path through a simple .Net IO function which can take either a local or an online path. The structure of a working server is explained here: https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-8#post-3388627

But for open world games and multiplayer genre everything gets 100 times more complex and all assets need to be packed on a server to finally be streamed in runtime in a more managed way. That's why we are thinking of a multi-purpose solution with different techniques to stream whatever assets on demand just as assetbundles in Unity but without needing to import or pre-build those assets if possible. This won't happen soon though.

 

Does this happen on each row/column updating or it is just for a specific row in the database? Maybe a short video will help describing it?!

What is the value of Active Grid Size in the RuntimeOffline script?

 

Hello Amir!
Back from discord, i'm trying to sort my issues in the most streamlined way so i'll stick to this thread and see what happens.
Regarding Terraland 3 local streaming capabilities, i need to create this huge 260x260 square km terrain, and i need to place roads and buildings at specific places (where these are actually located in the real world).

From the Terraland 2 thread (as you linked it to me):

Regarding how to increase terrain resolution on local streaming (pay attention to the bold section, extremely helpful):

I'll get back here and reply to this post when i have all my issues together