TerraLand 3 - Streaming Huge Real-World & Custom Terrains For Open-World Environments

@Liviuss We are currently working on it to improve stitching, in the meantime there is one useful operation you can do on downloaded tiles.

Bring up "TerraLand Terrain" component (Tools => TerraUnity => TerraLand => Terrain), go to processor "STITCH DATA TILES", press button "STITCH TILES" and select the Elevation folder in your server containing heightmap tiles and press "Select Folder".

This processor will load up each heightmap tile in the server, check all its neighbors and matches up all borders corresponding to neighbor tiles.

After doing the above operation, you can see the tile borders are now matched up without runtime border matching operation but still needs proper stitching function to remove seams. We'll remove the bugs in the auto stitcher in the RuntimeOffline component and let you know about it.

 

Hi,

Thanks for the suggestion. I have tried to do the stitching as you described but it doesn't help. Terrains still have gaps between them.

 

Hi,

There is an issue related to the offline streaming and fast moving camera.
If the speed of the camera is low then system is working fine and terrain chunks get loaded without issues
Example:

If camera is moving fast over the terrain, system does not have time to process chunks and it get screwed due to the way you process/queue chunks in coroutines.
Example:

My question is, do you guys can give a raw estimation on fixing this or it will not be fixed in the near future.
Thanks

 

@Liviuss Current streaming system is sensitive to player speed and breaks when it can't handle new row of tiles, that's mostly due to Unity's terrain API lacking threading support and everything should happen on the main thread as it also causes spikes when setting heights for tiles.

But hopefully we are considering making a new improved version of the streaming with the new terrain API and more robust solution to handle streaming tiles around player. Keep connected with us.

 

TerraLand is on sale for 50% and it's only $14.99 https://assetstore.unity.com/packages/tools/terrain/terraland-3-119097

 

Hello, I have purchased and used TerraLand 3 for a long time, but recently I downloaded the terrain, the satellite image and the terrain height are misaligned, the gap is about 1km, what's the matter? Also, if I use vpn, the map download fails, but I can’t download google maps without vpn in China. How to solve this problem?

 

@zetingq Is the issues related to streaming scenes or the static world mode in the editor? If that's a streaming issue with misaligned tiles, it can be related to this: THIS ANSWER

FYI, we also use VPN to connect to ESRI (Not Google Maps) servers as the source of data acquire in TerraLand with no issues at all. Please check back with your connection to see if it's healthy for ESRI's ArcGIS Online Services.

 

Good day!
I'm having problems with offline terraland ...
My sequence of actions:
1. I downloaded an example with an offline project. I started it, it worked.
2. Downloaded an offline terrain map 25 by 25 kilometers with low quality heights and photos (All .raw files in elevation and all jpeg files in imagery were downloaded). Replaced only the path to the folder. I started it, it worked.
3. Downloaded an offline map of 1,000 by 1,000 kilometers with high quality heights and photos (All .raw files were downloaded in elevation and SOME jpeg files in imagery. 1,225 jpeg out of 16,384 were not downloaded). Replaced only the folder path and unchecked the absolute path. I started it, and errors started ... The camera is very slow. I tried to run it with the "Only Elevation" checkbox activated. Did not help.

Please help, what could be the problem?
I am attaching screenshots.
Thank!
Sincerely, Belkin Vyacheslav

 

@BelkinVyacheslav Good day, First of all, redownload failed images from the section "FAILED TILES DOWNLOADER" in TerraLand Downloader or the tiles will not be in sync while in Play mode.

Regarding the bad performance and slow camera, please bear in mind that the tile resolutions you give before server generation is so dependent on the target platform and usually for desktops, optimal resolution for each elevation tile should be under 256 and 512 for each satellite image, instead give Tiles Grid a greater value to spread data in more tiles. Also 1000 x 1000 km is so huge and you may end up with a lot of tiles and inaccuracy on them, but it is doable if you give proper tiles count and individual resolutions. Get back to me with the results.

 

Good day!
About missing files.
Really, I used "FAILED TILES DOWNLOADER". Within 3 days I pressed this function (why was it impossible to loop the process of checking and downloading missing files ???). But very few are loaded. For example, in one pass, out of 1225 missing photos, 5 photos or 0 are loaded ... The source of the maps is BING. Tried it with Google, about the same.
Do I understand correctly that if I have files with part of the name "_Temp" in my jpeg folder, then it will not work normally? Is the "IMAGERY" folder ignored with the "Elevation only" flag? (The ELEVATION folder contains all 16,384 files in the RAW format. Everything in this folder was loaded and converted.)

Thank!
Sincerely, Belkin Vyacheslav

 

First of all, changing mapping sources in Interactive Map section does not change the download servers and heightmap/imagery source in TL is always ESRI only. Changing providers from there is just for previewing purposes.

You are right about auto recovery of the failed images but these failed queries increased recently as servers get more sensitive to network jitters and users could retrieve failed ones which were less than dozens in a normal scenario with clicking the Get Failed Images button. Anyways, if you have a good network connection, it doesn't have to happen like this or maybe the number of tiles and their resolutions are high so that there are no available imagery in those locations!

"_Temp" at the end of filenames means TL Downloader was unable to retrieve those tiles and first you need to re-download them to be put in there. As mentioned before, you need to have all tiles downloaded in server folder before jumping into the game.

Yes, "Elevation Only" checkbox ignores "Imagery" folder and only creates heightmaps at runtime.

 

Thanks, I understand.
1. Internet connection is good, I think you are right about the fact that there are simply no available images.
The location I'm uploading is one of the North African countries. Part of the unloaded area is the sea.
2. When using the "Elevation only" flag, images are not loaded, but errors (in the screenshot of message # 458) are still present.
3. Some of your suggestions for action from message # 458 I have not yet done, but I will.
4. Today I will put on download an area 1000 by 1000 in the region of Europe / USA with a low resolution of heights and images. I'll check if everything boots up and how it will work.

Then I will write about the results.
Thank!
Sincerely, Belkin Vyacheslav

 

1. I tried different combinations with parameters: Active Tiles Grid (increased to the maximum), Previev Heighmap / Satellite Resolution (reduced to the minimum). The camera is faster, but the error "NullReferenceException: Object reference not set to an instance of an object." generated constantly (in the screenshot from message No. 458).
2. Generated offline servers:

• Area 1000 by 1000 in the North African region with sea at maximum resolution of heights and images. (All heights were loaded. Images except 1,225). Does not work. Error from the first point.
• This is the same 1000 by 1000 area in the North Africa region with the sea at minimum resolution of heights and images. (Heights are all loaded. Images are all loaded). Does not work. Error from the first point.
• Another 1000 by 1000 area from the Eastern USA region without sea at minimum resolution of heights and images. (Heights are all loaded. Images are all loaded). Does not work. Error from the first point.

3.After I again specified "TerraLandStreaming_Zion" as the server. WORKING.

I don't know where to go next ...
Thanks!
Sincerely, Belkin Vyacheslav

 

@BelkinVyacheslav Thanks for the reports, please share your preset file which you have issues with here so that we can check out on our side.

Also please copy paste the whole error logs you receive in runtime here instead of sharing screenshots.

 

Good afternoon!
The project folder is available at the following link:
https://drive.google.com/drive/folders/1jwVV6O6Q65lN07aYRkbIKUr8nZM1T1NU?usp=sharing
Log file:
https://drive.google.com/file/d/1dJ2QIfQOJ30tn36s0hNPLgjrkjzQxPdW/view?usp=sharing

TerraLandStreaming_Zion - WORK.
My loaded server is 1000 by 1000 area with minimum heights and images resolutions - DOES NOT WORK.
Thanks!
Sincerely, Belkin Vyacheslav

 

@BelkinVyacheslav Thanks for sending but you only need to send us the preset file in .tlps format from the PRESET MANAGEMENT tab at the top right corner of TerraLand Downloader UI. This way we can have the same settings you have there before server generation and we can check if the settings are proper and tiles can be downloaded or not.

 

Sorry, I didn't understand right away ... English is not my native language.
Done, here is the configuration file before generating the server.

Thanks!
Sincerely, Belkin Vyacheslav

 

No problem, thanks for sending the preset file

Ok just checked out your preset. First of all as a general rule of thumb, 1000 km x 1000 km is a very huge area in practice and result may be inaccurate and in low resolution. However, it is finally doable but since your selected area covers parts of ocean where there are no imagery data available, you will receive warnings in the console from server and those tiles could not be downloaded.

These warnings when server can not resolve tiles are:

Code (CSharp):

1. System.Net.WebException: Error getting response stream (ReadDone2): ReceiveFailure ---> System.Exception:   at System.Net.WebConnection.HandleError

and

Code (CSharp):

1. System.Net.WebException: WebResponseKnownError
2.   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse

So to fit things up, first I've loaded your preset, then opened up server folder in my computer while downloading. In the Imagery folder I have monitored downloading tiles and as soon as I found out there are null (Not Downloaded) images, stopped server generation and gave each tile's sat image resolution to a lower value.

I've also reduced area size to 600 km x 600 km from the same center point ending at ocean borders so that there won't be any null tiles since your original select area had a lot of tiles in ocean areas where there are no data available.

Also changed each tile's heightmap resolution to be 256 for better runtime performance and not the waste of resampling steps, so with 128x128 (16284) tiles each with 256 pixels in resolution and the area of 600km2, you will end up with 20 meters per pixel for terrain surface which is a good value as an average resolution.

And please note that since the selected area is in Africa, so the resolutions may be very low.

Also reduced Smoothing Steps to 1 as the best option when the original heightmap outputs a reasonable resolution.

Lastly I ended up with the settings as attached a preset file for you to check out. FYI, the final area is 600km x 600km and it took around 40 minutes obtaining about 6GB of tiles data and 1 retry via "FAILED TILES DOWNLOADER" section to retrieve all tiles in server.

Load up the preset file in TerraLand Downloader and generate server, load "Runtime Assets Placement - Asheville" scene, change Server Path to created server in RuntimeOffline script, change settings on this script if needed and hit Play to see the world generating around you in game mode.

Note: When loading this scene as the template for the world, "Streaming Assets Manager" object in scene places items on your streaming terrain in runtime and also if you look at the "Info" folder in your server, you will see that it also downloads 2 global heightmap and satellite image tiles for the entire area at level start to cover far viewing distances.

 

Thank you very much!
Unloaded an area of 600 km x 600 km according to your preset file. The resolution is good, everything was unloaded completely, without errors! After loading, I opened the Asheville scene, changed the path to the server, and started. Tiles are loaded, the only thing: do not stick together (in the apendix logs and a screenshot). I will deal with the settings.

But I have one more question ...
What are the methods of loading the area in the square of which the ocean is present?
I don't need the ocean itself! Let there be at least black flat tiles ...

 

If you know the tiles with water areas in the server and their filenames are ended with "_Temp" so you can rename the file and remove the "_Temp" suffix from it and/or replace it with a black or whatever image with the exact same name so it's like it is valid and being already downloaded.

 

It's been more than a year since the original question and maybe it's too late to give you the solution but thought it would be useful for you and others to resolve the old issue when building Online Streaming (WorldExplorer) scenes.

So the reason for the issue of not downloading heightmap/imagery tiles when building was due to incompatible versions of DLLs in Plugins folder of the TerraLand project while working with .Net 4. The stupid part is that Unity does not fall into any issues in the Editor but the problems arise when building without even notifying any errors! That's why it was hard to find what was wrong.

Anyways, to overcome this issue, do the following steps:

• Find "Plugins" folder in TerraLand project under Assets/TerraLand/TerraLand Core/Plugins
• Remove these DLLs from there: System.Configuration.dll, System.Drawing.dll, System.Web.Services.dll & System.Windows.Forms.dll
• Find "csc.rsp" & "mcs.rsp" files under Assets and replace the first line of them with the following line and restart Unity

Code (CSharp):

1. -unsafe -r:System.Net.Http.dll -r:System.Web.dll -r:System.Web.Services.dll -r:System.Drawing.dll -r:System.Configuration.dll

Updated "csc.rsp" & "mcs.rsp" files are also attached for future reference.

Now all tiles will be retrieved from servers in standalone builds.

 

Don't miss the 50% sale on TerraLand until 4th December: https://assetstore.unity.com/packages/tools/terrain/terraland-3-119097

 

Compiling to Android fails on these errors...

Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(128,70): error CS0433: The type 'Rectangle' exists in both 'System.Drawing, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'

Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(128,33): error CS0115: 'EuclideanColorFilteringOperation.ProcessFilter(UnmanagedImage, Rectangle)': no suitable method found to override

Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(21,14): error CS0534: 'EuclideanColorFilteringOperation' does not implement inherited abstract member 'BaseInPlacePartialFilter.ProcessFilter(UnmanagedImage, Rectangle)'

Please advise

Many thanks

 

@rodtuckwell That means you have to go to Project Settings and change the API Compatibility Level to .NET 4.x since you probably changed build platform and did not set it as you did when installed TL in your project.

 

@TerraUnity many thanks!

 

Does this come with a parallax shader for the terrain? I also bought TerraWorld. I get errors when I try to run side by side but I am using TerraWorld alone and it works fine. I'm really interested in a high quality parallax/tessellation I've seen but there doesn't seem to be such a shader in the files.

 

Ok so I actually tried baking a map using TerraWorld and followed these instructions from here https://terraunity.com/how-to-generate-your-first-world-in-terraworld/

I got this, shown below attached. I used the auto-installer as well. These bugs happen when I select my location and then click a template.

 

Hi there

I purchased TerraLand and imported the package into unity 2019.4
when the importing is finished i restarted unity and tried to generate terrain as it shown in the tutorial.
TerraLand Static :generate terrain without elevation (Only imagery).
Dynamic Terrain : the same issue but in the elevation folder there some elevation tiles "tif" format named like 1-1_temp.tif.
i'm getting some errors : in the jointe file.

let me know if there is a solution for this issues

 

Sorry for the late reply. The custom terrain shader is included in TerraWorld package and is called TerraFormer. Please see its features here:

You can find TerraFormer shader and needed cginc files in project under TerraWorld\Graphics\Pro\Shaders\TerraFormer

Regarding TerraLand and TerraWorld packages side by side, it is not supported and not advertised any where at all, because they are 2 separate products with different approaches for world generartion with exclusive features but most of TL's features will most likely find their ways in our TerraWorld package in future.

 

None of these should happen in a healthy fresh project in Unity's Standard Rendering Pipeline. From the errors, it seems that you have imported TerraWorld in HDRP template which is not supported yet. Please read this at the first line of the product description where it says:

Please use this thread for TerraWorld related questions: https://forum.unity.com/threads/33-...vel-designer-for-real-world-locations.784217/

 

The issue with TerraLand and flat terrains where heightmap data were not downloading has been resolved now. Please PM me with Invoice No. to get the updated scripts with fixes.

The reason behind this was from ESRI's endpoint as they have completely deprecated all HTTP queries to their servers: https://www.esri.com/arcgis-blog/pr.../2019-arcgis-transport-security-improvements/

TL 3.5 will be uploaded soon with these fixes and announced new features and improvements on Discord here: https://discord.gg/uZKmAy9

 

Hello,

First thank you very much for Terraland, it is a great asset!

Lately I have received the "failed tiles downloads" everytimes I've tried downloading Offline maps. When I try the "get failed tiles" button, after a while the same error occur, again and again if I reclick on the button.
I haven't been able to download any map, tried with different locations and different parameters of quality.

Again thank you very much for Terraland, and for this very useful thread.

 

Hi, glad you liked it so far.

Recently ESRI servers changed their rules to query data from them and not getting data tiles is a known issue which has been already resolved and announced on our Discord here: https://discord.gg/uZKmAy9

Please PM me here or on Discord or email us to receive the updated scripts with the fix.

See this post for more info on this: https://forum.unity.com/threads/40-...orld-environments.532304/page-10#post-6637453

 

I've sent you a PM on this forum. Thanks a lot for your help.

 

@AntoineR45 PM Sent

 

Ver. 3.5 is now live on AssetStore with 50% off: https://assetstore.unity.com/packages/tools/terrain/terraland-3-119097

Changelog:

• Support for Bathymetry data and creating under-water heights for terrains
  
• Official SRP (HDRP, URP) support
  
• Pooling system for StreamingAssetsManager with much better performance
  
• Fixed flat terrains due to ESRI's updated https queries
  
• Fixed Misaligned streaming tiles in Offline Streaming system
  
• Fixed WorldExplorer demo not working in builds
  
• Auto retrieve of failed downloaded image tiles in Downloader's Static World mode
  
• Updated "Runtime Assets Placement" scenes for better runtime placement
  
• Few internal bug fixes

Get it before the new year's 50% sale ends ;-)

 

50% sale for TerraLand ends tomorrow: https://assetstore.unity.com/packages/tools/terrain/terraland-3-119097

 

I don't have the important section folder. I only have two files mcs.rsp and csc.rsp.

I'm experiencing web unknown response error while downloading elevation maps
And sharing on violation path error.

I'm using unity 2018.3

 

@mukki014 What do you mean by this:

If it's a fresh install of TerraLand in your project, then you won't experience any issues retrieving data from servers unless there is a specific issue on your side for the network connection. Please check back TL in a healthy project and report if you encounter any error logs in the console.

 

You don't need those config files anymore since TL ver. 3 as it will be handled automatically to setup connections.

 

Oh thank you. I didn't know that. But why I'm not able to get that height map/ elevation? I can get satellite image and do other stuff. I will try again.

 

Please give it a test and get back to us if you still encounter issues in receiving data.

 

Hi,
I'm having trouble producing proper elevation data to supply data for the server.

I've made sure that the server structure is OK, i've fixed the areaSize in the script to match the size of the terrain - it does load the data but it does not look right. apparently i don't know how to define the RAW export from Global Mapper. I tried exporting the UNITY RAW RASTER and got this result when loading it to TL:



I tried importing the RAW format and i got a flat terrain.

I'm using the SRTM WORLDWIDE ELEVATION DATA in Global Mapper .

If you can direct me to a source / explain how to export the data correctly so that it will be usable in TL it will greatly help . thank you

 

@mardulis Please send us a sample heightmap in original format as asked on Discord so that we can check out. And one question I have is that is there a specific reason which you need SRTM for terrains? TerraLand has built-in global heightmap and satellite imagery coverage as the highest resolution geospatial data source!

 

I need a source for the elevation which is not from UNITY ( i can not use the built in TL- downloader which works in the editor for my project ) so that i put it in the server structure folder and TL will create the terrain in runtime.

You have asked on DISCORD to send the original heightmap file so that you can export it using Global Mapper - Isn't Global Mapper the SW which exports the heightmap? what am i missing? I was regarding the RAW files from global mapper as the original heightmaps...

I've attached 2 zipped RAW files. Thanks.

 

@mardulis By original heightmap file I meant the data file before importing into Global Mapper and not the exported .raw file. Anyways I just checked the exported raw files and for the tile "EL_9_9" here are the results as follows:

We can check the raw files in Photoshop before importing in Unity and use TerraLand.

Byte Order: PC



Result:




Which the above is the improper heightmap output used on terrain as the picture in your previous post with jagged edges and messed up heights


Byte Order: Mac




Result:




Which above is the proper heightmap which needs to be used in TerraLand.

So to create a proper export out of global mapper, in BIL Export Options window, enable checkbox "Use Mororola Byte Order (Big Endian) for Elevations" as shown in the following image:




So TerraLand will properly generate heights out of exported raw file with the default byte order in Big Endian format.

 

Now things are much clearer! But still...

I'm sorry but now I have a problem to recreate the RAW file I've sent you - I'm getting a black file in photoshop with MAC endianness and some color variation with IBM (Which is not the right option as you've explained already)

In global mapper I'm trying to export an elevation grid in RAW or BIL format with the settings you've provided previously in the general tab . In the tiling tab I set the size in pixels for each cell (I.e 512 *512 pixels), the dataset is the SRTM map or ARTEM.
Thanks

 

To be more precise - i manage to get photoshop readable RAW files only with the UNITY RAW TERRAIN/TEXTURE export option. But this one still produces jagged terrain in TL - its visible that it has an extremely small dynamic range in the histogram. what am i missing? thanks.

 

After exporting a BIL file, i try to open it in photoshop and that's the error message i receive: