Search Unity

TerraLand 3 - Streaming Huge Real-World & Custom Terrains For Open-World Environments

Discussion in 'Assets and Asset Store' started by TerraUnity, May 20, 2018.

  1. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @Print3d Feel free to ask questions and share your progress on anywhere, yeah I agree that Discord style apps get flooded time to time and people lose track of subjects.

    Currently it is absolutely possible to have a huge area (260km2 as yours) split into tiles in a cached server and stream them in runtime. At level start all active tiles around player are accessible to populate assets on them and also after each row/column update, list of updating tiles are there to remove old assets on them and update it with corresponding assets for placement.

    I recommend not to use satellite images at all in the server. WHY? Because having satellite images in server means you have to download gigabytes of imagery data to be put in server and in runtime TL's streaming system needs to load each tile's image pixels from scratch and apply it to its terrain data for rendering. And in the end, no matter how much high-res the images are, they will never be suitable for close-ups and they soon become blurry/pixelated. Unless you are looking for simple GIS mapping applications which gets boring soon and graphical aspects are not that important in the end.

    As in "RuntimeOffline" script (which handles streaming in runtime), you have the option to define a custom terrain material for streaming terrains, you can use a shader which generates colors and does runtime splatting such as the one we showed in this video:



    In this video, no Satellite Images are used but only heightmaps and the colors on terrain surface is procedurally generated using a custom terrain shader which does blending between colors based on surface angle as a simple setup.

    For a more advanced texturing and splatting distribution on surface, you can go the other way which is supported in TL's streaming system which is splatting streaming terrains. Using this feature, you can only have heightmaps in place and texturing is done using pre-processed splatmaps (e.g. produced in World Machine) instead of satellite images, so detail textures can be tiled on surface just as we have painted them by a brush. Watch this video tutorial which explains how to do the same for your own:



    The splatting streaming terrains shown above is similar to having sat images in server but this time splatmaps instead which requires pre-processing and generating these maps and has the same runtime performance overhead as having satellite images.

    So the best option here is to only have heightmaps to generate heights (considering average of 10 meters per pixel for the original resolution) and let the texturing on the GPU and shaders to procedurally generate colors/splats based on slope, height, aspect, normals... This way, you save a lot of room for other performant elements in the scene while having better rendering on surface.

    I've seen a lot of these shading implementation before, @ForceVFX has done some here & here I'm not sure if there is procedural implementations involved or not but maybe you could ask him to help in this.

    Also for generating and placing geo-referenced elements such as buildings and roads, we have been doing it for years now but never released our solutions as a public package, you can see our recent WIP implementations here: https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-9#post-3497378

    The good news is we have plans to have this system included in our upcoming product after its initial release to generate geo-referenced cities and urban areas with all elements from GIS servers plus our own procedural algorithms for assets creation and placement in the scene. But that's not going to happen soon though.

    First, let's stick to accessing tiles in the runtime and try to populate assets on them, right? ;)
     
    Print3d likes this.
  2. Print3d

    Print3d

    Joined:
    Oct 22, 2017
    Posts:
    35
    Ok i got the first part and that's straightforward... What i'm missing is how i can manually place even a simple cube created with ProBuilder on the map at a specific place and get it to be generated on runtime with the terrain itself.
    I mean, the terrain is only showing in play mode... I know it gets generated, but how can i access it to add assets and the likes? I need to see the satellite imagery because i want it to be at that exact spot... Then, once i placed it i could delete the satellite data, no problem with that!
    To make it even clearer, i'd like to be able to edit the terrain like in the static world mode for the dynamic one: is that even possible? Thank you :)
     
  3. Deleted User

    Deleted User

    Guest






    My next demo..220 Kms..
    p-
     
  4. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    It is absolutely possible and I will prepare a scene demo for this as the demand is high.

    What do you exactly want to do with satellite images?
     
    Print3d likes this.
  5. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    What filters do you use for splatmap generation? (slope, height, aspect, normals!). We are planning to improve the LandCover/Splatmap generator in MapsMaker of TerraLand and your input will definitely help in this. This way you don't need external programs such as WM to process your heightmaps. Plus, we already have Erosion & Terrace filters in place which output pretty nice results, not on a par with WM though.

    These implementations are under heavy development for our upcoming product:

    UncannyValley_1.jpg
    UncannyValley_2.jpg
    UncannyValley_3.jpg
    UncannyValley_7.jpg
    UncannyValley_16.jpg
     
    Bartolomeus755 likes this.
  6. Deleted User

    Deleted User

    Guest

    show me the terrain!....naked, bare bones..just textures..it the only way to see anything. I see a little bit of splat textures, but a whole lot of rocks, tress, plants..


    new color map fromTL3 down loader, created from original colors and 5 splat channel map, that was created.

    This is the lowest quality, for my sat image tool, I want to achieve, now I can, thanks toTL3!!

    (I keep asking myself, why do I spend so much time on this terrain texture nonsense, when your going to cover it up with grass.trees & rocks, anyway...)...hmmm..art?

    I am still working, everyday, all day, on this terrain tool, I have a simple goal, take a real world location, and have full control on all aspects of the scene. I want to add, subtract stuff, add seasons, use it as a terrain template, it is hard to match the abstraction, of.....reality.

    Most valuable lesson: Start all terrains, with what shader you will be using..and work backwards.

    I promise, when my little 'World Machine OCD' episode is over, I will do what I can to make your next version STELLAR! (after the holidays)....Describing what filters I use, why and how.

    side note, I've upgraded my tool, it now generates, both 4 splat & 8 splat map channels, and all screen caps before..garbage, 'cause...my splat channels were overlapping, the erosion was wrong, etc.


    Quick Question: Can I get the entire image set, from the downloader..in one raw file? I am getting border seams, that are not there in World Composer, from the exact location, provider.

    p-
     
    Last edited by a moderator: Dec 27, 2018
    TerraUnity likes this.
  7. Print3d

    Print3d

    Joined:
    Oct 22, 2017
    Posts:
    35
    I just want to use those as a reference to model my environment exactly as it is in real life. Then i could overwrite those with different textures and so on. But for roads, buildings, paths and cliffs i need it first as a reference
     
    Last edited: Dec 27, 2018
  8. Deleted User

    Deleted User

    Guest

    THANK YOU!!...."overwrite those with different textures"....gave me a whole new idea, with color maps and splat textures..

    .
     
    Last edited by a moderator: Dec 26, 2018
    Print3d likes this.
  9. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @Print3d & @ForceVFX Thank you, just working on a new scene demo to cover all your needs and get you started.

    Did not completely understand what you mean here? But regarding data provider in TerraLand already answered your question here: https://forum.unity.com/threads/rel...world-environments.532304/page-4#post-3873139

    TL's data source for obtaining heightmap/imagery is ESRI only. Would you please elaborate more on the border seams you are mentioning, maybe pictures or videos!
     
    Last edited: Dec 26, 2018
    Print3d likes this.
  10. Print3d

    Print3d

    Joined:
    Oct 22, 2017
    Posts:
    35
    Not sure what i did there but no problem mate :) your projects are shaping up nicely!
     
  11. davedev

    davedev

    Joined:
    Jan 12, 2009
    Posts:
    74
    I am having a height map issue with a large terrain when rendering to a resolution of High. The same terrain creates heights when rendered as Medium resolution, but shows no heights when rendered as hight and also displays a weird, wall-like artifact when created at High resolution.

    high-med.jpg
    Do you know how I can fix this?

    I’ve attached the presets for the Downloader.

    What would be the best settings for a static, large, 1000 km x 1000 km, satellite and height mapped terrain viewed from higher 500m+ altitudes?
     

    Attached Files:

  12. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @davedev First of all a 1000x1000 km area is so large that the result with a static scene won't have enough details in most cases and the Dynamic World mode in TL is recommended where you have more controls on each tile heightmap/imagery resolution and also the total number of tiles in the runtime scene.

    Unless you are looking for a static scene and does not care about the final resolutions for a large area mostly suited for flight sims and high altitude fly overs.

    Regarding the heightmap artifact, I have checked out the preset files and the same happens here too; so after further investigation and testing on it, seems like the servers does not like the area size with that resolution and can not render the area into a heightmap. So that's the ESRI's server limitation. 600km instead of 1000km works though.
     
  13. Skribble

    Skribble

    Joined:
    Oct 29, 2012
    Posts:
    16
    Hello there,

    I bought TerraLand from Unity Asset Store and i've created an account in your website, also i sent my receipt invoice.

    I've been playing around and trying to follow tutorials and so, looking great so far. I see a lot of potential in this asset and totally worth it.

    One question i have is.. im using the "Offline Streaming" demo included in the project and im loading an area of 20kms x 20kms with 8x8 tiles, it works ok but when i load the scene, i dont see the assets being unloaded / loaded.

    how this is handled ? there's a view size limit for determine the distances for the assets to load / unload ?

    Also, i would like to add more content to the map ( houses, rocks, trees.. etc.. ) there's any tutorial where i can see how edit the map in the editor and save those assets for loading them / unloading them as happens with the terrain?


    To give you a better overview.. im working on a mmo game ( what a surprise :D ) and the map side is working great, but i need to add more content like characters, enemies and buildings..


    Best Regards
     
  14. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hello

    I have a couple of pre-purchase questions. Will TL3 work with 2018.3? Does TL3 work with HDRP?

    Thanks

    <edit> Their website is down? Are you still around @TerraUnity?
    <edit edit> Still active on Discord. New release imminent.
     
    Last edited: Jan 12, 2019
  15. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @Skribble Thanks for the kind words, I will upload a video which does what you asked regarding accessing streaming terrain tiles and populate own assets on them. This feature will be included in the next update release happening in a few days. Keep an eye here for the video showcase.
     
  16. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Yes, a new release is coming in a few days with official Unity 2018.3.x compatibility.

    Yes, TerraLand has no conflicts with HDRP as it does not include any custom graphics assets and shaders, so it is already HDRP/LWRP compatible. we use it extensively in an in-house HDRP project.

    Thanks fro the edits, the website is up now and the new update release is on the way as already mentioned.
     
  17. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hey @TerraUnity I have another question. Can I use TerraLand for a flight sim, viewing terrain from a great height.

    I'm particularly interested in views of terrain that show distances over 10,000m without Camera Far Plane Clipping.

    The picture below is from Google Earth, 13,000m up looking over the English Channel. Can I do this with TerraLand?



    There is another (apparently dead) product on the Asset Store which has a picture of an aeroplane, some mountains and some very distant terrain in the background. That is the image I am particularly hoping to make. All of my experiments with meshes show that the Camera Far Clipping Plane cuts in around 10,000m but like the image above the view from 13,000m high means you can see well over 13,000m in the distance.

    I hope this is clear. Please let me know if I can add more to the question.

    Thanks
     
  18. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @ghtx1138 You can absolutely create huge areas using TerraLand, have you seen this video?



    As you already know, having these large scenes is not possible with static terrains in the editor, so a runtime solution is needed, that's why we made TerraLand's Offline Streaming system which streams tiles in the runtime from a cached server on your local/online hard drive location.

    In the video above the whole area is 160x160 km and as we set it up to be 1 unit = 1 meter so the background terrain covers a 160,000 x 160,000 units area in the scene. The high-res tiles stream around the player on demand.

    The floating point errors are also handled by shifting the origin after a certain distance using a camera script explained here: https://forum.unity.com/threads/rel...-open-world-environments.532304/#post-3504450

    And last but not least, a new update is going to be released in a few days focused on streaming features, which will automatically create 2 layers of background mountains to cover hundreds of kilometers automatically at level start and populate your own assets on streaming terrains based on user specified rules. I'll upload a showcase video when the demo scenes are ready.
     
  19. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    @TerraUnity thanks for replying. I apologise for asking the question again - can I go high up in the air and see a large terrain? Like the google Earth picture I posted above.

    I understand the streaming tiles and floating origin concepts. But every demo I have seen is around the top of the mountain height or lower (not just TerraLand - everyone seems to have this "problem"). I am interested in 10km+ heights. My experiments appear to show that 10,000m is the limit of the Camera Far Clipping Plane. And I was wondering if Terraland has solved that problem?

    Thanks and my apologies again for asking the same question again.

    Cheers
     
  20. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @ghtx1138 no problem in asking more questions ;) I recommend watching the video again and read what is said.

    How come did you find that 10k units is the maximum far clipping plane of the camera? The background mountain in the video is 160,000 units and when level starts you are at the center of this region, the camera can see through the edge of this terrain so its far clipping plane is set to be more than 80,000 units.

    It doesn't matter if you are on the ground or 10k high above, you can see the terrains fading out into the global fog at far viewing distances, and obviously you can alter the density of the fog to increase/reduce see-able area.

    FYI, in HDRP there is a great feature called Camera Relative Rendering to translate all rendering aspects into the camera position instead of the world origin which gives much higher precision rendering in large worlds, but you still have to deal with physics origin shifting... however in Unity 2018.3 you can also shift physics origin in settings: https://docs.unity3d.com/Manual/class-PhysicsManager.html and in code: https://docs.unity3d.com/ScriptReference/Physics.RebuildBroadphaseRegions.html

    P.S. 1 unit = 1 meter: https://forum.unity.com/threads/wil...new-quality-in-rendering.512495/#post-3357172
     
  21. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    @TerraUnity Yes you are right - I was very wrong about a 10k unit Far Clipping Plane. Thanks

    Also thanks for letting me know about physics floating origin.

    I look forward to the update.

    Cheers
     
    TerraUnity likes this.
  22. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    <edit> Solved. Had not dragged terrain into selector at top of panel - thought that "selecting" meant selecting in hierarchy. Cheers

    @TerraUnity I have purchased the asset - have asked a question on Discord about converting materials to HDRP.

    I'm not sure Discord or the forum are adequate support tools (I don't know what else to suggest). Great ideas from earlier years are spread over the TL2 & TL3 Unity forum pages. Ideas disappear and get lost on Discord.

    It seems to me there are 3 classes of posts (maybe more) - pre-sale questions, support questions (probably mostly FAQ - eg how many times have you been asked about ESRI? :)), bugs that require preset files and other files to be transferred to you for deeper analysis. And of course marketing posts about future versions coming soon. In addition we have other TL users sharing their work and knowledge with other users. I've read all TL3 posts and am about to go back into TL2 posts - and I am writing stuff down and will probably re-read it all once I know what I am looking for.

    I like this product, it works very well and simply (after 24 hours) - but I get the feeling there is a lot going on under the hood - tools appear to be available in many different panels - and your doco is not that comprehensive sorry to say. Maybe for coders it is enough. But for me there are many concepts I must understand.

    Anyways enough of my ranting. I have a support question.

    I cannot get Terrain To Mesh to work. I have selected a terrain but am getting this error. Thanks for your suggestions

    PS Your new website server is terribly slow. See here for Google page insights

    Cheers

    TerrainToMesh.png
     
    Last edited: Jan 18, 2019
  23. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hi @TerraUnity I have a support question about Generate Filters. When I press the button I can see processing happening but no asset is created. When I click Generate Splatmap button I can see the _LandCover asset created. Is this the same thing? Is generate filters creating a "file" which is used for the Generate Splatmap function?

    Thanks

    GenerateFilters.png
     
  24. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hi @TerraUnity (me again :confused:)

    Can I stitch satellite images together? I've exported the 4 terrain tiles to a single mesh with the single World elevation tiff and would like to apply the 4 satellite images to the mesh. I guess the way to do this is to make a single image from the 4 satellite images? Please let me know.

    Thanks
     
  25. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Exactly, processing terrain(s) must be inserted into the field of Offline Data section.

    Yes, I agree that having all these sources for support questions is a mess and people keep repeating the same questions even though they are already answered in one of these sources and for users it is normal to get confused sometimes. Also agree that TerraLand lacks an updated text documentation but we tried to make video tutorials to cover most of user questions and how to use the plugins. The most 2 main video tutorials for TerraLand are:






    Also the TL 2 thread contains a lot of useful information and searching in that thread reveals some behind the scenes implementations.

    Thanks fro the report, yes we are aware of the issue, the current hosting provider sucks, we will change the hosting provider very soon.
     
  26. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Yes, the "Generate Filters" button is the initializer for generating landcovers/splatmaps, we had to have it as a separate button in order to select between automatic and manual setup modes and then generate filters using this button, it will setup needed parameters before generating the landcover file. This post is a comprehensive tutorial on how current MapsMaker work: https://forum.unity.com/threads/ter...eal-world-gis-data.377858/page-5#post-2956710
     
  27. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    The opposite is possible through Image Slicer processor where large images can be sliced into user defined grid tiles but stitching them is not built-in yet. However we already have the implementation done in other places; I will add it as one of the new processors in TerraLand Terrain in the next version. Let me know if you need it now and I'll share the code here.
     
  28. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hi @TerraUnity thank you for your (many :) ) replies. Yes I would very much like to use the Terrain Satellite Image Stitcher code if it is available. I have tried writing my own but it would be great to have something that works with the TL3 code.

    Google has a custom search engine tool available. I'll mock something up from the TL threads and let you know how it goes https://cse.google.com/cse/ I don't think I can index Discord but I'll have a quick look.

    Thanks
     
  29. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Here you go:

    Code (CSharp):
    1. private void CombineImages(string path, string format, int xTiles, int yTiles, int cellResolution)
    2.     {
    3.         int width = xTiles * cellResolution;
    4.         int length = yTiles * cellResolution;
    5.         Texture2D image = new Texture2D(width, length);
    6.         string[] names = Directory.GetFiles(@path, "*" + format);
    7.         //names = LogicalComparer(names); Uncomment if you need logical comparer sorting of file names
    8.         int index = 0;
    9.  
    10.         for (int y = (yTiles - 1); y >= 0; y--)
    11.         {
    12.             for (int x = 0; x < xTiles; x++)
    13.             {
    14.                 string name = names[index];
    15.                 byte[] bytes = File.ReadAllBytes(name);
    16.                 Texture2D cellImage = new Texture2D(cellResolution, cellResolution);
    17.                 cellImage.LoadImage(bytes);
    18.  
    19.                 image.SetPixels(x * cellResolution, y * cellResolution, cellResolution, cellResolution, cellImage.GetPixels());
    20.  
    21.                 index++;
    22.             }
    23.         }
    24.  
    25.         image.Apply(true);
    26.     }
     
    Last edited: Feb 16, 2019
  30. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
  31. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    I think your videos are great and very well produced. I will copy your speaking style and pace if I ever make tutorial videos ;) - seriously one of the best I have found.

    I like videos but sometimes I want to read stuff and see how it all fits together.

    I tried Google Custom Search on the TL2 thread and I was not happy with the results
     
    TerraUnity likes this.
  32. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hi @TerraUnity

    This is a support question about the accuracy of Lat Long 2 Unity script. I know it is a kind of crazy question and I am not complaining (I love the product and service and will write a good review when I have learnt the product a bit more) but just want to ask how I would go about increasing the accuracy of the script. I've used Google Maps a lot in the past and GIS applications so I know the maths is out there somewhere. If you could give me some suggestions I would be most thankful.

    I'm not making games - I'm using TerraUnity to make videos. I'd like to do some WW2 history videos and I need high accuracy for these very large areas.

    I have created a 1000km X 1000km area of Europe (I hope to stitch a number of these size tiles together to make the whole of Europe eventually). I have used your Lat Long 2 Unity script to place a red sphere. You can just see it in the image below - the yellow arrow shows where the sphere is on the map. The red arrow shows (approximately) where it should be.

    AccuracyDover1000km.png

    Here's a 20km tile with the red sphere at the exact and correct location.

    AccuracyDover20kmB.png

    Both balls are driven by the same script

    GeoLocation.png

    Can you suggest to me what I should look at to increase accuracy on the very large maps? I appreciate error will arise especially with big maps and pixel values - but is there anything to be done?

    Thanks and Cheers
     
    TerraUnity likes this.
  33. blacksun666

    blacksun666

    Joined:
    Dec 17, 2015
    Posts:
    214
    This https://www.movable-type.co.uk/scripts/latlong.html includes the ‘haversine’ formula (including some javascript) which takes into account the curvature of the earth. You should be able to use this in a modified form with your 'origin' of the map and the target location to work out the x and y offsets then you just need to convert to unity offsets and scale in a similar way to the way Terraland does it. I found the Terraland way works ok for short distances but when the map is large the errors introduced by assuming the earth is flat starts to have an effect.
     
    TerraUnity likes this.
  34. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hey @blacksun666 thanks very much for your reply - I have experience with haversine (a long time ago and I remember it made my head hurt real bad :confused:) Will check it out and see if I can mash something together.

    Cheers

    <edit> found this https://github.com/ombharatiya/Unity-GPS-Dist-Count/blob/master/Assets/GPSManager.cs

    Code (CSharp):
    1.     float Haversine(ref float lastLatitude,ref float lastLongitude)
    2.     {
    3.         float newLatitude = Input.location.lastData.latitude;
    4.         float newLongitude = Input.location.lastData.longitude;
    5.         float deltaLatitude = (newLatitude - lastLatitude) * Mathf.Deg2Rad;
    6.         float deltaLongitude = (newLongitude - lastLongitude) * Mathf.Deg2Rad;
    7.         float a = Mathf.Pow(Mathf.Sin(deltaLatitude / 2), 2) + Mathf.Cos(lastLatitude * Mathf.Deg2Rad) * Mathf.Cos(newLatitude * Mathf.Deg2Rad) *
    8.             Mathf.Pow(Mathf.Sin(deltaLongitude / 2), 2);
    9.         lastLatitude = newLatitude;
    10.         lastLongitude = newLongitude;
    11.         float c = 2 * Mathf.Atan2(Mathf.Sqrt(a), Mathf.Sqrt(1 - a));
    12.         return EARTH_RADIUS * c;
    13.      
    14.        
    15.     }
     
    TerraUnity and blacksun666 like this.
  35. blacksun666

    blacksun666

    Joined:
    Dec 17, 2015
    Posts:
    214
    Forgot to say I originally used the Terraland code but found it introduced errors due to it's use of floating point numbers (7 digit precision but my coordinates were using 10 digit precision), using the same formulae but changing everything to doubles made things a lot more accurate. Might be worth trying that first as it's a simpler change to make, depends on your accuracy of Geo coordinates. You still have to convert to float at the end when working with transforms etc.(vector3).
     
  36. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    @blacksun666 OK thanks I'll try that first. Turns out there are several haversine functions for Unity available via Google so it shouldn't be too difficult once a refamiliarise myself with the concept.
    Cheers
     
  37. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Please excuse this long post. I think there's something going on with the stability of the origin in the bottom left corner through various Area Sizes. When you look carefully the satellite imagery (Imagery Origin) begins to "drift" away from the origin of the tiles (Terrain Origin). I might be very wrong and misunderstand what is happening but would appreciate your thoughts.

    There are 4 terrain tiles in the following image - the bottom left corner of Tile Terrain 1-2 is effectively the Terrain Origin when the tile is selected - the other 3 tiles join at the origin point (this is of course a relative observation - one might choose one of the other touching corners of other tiles to represent the origin - it just happens to be where Unity shows the gizmo when tile 1-2 is selected). This images shows 50km around Dover in the English Channel. The satellite Imagery Origin = Terrain Origin.

    Haversine-Dover50.png


    The next image if from a Area Size 200km terrain - you can see that the satellite imagery origin is starting to drift from the Tile Terrain 1-2 bottom left corner or Terrain origin. (I stuck a sphere into the scene using the Lat Long 2 Unity script) Imagery Origin != Terrain Origin

    Haversine-Dover200.png

    At 500km the imagery has moved significantly from the Tile Terrain 1-2 bottom left corner - or origin. Thinking it might be the number of images per terrain tile - "Grid per terrain" I increased the value to 4 or 64 images in total. This didn't help

    Strangely the heightmap appears to move with the imagery.

    Haversine-Dover500-4GridsPerTerrain.png

    I'm not sure I understand what is happening here - as the total Area Size in TL3 increases the origin moves from the top left corner of Tile Terrain 1-2 and begins to drift.

    It could be ESRI I guess - but it might also indicate that the TL3 algorithm that puts imagery (and heightmaps) on terrain tiles drifts from the origin. It might be that the ESRI imagery tile sizes != terrain colormap sizes. I don't know.

    This assumes of course that the Terrain Origin and the Imagery Origin should remain constant through any Area Size changes.

    I look forward to your thoughts.

    <edit>spelling
     
    TerraUnity likes this.
  38. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Just another pic to help illustrate the "problem". Here we can see the terrain at 50km X 50km inside Unity and the 4 images from the satellite images folder. You can clearly see that the lat/long origin/s of the 4 satellite images (2-2 bottom right, 2-3 bottom left, 3-2 top right, 3-3 top left) corresponds with the origin on the terrain.

    At higher Area Sizes the terrain origin is no longer at the intersection of the 4 inner image tiles.

    TileOrder20km.png
     
    TerraUnity likes this.
  39. Deleted User

    Deleted User

    Guest

    Hello,

    We're going to purchase this amazing asset, just need to check 3 questions:
    1. Can we cache very big terrain into our project (as we have restriction not to use online connection), almost whole world.
    2. Is there any way to generate ocean height map to simulate the ocean
    3. Does the asset support buildings/trees.

    Thank you in advance.
     
  40. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Thanks for the compliment, don't forget to credit me then :cool: yes agreed on text documents, we hope to prepare updated ones soon. Do not hesitate to ask questions here if you didn't find answers through searching.
     
  41. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    @ghtx1138 @blacksun666 Will look into improving the geo-location code in longer distances. Are you sure it's related to taking account the haversine in the formula? Does the modified code remove the errors?

    Asking this because found some inaccuracies in projections used in TL for longer distance calculations which I'm going to give more details in the next post.
     
  42. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Yes, it is more accurate with doubles if you need reference to those values outside Unity but as you said all coordinates need to be converted into floats to do positioning in Unity; heard that a double precision math library is on the way for Unity though.
     
  43. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    You are exactly right about the drifting origin for the tiles and returned coordinates to obtain heightmaps/satellite images. Actually I saw these drifting errors before in larger areas but never knew that it is that much, because who wants a terrain more than 200 kilometers?! :p and also it would be trivial when not doing geo-referencing or overlapped terrains setup.

    So after several hours of fiddling with the existing codes and R&D found that neither ESRI nor the formula return wrong outputs; it's the projection and spatial reference which has this kind of artifact while calculating distances in larger areas of Earth surface. You know that the Earth is a not a perfect sphere (unfortunately :D) and most of the coordination conversion algorithms are linear instead of being spherical. So with all being said here are some articles regarding the reasons why this kind of conversions and projections produce errors as the area gets bigger:

    https://gis.stackexchange.com/quest...ations-support-web-mercator-epsg-3857-for-wms
    https://gis.stackexchange.com/questions/283661/postgis-epsg-3857-magic-number

    This thread goes further and suggests some solutions:

    https://gis.stackexchange.com/quest...tance-measurements-in-web-mercator-projection
    &
    I'll try these first to see what comes up to get better results, maybe!


    And the most reliable solution is the ESRI Geometry Service to do the conversions and other useful calculations for us. Regarding the Mercator projection as the subject spatial reference and conversions: https://www.esri.com/arcgis-blog/pr...s-when-your-map-uses-the-mercator-projection/
    So in summary, the more you get far from the Equator in North or South the more distortion errors will occur in returned geometries unless using a projection suited for distance measurements.

    Ok, time for a real test in TerraLand. The following screenshots show 2x2 generated terrains centered at (0, -77.9879552734375) "Cayambe Ecuador", one with 100km2 (red) and the other one with 500km2 (white) in size:

    Cayambe0_1.JPG

    Cayambe0_2.JPG


    As the given latitude is 0, so we are on the equator and will end up with exact centers of surfaces no matter the area size.


    The following screenshots show same terrains this time centered at (45, -77.9879552734375) in southeast Ontario with the same longitude but with a latitude of +45 (in the middle of Equator and North Pole):

    Ontario_45_1.JPG

    Ontario_45_2.JPG

    Note: Elevation Exaggeration is set to 10 for better visual reference.

    The first image clearly shows that terrains do not overlap properly but the latter shows they do overlap exactly but we had to move the bigger terrain 4600 units in z to remove the produced error. Hopefully we are fine with any values for longitude as the measurements in this coordinate are exact and there's no need to move terrains in x to overlap.


    Hopefully there are ways to remove/reduce these artifacts which I'll consider implementing them in TerraLand, I'll keep you updated about the updated codes in this regard.

    UPDATE: Current geo-referencing demo scene in TerraLand has precise geo-referencing solution in place which ouputs accurate results synced with TerraLand generated terrains.
     
    Last edited: May 29, 2021
    henryqng likes this.
  44. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Thanks, Yes, through TerraLand's Offline Streaming system you don't need online connections at all but regarding the whole Earth scenario, it is technically possible to stream unlimited amount of data in runtime but you have to deal with the storage considerations as it will be huge if you choose to have high-res tiles for the entire Earth surface.

    Yes, TL already deals with the underwater (below 0) values in the heightmap data and perform a local normalization on heights to consider negative values too. But the built-in heightmap downloader in TL uses ESRI's world elevation data which its included bathymetry data is very low resolution; however, you may know that in TL you can simply use custom heightmap/imagery data from any external sources of yours through the processors in "TerraLand Terrain" component.

    This question is a broad one and does not indicate if you mean automatic placement/generation based on geo data sources or simple terrain analysis for placement?

    Either way, it is possible through our upcoming product which generates geo-referenced cities with all natural/urban elements using GIS data and built-in algorithms.

    But for TerraLand, we are going to release a new update in a few days with a feature to populate user custom assets (e.g. trees, buildings, rocks etc.) on streaming terrains in runtime. That would be a starting point for users to discover how to access generated terrains in the editor or streaming terrains in runtime and start playing with its data.
     
    Last edited: Jan 22, 2019
  45. Deleted User

    Deleted User

    Guest

    Hello,

    Thank you for prompt and complete feedback
     
    TerraUnity likes this.
  46. FuzzyShan

    FuzzyShan

    Joined:
    Jul 30, 2012
    Posts:
    182
    why should we do size Exaggeration? this seems to be pointless, as it makes the terrain bigger and not referring to world units as in unity's 1 unit = 1 meter?
     
  47. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Maybe pointless in your case, but for other thousands of users... Anyways set its value to 1 and every 1 unit equals 1 meter.
     
    henryqng likes this.
  48. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,254
    Hey everyone, our website will be down from today for a few days in favor of migrating its server into a faster one. Please use our Discord channel to get support during the migration. Thanks ;)
     
  49. KiteLionGames

    KiteLionGames

    Joined:
    Dec 27, 2016
    Posts:
    7
    Hey TerraUnity, thank you for being such an active supporter of your product! I haven't gotten a chance to use it yet, here's my problem:

    When loading up the plugin for the first time I get:
    upload_2019-2-20_22-10-10.png
    Microsoft (R) Visual C# Compiler version 2.9.1.65535 (9d34608e)
    Copyright (C) Microsoft Corporation. All rights reserved.

    error CS1703: Multiple assemblies with equivalent identity have been imported: 'C:\SoftwareDevelopment\Exteel 2\Assets\TerraLand\TerraLand Core\Plugins\System.Drawing.dll' and 'C:\Program Files\Unity\Hub\Editor\2018.3.5f1\Editor\Data\NetStandard\compat\2.0.0\shims\netfx\System.Drawing.dll'. Remove one of the duplicate references.​

    So I delete the one under /netfx/ and get:
    upload_2019-2-20_22-15-0.png
    Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(128,70): error CS0433: The type 'Rectangle' exists in both 'System.Drawing, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
    Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(128,33): error CS0115: 'EuclideanColorFilteringOperation.ProcessFilter(UnmanagedImage, Rectangle)': no suitable method found to override
    Assets\TerraLand\TerraLand Core\Scripts\EuclideanColorFilteringOperation.cs(21,14): error CS0534: 'EuclideanColorFilteringOperation' does not implement inherited abstract member 'BaseInPlacePartialFilter.ProcessFilter(UnmanagedImage, Rectangle)'​

    If I delete the one from Terra:
    upload_2019-2-20_22-19-1.png

    Ideas or help would be greatly appreciated!

    EDIT: Fixed but not a permanent fix: the errors go away when I switch to "Scripting Runtime Version: .Net 3.5 (Deprecated)".
     
    Last edited: Feb 21, 2019
    briank likes this.
  50. mattis89

    mattis89

    Joined:
    Jan 10, 2017
    Posts:
    1,151
    Hello! If I want to use only streaming and allready have an 16k terrain splitted into 2k chunks (I have quite many 2k terrain chunks) textured and shaded with CTS... I guess its possibly?