Understanding CopyActiveRenderTextureToHeightmap

Hi there

I have a question regarding this newer Terrain method: https://docs.unity3d.com/2019.2/Doc...nData.CopyActiveRenderTextureToHeightmap.html

Must the GPU be synchronized back to the CPU at all?

If I use TerrainHeightmapSyncControl.None as the third parameter - must I later on synchronize or can I leave it all in the GPU and have a fully working Terrain with Grass etc.?

Also, if I use TerrainHeightmapSyncControl.HeightAndLod - will the readback to CPU be done asynchronously? If not, can I use TerrainHeightmapSyncControl.None and then do the readback myself async?

Hope somebody can help me with this.

Thank you.

 

I did some more digging.

I guess one reason for CopyActiveRenderTextureToHeightmap is to get around the slow TerrainData.SetHeights() - however, CopyActiveRenderTextureToHeightmap still seems slow because it doesn't copy the data back to the CPU asynchonously. If I'm interpreting this correctly, I'm still getting a blocking Gfx.ReadbackImage instead of the newer https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html



Perhaps somebody from Unity can elaborate since this is all pretty new? @wyatttt?

Many thanks!

 

Hey @wyatttt

Thanks a lot for replying to this. So in the mean time, I figured out that I will be needing a CPU readback for my case.

Let's say I do the async readback myself, where must I store the result in TerrainData and trigger the LOD calculations? I can only find SetHeights (or SetHeightsDelayLOD) but this will rebuild the Terrain and is actually the method I've been trying to circumvent.

Or if that's not possible, how about adding TerrainData.SyncHeightmap(bool async) or similar to the API?

It's not about async being faster (it should be smoother but probably slower), the issue is that SetHeights and now also CopyActiveRenderTextureToHeightmap cause a block resulting in an fps drop which makes procedural/inifinite terrain generation not so smooth as it could be.

Thanks

 

Hi @wyatttt

Thanks for that, but it seems that he solved using a RenderTexture to set the heightmap without using CopyActiveRenderTextureToHeightmap but I cannot see a solution in those posts to get it from the GPU back to the CPU without the bottleneck. I already have the data in the CPU in my own array, I just want to set it somewhere in the Terrain.

Do you have a solution for this? Is there a way to get a pointer to the CPU Heightmap Data or something similar?

Thank you very much!

 

Hello!

Now we're on 2019.2.0b7, and I'm using the following (note I'm not using render textures, I'm doing everything using the jobs system, and writing to the texture using Texture2D.GetRawTextureData and writing to that in the jobs. This is primarily because I am not good at shader coding though more than anything else):

Code (CSharp):

1.  
2.  
3. // the Vector2Int here is an index into the terrain patch
4.             List<(HeightCalculationJob, JobHandle, Vector2Int)> calcJobs = new List<(HeightCalculationJob, JobHandle, Vector2Int)>();
5.  
6.             foreach (var heightJob in calcJobs)
7.             {
8.                 int tx = heightJob.Item3.x;
9.                 int ty = heightJob.Item3.y;
10.                 var dirtyRect = forDirtyRects[tx, ty];
11.  
12.                 heightJob.Item2.Complete();
13.  
14.                 forHeightTextures[tx, ty].Apply();
15.  
16.                 Graphics.CopyTexture(forHeightTextures[tx, ty], forTerrainComps[tx, ty].terrainData.heightmapTexture);
17.  
18.                 var dirtySize = dirtyRect.Size;
19.                 forTerrainComps[tx, ty].terrainData.DirtyHeightmapRegion(new RectInt(dirtyRect.min.x, dirtyRect.min.y, dirtySize.x, dirtySize.y), TerrainHeightmapSyncControl.None);
20.  
21.                 var job = heightJob.Item1;
22.                 job.GetRemainingDirtyResults(ref forDirtyRects[tx, ty]);
23.  
24.                 if (wasDetailDataChanged || !forDirtyRects[tx, ty].isDirty)
25.                 {
26.                     forTerrainComps[tx, ty].terrainData.SyncHeightmap();
27.                 }
28.             }

So each job calculates the heightmap that I want to set and puts it on a Texture (forHeightTextures). I then copy that to the terrain component's heightmapTexture, then mark the region within the heightmap that I was actually working on dirty, passing in None to syncControl. This is all super fast and allows me to interpolate heights over time to create a smooth transition between terrain types - you can see it in action here.

Then, I check if the interpolation has completed, and only then do I call SyncHeightmap. This is the expensive part. It causes the phys geo to be updated, and also the grass(detail) billboards that are on that part of the heightmap get set to the correct height. This causes a pop, which I'd rather avoid. But without it it just runs too slow for our needs.

Today I tried passing in TerrainHeightmapSyncControl.TerrainAndLOD to DirtyHeightmapRegion, and this then updates the phys geo & grass sprites immediately, but is expensive again. I'm not totally sure what the difference betwen doing that and calling SyncHeightmap is, but I guess SyncHeightmap syncs the whole thing, rather than just the dirty region passed in to DirtyHeightmapRegion. But not totally sure - I need to profile it. But just changing that over appears to be still too slow for our game (we have some effects that can cause a number of changes to the heightmap over time, which needs to be buttery smooth. So we're living with the pops for now.

It would be great though if there was a mode that could be passed in to DirtyHeightmapRegion which only did the necessary calculations for the grass billboards to be correctly positioned every frame, without doing the phys/cpu/LOD update, which I don't care about so much. Would that be a possibility, @wyatttt ?

It would be nice to have a direct API for the terrain that took NativeArrays, though having this texture-based API instead is not so bad, as the texture API itself can take NativeArrays. And the bottleneck appears to be the sync step anyway.

 

Hi @isaac-ashdown

Yes that's where I am now, living with the pop. The pop comes from when the GPU texture is read back to the CPU, that's the bottleneck. I'm pretty sure that the lag will always be the same, whether you do it immediately with TerrainHeightmapSyncControl.TerrainAndLOD or TerrainHeightmapSyncControl.Non and then sync later on. SetHeights has pretty much the same performance for me now again.

Unity does have a method to read RenderTextures Asynchronously back to CPU which would probably remove the pop: https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html

But it's not implemented for the Terrain readback. Keeping my fingers crossed for a solution.

 

Hi @isaac-ashdown

I've had some good results by using a Coroutine to apply the heightmap over several frames.

So I'm using a 1024 Heightmap on a Terrain, but I'm splitting it up into several chunks (4 or 16 works good for my case) and applying them partially to the terrain. Performance is much better, you should try it.

I'm using CopyActiveRenderTextureToHeightmap - I will try the same approach with SetHeights perhaps later on.

Edit: I see I could probably also use DirtyHeightmapRegion for this.

Good luck

 

Hi @wyatttt

Are there perhaps any plans to make the CopyActiveRenderTextureToHeightmap readback optionally Async?

Splitting CopyActiveRenderTextureToHeightmap into several calls helps a lot, but having it Async would most likely be the most performant solution.

Thank you

 

Hi @iddqd - which call is it exactly you're spliiting up over several frames? Doesn't that lead to a noticable effect whereby parts of the heightmap aren't lined up with each other?

Would you mind posting your coroutine code?

 

Yes it would be interesting to see your coroutine code, @iddqd!
Maybe you could share it?

 

So let's say I apply a 1024 Heightmap to the Terrain, I now do this as follows using CopyActiveRenderTextureToHeightmap.

(rtHeight is the Heightmap in a Rendertexture)

Code (CSharp):

1. public static IEnumerator RenderTextureToTerrainChunks(Terrain currentTerrain, RenderTexture rtHeight, int chunks, float secondsForFullHeighmap)
2.     {
3.         int numChunksPerRow = (int)Mathf.Sqrt(chunks);
4.         int chunkResolution = 1024 / numChunksPerRow;
5.         float waitTime = secondsForFullHeighmap/ chunks;
6.  
7.         for (int x = 0; x < numChunksPerRow; x ++)
8.         {
9.             for (int y = 0; y < numChunksPerRow; y++)
10.             {
11.                
12.                 yield return new WaitForSecondsRealtime(waitTime);
13.  
14.                 int offset = 64;
15.                 RectInt rect = new RectInt(offset + (chunkResolution * x), offset + (chunkResolution * y), chunkResolution, chunkResolution);
16.                 Vector2Int vect = new Vector2Int(chunkResolution * x, chunkResolution * y);
17.                
18.                 if(x == numChunksPerRow -1)
19.                 {
20.                     rect.width += 1; // make it 1025
21.                 }
22.  
23.                 if(y == numChunksPerRow -1)
24.                 {
25.                     rect.height += 1; // make it 1025
26.                 }
27.  
28.                 RenderTexture rtActiveOld = RenderTexture.active;
29.                 RenderTexture.active = rtHeight;
30.                 currentTerrain.terrainData.CopyActiveRenderTextureToHeightmap(rect, vect, TerrainHeightmapSyncControl.HeightAndLod);
31.                 RenderTexture.active = rtActiveOld;
32.             }
33.         }
34.     }

This will split the 1024 HM into let's say 16 chunks and apply these 16 over a duration of secondsForFullHeighmap. So yes, there will be times when the full Terrain is not displayed correctly with incorrect seams, you will maybe want to hide this somehow (add the terrain a few km ahead of the player perhaps).

Also what I think helped performance was to use https://docs.unity3d.com/2019.2/Doc...ainData.CopyActiveRenderTextureToTexture.html to apply the Alpha/Splat maps.

 

Of course, I think it would be best if Unity added something like this:
TerrainHeightmapSyncControl.AsyncHeightOnly
TerrainHeightmapSyncControl.AsyncHeightAndLod

 

I've opened up a discussion about the lack of this option a few months ago here. Would be really great if this was actually implemented.

I've been trying to come up with a way to have efficient manipulation of the heightmap for months now with all of the LOD and physics synced properly. These are my findings:

- If you want scarce heightmap manipulation, about 1-3k heightmaps in total each frame, use the job system to calculate the data and update it through SetHeights() or SetHeightsDelayLOD(), otherwise the overhead will greatly stall the CPU. Doing it this way will sync the LOD and the collision data without the need to create your own workarounds/systems to sync everything.

@wyatttt do you perhaps know if any NativeArray support is coming to the SetHeights() method similar to what is happening in this link to meshes?

-If you want to update 3k - 4kk or more heightmaps each frame without lag, use the CopyActiveRenderTextureToHeightmap() and do all heightmap manipulation via textures. Currently there is no way to actually sync heightmaps or LOD without huge stalls with Unity's API each frame. Though i've come up with a few workarounds for this using AsyncGPUReadback.

1. To calculate detail positions/rotations, do all of the calculations inside a shader relative to the texture values rather than the data on the CPU. Gather detail positions relative to the heightmap, send them to a shader, calculate new positions relative to the texture and retrieve the data with AsyncGPUReadback and then assign it back manually.

2 To sync terrain colliders for physics, split the terrain collider into multiple meshes and retrieve the data using AsyncGPUReadback. I can retrieve about 1kk heightmaps each frame from the GPU no problem with this. However, after assigning the vertices to the mesh, there are baking problems with physX which really only allow about 2k-3k vertice manipulation on mesh colliders each frame before severe lag kicks in.

@wyatttt do you perhaps know why physX needs to rebake CollisionData for a mesh after assigning new vertices, but not for the terrain? After using SyncHeightmaps() i've noticed that the lag only comes from GPU stall and an internal RecomputeInvalidPatches() function. I created a thread here, though haven't gotten any response yet.

3. Vert/Tris LOD syncing isn't possible through AsyncGPUReadback due to Unity's black box problem discussed here.

4. If you need the manipulated heightmap visual effects to be synchronized with physics you'll also need an additional heightmap for every other heightmap. Once you start deforming the heightmap, update the first heightmap instead, use it as a data source for asynchronous detail position/rotation and mesh creation. Once the physics data is retrieved and applied, update the second heightmap, which would be the actual terrain heightmap.

If the Async functions were to be introduced, it'd also be great if an additional source besides the terrain's heightmap could be used for LOD and terrain collider data, such as an additional heightmap to which we could write first. This would allow to add an effect to the heightmap which is invisible, promt an async request and update all of the data in one go seamlessly : visible heightmap, terrain collider, detail positions/rotations, and vert/tris LOD.

 

Interesting stuff, thanks for that input.

 

I have a weird issue with an Asset Store product that is using DirtyHeightmapRegion. I've tested with varying sizes for the region that is dirtied, all the way from a 65 x 65 region down to a 5 x 5 region of the heightmap, and no matter the size the performance of the method is the same (~54ms on built Player). I'd post the code but it's obviously not mine to post. Any ideas on what could be happening here?

 

A bit of an old thread, but I have a heightmap as a render texture and when I use this function it appears it is incorrectly setting the heights and resulting in some weird clipping issues. It seems that this method is creating terrains with 0.5 as the maximum height value, not 1, whereas setting heights from an float[,] uses 1 as the maximum height.

On 2019.4.5f1
Terrain size: (1024, 1024, 1024)
Heightmap Resolution: 513
Heightmap full of values of 0.5

The terrain is a plane at point 1024, whereas I would expect it to be at 512.

Am I forgetting to do something here?

 

The values in the heightmap are at 0.5 which maxes the terrain height. Values above 0.5 "work" but cause very weird clipping issues since it sounds like they are out of range..

Sounds like it is working as designed then. Would be cool if the docs specified that the value ranges are different.

Thanks for the quick response, now that I know to shift my heightmap scale everything is pretty awesome.

 

Sure, here is a picture of the weird clipping issues. This is in scene view, but play mode does the same thing.

 

The reason why clipping happens is because any value in the shader that's higher than 0.5 will loop back from -0.5. E.G. 0.7 will be treated as -0.3, 0.55 as -0.45. This makes the heightmap go below it's 0 range. By the looks of it, the PatchExtents which is there to represent the bounds of a single mesh node the terrain uses also falsely calculates the bounds when some values are below 0. More info here.