Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Understanding CopyActiveRenderTextureToHeightmap

Discussion in 'Scripting' started by iddqd, Jul 7, 2019.

  1. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hi there

    I have a question regarding this newer Terrain method: https://docs.unity3d.com/2019.2/Doc...nData.CopyActiveRenderTextureToHeightmap.html

    Must the GPU be synchronized back to the CPU at all?

    If I use TerrainHeightmapSyncControl.None as the third parameter - must I later on synchronize or can I leave it all in the GPU and have a fully working Terrain with Grass etc.?

    Also, if I use TerrainHeightmapSyncControl.HeightAndLod - will the readback to CPU be done asynchronously? If not, can I use TerrainHeightmapSyncControl.None and then do the readback myself async?

    Hope somebody can help me with this.

    Thank you.
     
  2. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    I did some more digging.

    I guess one reason for CopyActiveRenderTextureToHeightmap is to get around the slow TerrainData.SetHeights() - however, CopyActiveRenderTextureToHeightmap still seems slow because it doesn't copy the data back to the CPU asynchonously. If I'm interpreting this correctly, I'm still getting a blocking Gfx.ReadbackImage instead of the newer https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html

    upload_2019-7-8_9-34-48.png

    Perhaps somebody from Unity can elaborate since this is all pretty new? @wyatttt?

    Many thanks!
     
  3. wyatttt

    wyatttt

    Unity Technologies

    Joined:
    May 9, 2018
    Posts:
    250
    heya

    depends on what you want to do. if you are doing stuff on the CPU based on the heightmap then you will want to ie if you want edit-time painting or physics to work.

    if you don't need to do anything based on the modified terrain (ie cpu collisions) you could probably get by with the sync control set to None. changes will get wiped when serialization kicks in because the heightmap is uploaded again to the GPU

    for the grass, i don't believe it will work correctly. the grass meshes/patches are generated on a CPU thread and reads directly from the CPU-side heightmap.

    it will not use async readback and you are getting a block as the texture data is bussed to the CPU. i want to say the reason we dont use the async readback is that those changes made it into editor source after our terrain changes. you can get around this somewhat by doing as you said and handle the syncing yourself and doing it only when you need to

    using the async readback might be a little faster but you'd still have to set the heights manually when the request is done
     
    Last edited: Jul 9, 2019
  4. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hey @wyatttt

    Thanks a lot for replying to this. So in the mean time, I figured out that I will be needing a CPU readback for my case.

    Let's say I do the async readback myself, where must I store the result in TerrainData and trigger the LOD calculations? I can only find SetHeights (or SetHeightsDelayLOD) but this will rebuild the Terrain and is actually the method I've been trying to circumvent.

    Or if that's not possible, how about adding TerrainData.SyncHeightmap(bool async) or similar to the API?

    It's not about async being faster (it should be smoother but probably slower), the issue is that SetHeights and now also CopyActiveRenderTextureToHeightmap cause a block resulting in an fps drop which makes procedural/inifinite terrain generation not so smooth as it could be.

    Thanks
     
  5. wyatttt

    wyatttt

    Unity Technologies

    Joined:
    May 9, 2018
    Posts:
    250
  6. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hi @wyatttt

    Thanks for that, but it seems that he solved using a RenderTexture to set the heightmap without using CopyActiveRenderTextureToHeightmap but I cannot see a solution in those posts to get it from the GPU back to the CPU without the bottleneck. I already have the data in the CPU in my own array, I just want to set it somewhere in the Terrain.

    Do you have a solution for this? Is there a way to get a pointer to the CPU Heightmap Data or something similar?

    Thank you very much!
     
  7. isaac-ashdown

    isaac-ashdown

    Joined:
    Jan 30, 2019
    Posts:
    24
    Hello!

    Now we're on 2019.2.0b7, and I'm using the following (note I'm not using render textures, I'm doing everything using the jobs system, and writing to the texture using Texture2D.GetRawTextureData and writing to that in the jobs. This is primarily because I am not good at shader coding though more than anything else):

    Code (CSharp):
    1.  
    2.  
    3. // the Vector2Int here is an index into the terrain patch
    4.             List<(HeightCalculationJob, JobHandle, Vector2Int)> calcJobs = new List<(HeightCalculationJob, JobHandle, Vector2Int)>();
    5.  
    6.             foreach (var heightJob in calcJobs)
    7.             {
    8.                 int tx = heightJob.Item3.x;
    9.                 int ty = heightJob.Item3.y;
    10.                 var dirtyRect = forDirtyRects[tx, ty];
    11.  
    12.                 heightJob.Item2.Complete();
    13.  
    14.                 forHeightTextures[tx, ty].Apply();
    15.  
    16.                 Graphics.CopyTexture(forHeightTextures[tx, ty], forTerrainComps[tx, ty].terrainData.heightmapTexture);
    17.  
    18.                 var dirtySize = dirtyRect.Size;
    19.                 forTerrainComps[tx, ty].terrainData.DirtyHeightmapRegion(new RectInt(dirtyRect.min.x, dirtyRect.min.y, dirtySize.x, dirtySize.y), TerrainHeightmapSyncControl.None);
    20.  
    21.                 var job = heightJob.Item1;
    22.                 job.GetRemainingDirtyResults(ref forDirtyRects[tx, ty]);
    23.  
    24.                 if (wasDetailDataChanged || !forDirtyRects[tx, ty].isDirty)
    25.                 {
    26.                     forTerrainComps[tx, ty].terrainData.SyncHeightmap();
    27.                 }
    28.             }
    So each job calculates the heightmap that I want to set and puts it on a Texture (forHeightTextures). I then copy that to the terrain component's heightmapTexture, then mark the region within the heightmap that I was actually working on dirty, passing in None to syncControl. This is all super fast and allows me to interpolate heights over time to create a smooth transition between terrain types - you can see it in action here.

    Then, I check if the interpolation has completed, and only then do I call SyncHeightmap. This is the expensive part. It causes the phys geo to be updated, and also the grass(detail) billboards that are on that part of the heightmap get set to the correct height. This causes a pop, which I'd rather avoid. But without it it just runs too slow for our needs.

    Today I tried passing in TerrainHeightmapSyncControl.TerrainAndLOD to DirtyHeightmapRegion, and this then updates the phys geo & grass sprites immediately, but is expensive again. I'm not totally sure what the difference betwen doing that and calling SyncHeightmap is, but I guess SyncHeightmap syncs the whole thing, rather than just the dirty region passed in to DirtyHeightmapRegion. But not totally sure - I need to profile it. But just changing that over appears to be still too slow for our game (we have some effects that can cause a number of changes to the heightmap over time, which needs to be buttery smooth. So we're living with the pops for now.

    It would be great though if there was a mode that could be passed in to DirtyHeightmapRegion which only did the necessary calculations for the grass billboards to be correctly positioned every frame, without doing the phys/cpu/LOD update, which I don't care about so much. Would that be a possibility, @wyatttt ?

    It would be nice to have a direct API for the terrain that took NativeArrays, though having this texture-based API instead is not so bad, as the texture API itself can take NativeArrays. And the bottleneck appears to be the sync step anyway.
     
    Last edited: Jul 10, 2019
    Rowlan and iddqd like this.
  8. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hi @isaac-ashdown

    Yes that's where I am now, living with the pop. The pop comes from when the GPU texture is read back to the CPU, that's the bottleneck. I'm pretty sure that the lag will always be the same, whether you do it immediately with TerrainHeightmapSyncControl.TerrainAndLOD or TerrainHeightmapSyncControl.Non and then sync later on. SetHeights has pretty much the same performance for me now again.

    Unity does have a method to read RenderTextures Asynchronously back to CPU which would probably remove the pop: https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html

    But it's not implemented for the Terrain readback. Keeping my fingers crossed for a solution.
     
  9. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hi @isaac-ashdown

    I've had some good results by using a Coroutine to apply the heightmap over several frames.

    So I'm using a 1024 Heightmap on a Terrain, but I'm splitting it up into several chunks (4 or 16 works good for my case) and applying them partially to the terrain. Performance is much better, you should try it.

    I'm using CopyActiveRenderTextureToHeightmap - I will try the same approach with SetHeights perhaps later on.

    Edit: I see I could probably also use DirtyHeightmapRegion for this.

    Good luck
     
    Last edited: Jul 17, 2019
    wyatttt likes this.
  10. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Hi @wyatttt

    Are there perhaps any plans to make the CopyActiveRenderTextureToHeightmap readback optionally Async?

    Splitting CopyActiveRenderTextureToHeightmap into several calls helps a lot, but having it Async would most likely be the most performant solution.

    Thank you
     
  11. isaac-ashdown

    isaac-ashdown

    Joined:
    Jan 30, 2019
    Posts:
    24
    Hi @iddqd - which call is it exactly you're spliiting up over several frames? Doesn't that lead to a noticable effect whereby parts of the heightmap aren't lined up with each other?

    Would you mind posting your coroutine code?
     
  12. thomas-weltenbauer

    thomas-weltenbauer

    Joined:
    Oct 23, 2013
    Posts:
    46
    Yes it would be interesting to see your coroutine code, @iddqd!
    Maybe you could share it?
     
  13. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    So let's say I apply a 1024 Heightmap to the Terrain, I now do this as follows using CopyActiveRenderTextureToHeightmap.

    (rtHeight is the Heightmap in a Rendertexture)

    Code (CSharp):
    1. public static IEnumerator RenderTextureToTerrainChunks(Terrain currentTerrain, RenderTexture rtHeight, int chunks, float secondsForFullHeighmap)
    2.     {
    3.         int numChunksPerRow = (int)Mathf.Sqrt(chunks);
    4.         int chunkResolution = 1024 / numChunksPerRow;
    5.         float waitTime = secondsForFullHeighmap/ chunks;
    6.  
    7.         for (int x = 0; x < numChunksPerRow; x ++)
    8.         {
    9.             for (int y = 0; y < numChunksPerRow; y++)
    10.             {
    11.                
    12.                 yield return new WaitForSecondsRealtime(waitTime);
    13.  
    14.                 int offset = 64;
    15.                 RectInt rect = new RectInt(offset + (chunkResolution * x), offset + (chunkResolution * y), chunkResolution, chunkResolution);
    16.                 Vector2Int vect = new Vector2Int(chunkResolution * x, chunkResolution * y);
    17.                
    18.                 if(x == numChunksPerRow -1)
    19.                 {
    20.                     rect.width += 1; // make it 1025
    21.                 }
    22.  
    23.                 if(y == numChunksPerRow -1)
    24.                 {
    25.                     rect.height += 1; // make it 1025
    26.                 }
    27.  
    28.                 RenderTexture rtActiveOld = RenderTexture.active;
    29.                 RenderTexture.active = rtHeight;
    30.                 currentTerrain.terrainData.CopyActiveRenderTextureToHeightmap(rect, vect, TerrainHeightmapSyncControl.HeightAndLod);
    31.                 RenderTexture.active = rtActiveOld;
    32.             }
    33.         }
    34.     }
    This will split the 1024 HM into let's say 16 chunks and apply these 16 over a duration of secondsForFullHeighmap. So yes, there will be times when the full Terrain is not displayed correctly with incorrect seams, you will maybe want to hide this somehow (add the terrain a few km ahead of the player perhaps).

    Also what I think helped performance was to use https://docs.unity3d.com/2019.2/Doc...ainData.CopyActiveRenderTextureToTexture.html to apply the Alpha/Splat maps.
     
  14. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Of course, I think it would be best if Unity added something like this:
    TerrainHeightmapSyncControl.AsyncHeightOnly
    TerrainHeightmapSyncControl.AsyncHeightAndLod
     
  15. crysicle

    crysicle

    Joined:
    Oct 24, 2018
    Posts:
    52
    I've opened up a discussion about the lack of this option a few months ago here. Would be really great if this was actually implemented.

    I've been trying to come up with a way to have efficient manipulation of the heightmap for months now with all of the LOD and physics synced properly. These are my findings:

    - If you want scarce heightmap manipulation, about 1-3k heightmaps in total each frame, use the job system to calculate the data and update it through SetHeights() or SetHeightsDelayLOD(), otherwise the overhead will greatly stall the CPU. Doing it this way will sync the LOD and the collision data without the need to create your own workarounds/systems to sync everything.

    @wyatttt do you perhaps know if any NativeArray support is coming to the SetHeights() method similar to what is happening in this link to meshes?

    -If you want to update 3k - 4kk or more heightmaps each frame without lag, use the CopyActiveRenderTextureToHeightmap() and do all heightmap manipulation via textures. Currently there is no way to actually sync heightmaps or LOD without huge stalls with Unity's API each frame. Though i've come up with a few workarounds for this using AsyncGPUReadback.

    1. To calculate detail positions/rotations, do all of the calculations inside a shader relative to the texture values rather than the data on the CPU. Gather detail positions relative to the heightmap, send them to a shader, calculate new positions relative to the texture and retrieve the data with AsyncGPUReadback and then assign it back manually.

    2 To sync terrain colliders for physics, split the terrain collider into multiple meshes and retrieve the data using AsyncGPUReadback. I can retrieve about 1kk heightmaps each frame from the GPU no problem with this. However, after assigning the vertices to the mesh, there are baking problems with physX which really only allow about 2k-3k vertice manipulation on mesh colliders each frame before severe lag kicks in.

    @wyatttt do you perhaps know why physX needs to rebake CollisionData for a mesh after assigning new vertices, but not for the terrain? After using SyncHeightmaps() i've noticed that the lag only comes from GPU stall and an internal RecomputeInvalidPatches() function. I created a thread here, though haven't gotten any response yet.

    3. Vert/Tris LOD syncing isn't possible through AsyncGPUReadback due to Unity's black box problem discussed here.

    4. If you need the manipulated heightmap visual effects to be synchronized with physics you'll also need an additional heightmap for every other heightmap. Once you start deforming the heightmap, update the first heightmap instead, use it as a data source for asynchronous detail position/rotation and mesh creation. Once the physics data is retrieved and applied, update the second heightmap, which would be the actual terrain heightmap.

    If the Async functions were to be introduced, it'd also be great if an additional source besides the terrain's heightmap could be used for LOD and terrain collider data, such as an additional heightmap to which we could write first. This would allow to add an effect to the heightmap which is invisible, promt an async request and update all of the data in one go seamlessly : visible heightmap, terrain collider, detail positions/rotations, and vert/tris LOD.
     
    Last edited: Aug 16, 2019 at 2:55 PM
    iddqd likes this.
  16. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    394
    Interesting stuff, thanks for that input.