Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

CustomRenderTexture with TerrainPaintUtility

Discussion in 'World Building' started by palten08, Aug 4, 2019.

  1. palten08

    palten08

    Joined:
    Jul 31, 2019
    Posts:
    6
    I assume this is possible, but so far I've only been able to make modifications to the terrain in a rectangle, not the shape of the texture I'm assigning to my CustomRenderTexture (Maybe I'm missing something?)

    Here's what I've tried so far:

    1. Created a gray-scale JPG for my test brush
    2. Added this script to my terrain object:
    Code (CSharp):
    1. public class TerrainTest : MonoBehaviour
    2. {
    3.     [SerializeField] ComputeShader shader;
    4.     public RenderTexture terrainRenderTexture;
    5.     public CustomRenderTexture brush;
    6.     public Texture brushTexture;
    7.     public Vector2 rectSize;
    8.  
    9.     private Rect rect;
    10.     private Terrain terrain;
    11.     private PaintContext paintContext;
    12.  
    13.     void Start()
    14.     {
    15.         terrain = GetComponent<Terrain>();
    16.     }
    17.  
    18.     void Update()
    19.     {
    20.         if (Input.GetMouseButton(0))
    21.         {
    22.             RaycastHit rayHit;
    23.             var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    24.  
    25.             if (Physics.Raycast(ray, out rayHit))
    26.             {
    27.                 Vector2 terrainPoint = new Vector2(rayHit.point.x, rayHit.point.z);
    28.                 rect = new Rect(terrainPoint, rectSize);
    29.  
    30.                 ModifyTerrain(rect);
    31.             }
    32.         }
    33.     }
    34.  
    35.     void ModifyTerrain(Rect selectionRect)
    36.     {
    37.         paintContext = TerrainPaintUtility.BeginPaintHeightmap(terrain, selectionRect);
    38.  
    39.         terrainRenderTexture = paintContext.sourceRenderTexture;
    40.  
    41.         brush = new CustomRenderTexture(terrainRenderTexture.width, terrainRenderTexture.height);
    42.         brush.enableRandomWrite = true;
    43.         brush.format = RenderTextureFormat.R16;
    44.         brush.initializationTexture = brushTexture;
    45.         brush.Create();
    46.  
    47.         Graphics.CopyTexture(terrainRenderTexture, brush);
    48.  
    49.         shader.SetTexture(shader.FindKernel("CSMain"), "Result", brush);
    50.         shader.Dispatch(shader.FindKernel("CSMain"), 1, 1, 1);
    51.  
    52.         Graphics.CopyTexture(brush, paintContext.destinationRenderTexture);
    53.  
    54.         TerrainPaintUtility.EndPaintHeightmap(paintContext, "Terrain");
    55.  
    56.         terrain.terrainData.SyncHeightmap();
    57.     }
    58. }
    3. Used this compute shader (I'm honestly not even sure what this compute shader is actually modifying, I just stumbled across someone else's code while trying to look up examples of TerrainPaintUtility in action and this is the only thing I could find):
    Code (CSharp):
    1.  
    2. #pragma kernel CSMain
    3.  
    4. RWTexture2D<float4> Result;
    5.  
    6. [numthreads(8,8,1)]
    7. void CSMain (uint3 id : SV_DispatchThreadID)
    8. {
    9.     Result[id.xy] += 0.0001; //Still can't quite figure out what on the object that gets passed to this is actually being modified.
    10.     Result[id.xy] = clamp(Result[id.xy], 0, 1);
    11. }
    Is my methodology flawed here? I assumed that TerrainPaintUtility would allow you to "paint" onto the heightmap, thus allowing you to use "brushes" with different shapes, instead of doing things the terrainData.SetHeight() way where you need to find points mathematically (and thus might not let you implement selection areas in shapes other than basic circles and squares)
     
  2. palten08

    palten08

    Joined:
    Jul 31, 2019
    Posts:
    6
    Update: Using "CopyActiveRenderTextureToHeightMap", I was able to apply my brush shape to the terrain, however it still isn't without issue:

    Here's the modification shown on the terrain object itself
    Here's the resultant heightmap

    While I at least was able to see a modification that actually follows the shape outlined in the brush's JPG, it appears that the modification is not additive like I was hoping it would be, it's essentially the same thing as dragging the heightmap into paint and pasting the brush JPG over it, rather than modifying the alpha channel with the alpha values from the brush (As shown here). In addition, the brush itself is all jacked up, so I imagine some settings somewhere are not correct, I just can't identify which ones they might be.

    Here's my code:

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.Experimental.TerrainAPI;
    3.  
    4. public class TerrainTest : MonoBehaviour
    5. {
    6.     public Texture brushTexture;
    7.     public Vector2 rectSize;
    8.  
    9.     private Vector3 terrainPosition;
    10.     private Rect rect;
    11.     private Terrain terrain;
    12.     private PaintContext paintContext;
    13.  
    14.     void Start()
    15.     {
    16.         terrain = GetComponent<Terrain>();
    17.         terrainPosition = terrain.transform.position;
    18.         Texture2D texture2D = new Texture2D(brushTexture.width, brushTexture.height, TextureFormat.RGBA32, false);
    19.     }
    20.  
    21.     void Update()
    22.     {
    23.         if (Input.GetMouseButton(0))
    24.         {
    25.             RaycastHit rayHit;
    26.             var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    27.  
    28.             if (Physics.Raycast(ray, out rayHit))
    29.             {
    30.                 int terrainCoordinateX = Mathf.FloorToInt(((rayHit.point.x - terrainPosition.x) / terrain.terrainData.size.x) * terrain.terrainData.heightmapResolution);
    31.                 int terrainCoordinateY = Mathf.FloorToInt(((rayHit.point.z - terrainPosition.z) / terrain.terrainData.size.z) * terrain.terrainData.heightmapResolution);
    32.                 ModifyTerrain(terrainCoordinateX, terrainCoordinateY);
    33.             }
    34.         }
    35.     }
    36.  
    37.    
    38.     void ModifyTerrain(int xCoordinate, int yCoordinate)
    39.     {
    40.         RenderTexture currentRenderTexture = RenderTexture.active;
    41.         RenderTexture renderTexture = new RenderTexture(brushTexture.width, brushTexture.height, 32);
    42.         Graphics.Blit(brushTexture, renderTexture);
    43.         RenderTexture.active = renderTexture;
    44.         RectInt sourceRect = new RectInt(new Vector2Int(0, 0), new Vector2Int(renderTexture.width, renderTexture.height));
    45.         terrain.terrainData.CopyActiveRenderTextureToHeightmap(sourceRect, new Vector2Int(xCoordinate, yCoordinate), TerrainHeightmapSyncControl.None);
    46.         terrain.terrainData.SyncHeightmap();
    47.     }
    (I realize the modification function itself is the opposite of performant in this state, but this was just to see if it would work before going any further)
     
    EirikWahl likes this.
  3. crysicle

    crysicle

    Joined:
    Oct 24, 2018
    Posts:
    78
    From the picture and code provided it looks like you're directly assigning the brush texture to the heightmap instead of adding it to the current one, which is why the effect is the one you see. I'm not sure if manipulating RenderTextures is only available in compute shaders, but that's the only way i've found how. In any case, it will have to be done inside some kind of shader because we're looking to manipulate textures. In order to achieve the effect of actually painting on the heightmap you need to copy the current heightmap. This can be done via CopyTexture or maybe even by PaintContext. You then need to send both the brush and the copied heightmap texture to the compute shader and dispatch them with logic similar to this:

    1. Code (CSharp):
      1. #pragma kernel CSMain
      2.  
      3. RWTexture2D<half> heightmap; // Halfs can be used for full precision instead of float because heightmaps are int16 anyway.
      4. Texture2D<half4> brush; // Brush texture which you're using to paint on top of the heightmap
      5. half height_multiplier = 0.01; // Brush intensity
      6.  
      7. [numthreads(32,32,1)] // search a bit more about compute shaders to figure out how to use numthreads properly
      8. void CSMain (uint3 id : SV_DispatchThreadID)
      9. {
      10.     half new_heightmap = heightmap[id.xy].x + brush[id.xy].w * height_multiplier; // Use whichever channel you want from the brush. You'll also have to figure out how to sample proper uv coordinates from the brush texture instead of using "id.xy".
      11.     heightmap[id.xy].x = clamp(new_heightmap , 0, 0.5);    // 0.5, because that's the range devs decided to treat the texture internally. Bugs occur beyond 0.5.
      12. }
    After you generate the new heightmap just copy it back via "CopyActiveRenderTextureToHeightmap".
     
  4. palten08

    palten08

    Joined:
    Jul 31, 2019
    Posts:
    6

    I had sidelined this for a bit to work on some other stuff so sorry for the delayed response!

    Your reply makes sense, although I think I've run into a syntax problem with Unity's API on how to actually accomplish this

    Code (CSharp):
    1.     void ModifyTerrain(Rect selection)
    2.     {
    3.         paintContext = TerrainPaintUtility.BeginPaintHeightmap(terrain, selection);
    4.  
    5.         RenderTexture terrainRenderTexture = paintContext.sourceRenderTexture;
    6.  
    7.         terrainPaintShader.SetTexture(terrainPaintShader.FindKernel("CSMain"), "originalHeightmap", terrainRenderTexture);
    8.         terrainPaintShader.SetTexture(terrainPaintShader.FindKernel("CSMain"), "brush", brushTexture);
    9.         terrainPaintShader.Dispatch(terrainPaintShader.FindKernel("CSMain"), 1, 1, 1);
    10.  
    11.         Graphics.CopyTexture(terrainRenderTexture, paintContext.destinationRenderTexture);
    12.  
    13.         TerrainPaintUtility.EndPaintHeightmap(paintContext, "Terrain");
    14.  
    15.         terrain.terrainData.SyncHeightmap();
    16.  
    17.     }
    When I run this, however, I get: "IndexOutOfRangeException: Invalid kernelIndex (0) passed, must be non-negative less than 1.
    UnityEngine.ComputeShader.SetTexture (System.Int32 kernelIndex, System.String name, UnityEngine.Texture texture) (at <63fe705fb8b344d5be62daf83244d046>:0)"

    So I'm thinking that I'm not passing my parameters to the compute shader properly. I tried searching for "compute shader multiple parameters" but, for the life of me, I couldn't find any relevant threads or documentation.
     
  5. crysicle

    crysicle

    Joined:
    Oct 24, 2018
    Posts:
    78
    I think that error comes from 2 situations only:
    1. The kernel which you're looking for doesn't exist. Which means either the string provided in the FindKernel is incorrect or the kernel is never defined inside the compute shader with a #pragma or the kernel is defined in the compute shader, but a function with the same name is not.
    2. The kernel was found, but the variable you're trying to assign a value to inside one of the kernels doesn't exist. Which either means the variable you're trying to assign was never defined in the shader or you mistyped the name of the variable you're trying to assign the value to.

    Assigning multiple parameters to the compute shaders isn't the issue. As far as i know shaders support hundreads of parameters to which you can assign values to. If you'd post the shader code you're using, the mistake should be quite easy to find.
     
  6. palten08

    palten08

    Joined:
    Jul 31, 2019
    Posts:
    6
    Sorry about that, I meant to specify that I was using the shader code you had provided:

    Code (CSharp):
    1. // Each #kernel tells which function to compile; you can have many kernels
    2. #pragma kernel CSMain
    3.  
    4. // Create a RenderTexture with enableRandomWrite flag and set it
    5. // with cs.SetTexture
    6. RWTexture2D<half> originalHeightmap;
    7. Texture2D<half4> brush;
    8. half height_multiplier = 0.01;
    9.  
    10. [numthreads(8,8,1)]
    11. void CSMain (uint3 id : SV_DispatchThreadID)
    12. {
    13.     half newHeightmap = originalHeightmap[id.xy].x + brush[id.xy].w * height_multiplier;
    14.     originalHeightmap[id.xy].x = clamp(newHeightmap, 0, 0.5);
    15. }
    16.  
    I don't see any typos, is there a chance this is a type issue? C# is my first statically-typed language so I'm sure it's a possibility, although I would have imagined that Unity or Visual Studio would have attempted to tell me about the type mis-match
     
  7. crysicle

    crysicle

    Joined:
    Oct 24, 2018
    Posts:
    78
    Upon second viewing i noticed that the texture you're sending to the shader is a PaintContext one. Those do not have the enableRandomWrite set to true. Not sure why though, probably the devs just forgot about it. You'll have to do an additional texture copy to create a proper RenderTexture which you can send to the shader.

    Code (CSharp):
    1. // Replace the
    2. RenderTexture terrainRenderTexture = paintContext.sourceRenderTexture;
    3. // with
    4. RenderTexture terrainRenderTexture = new RenderTexture(paintContext.sourceRenderTexture.width, context.sourceRenderTexture.height, 0, RenderTextureFormat.R16);
    5. terrainRenderTexture.enableRandomWrite = true;
    6. Graphics.CopyTexture(context.sourceRenderTexture, terrainRenderTexture);
    7.  
    You'll have to dispose of this texture on your own after you're done with it with terrainRenderTexture.Release().
     
  8. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,685
    hey @palten08 and @crysicle
    I'm trying to get this to work, but I don't get any change in the terrain.
    Did i miss something?

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.Experimental.TerrainAPI;
    3.  
    4. public class TerrainModifier : MonoBehaviour
    5. {
    6.     [SerializeField] ComputeShader terrainPaintShader;
    7.     public Texture brushTexture;
    8.     public Vector2 rectSize;
    9.  
    10.     private Rect rect;
    11.     private Terrain terrain;
    12.     private PaintContext paintContext;
    13.  
    14.     void Start()
    15.     {
    16.         terrain = GetComponent<Terrain>();
    17.     }
    18.  
    19.     void Update()
    20.     {
    21.         if (Input.GetMouseButton(0))
    22.         {
    23.             RaycastHit rayHit;
    24.             var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    25.  
    26.             if (Physics.Raycast(ray, out rayHit, Mathf.Infinity))
    27.             {
    28.                 Vector2 terrainPoint = new Vector2(rayHit.point.x, rayHit.point.z);
    29.                 rect = new Rect(terrainPoint, rectSize);
    30.                 ModifyTerrain(rect);
    31.                 Debug.DrawRay(ray.origin, ray.direction * 500, Color.red);
    32.             }
    33.         }
    34.     }
    35.  
    36.     void ModifyTerrain(Rect selection)
    37.     {
    38.         paintContext = TerrainPaintUtility.BeginPaintHeightmap(terrain, selection);
    39.  
    40.         RenderTexture terrainRenderTexture = new RenderTexture(paintContext.sourceRenderTexture.width, paintContext.sourceRenderTexture.height, 0, RenderTextureFormat.R16);
    41.         terrainRenderTexture.enableRandomWrite = true;
    42.         Graphics.CopyTexture(paintContext.sourceRenderTexture, terrainRenderTexture);
    43.  
    44.         terrainPaintShader.SetTexture(terrainPaintShader.FindKernel("CSMain"), "originalHeightmap", terrainRenderTexture);
    45.         terrainPaintShader.SetTexture(terrainPaintShader.FindKernel("CSMain"), "brush", brushTexture);
    46.         terrainPaintShader.Dispatch(terrainPaintShader.FindKernel("CSMain"), 1, 1, 1);
    47.  
    48.         Graphics.CopyTexture(terrainRenderTexture, paintContext.destinationRenderTexture);
    49.  
    50.         TerrainPaintUtility.EndPaintHeightmap(paintContext, "Terrain");
    51.  
    52.         terrain.terrainData.SyncHeightmap();
    53.  
    54.     }
    55. }
    56.  
    ComputeShader:

    Code (CSharp):
    1. // Each #kernel tells which function to compile; you can have many kernels
    2. #pragma kernel CSMain
    3.  
    4. // Create a RenderTexture with enableRandomWrite flag and set it
    5. // with cs.SetTexture
    6. RWTexture2D<half> originalHeightmap;
    7. Texture2D<half4> brush;
    8. half height_multiplier = 0.01;
    9.  
    10. [numthreads(8, 8, 1)]
    11. void CSMain(uint3 id : SV_DispatchThreadID)
    12. {
    13.     half newHeightmap = originalHeightmap[id.xy].x + brush[id.xy].w * height_multiplier;
    14.     originalHeightmap[id.xy].x = clamp(newHeightmap, 0, 0.5);
    15. }
    16.  
    Terrain Inspector:
    upload_2019-11-6_11-57-33.png

    Also set the terrain tools to paint and Raise/Lower don't know if that has any effect, but i did notice that we can paint displacement during run time in the sceneview which effects the terrain in the gameview, without any lag so that's a good sign.
     
  9. crysicle

    crysicle

    Joined:
    Oct 24, 2018
    Posts:
    78
    1. The amount of heightmaps you're manipulating is not 100x100, but 8x8. That's because the numthreads in the shader is [8,8,1] and you're dispatching the shader with shader.dispatch(kernel, 1, 1, 1) which simply multiplies the numthreads. To actually manipulate 100x100, you'll need to divide the size of the rect by the number of numthreads for each axis instead of just passing 1 for each of them.

    Should look something like this:
    shader.dispatch(kernel, Mathf.Ceilling(rect.width / numthreads.x), Mathf.Ceilling(rect.height / numthreads.y), 1)

    2. The starting point of sampling from the paint texture inside the shader is in the bottom corner because we're not adding any offset. All of the pixels then get accessed by the manipulation size, in this case 8x8, because id always starts at 0 and goes from 0 to numthread amount for each axis when a shader is dispatched.

    Here's an image of what's going on - https://i.imgur.com/ZMHuWgZ.png
    Note that inside the shader we're not giving any offset yet, but if we did, it's how it would sample from the brush texture.

    Based on your TerrainBrush in the editor it looks like the beginning 8x8 pixels of that texture are fully transparent, meaning we're adding 0 * height_multiplier to the heightmap. I'm not sure if you tried using other textures, but a fully white texture should probably change an 8x8 grid in some way. Because the w coordinate of the texture is used, its possible your textures completely lack transparency, the shader might be defaulting to 0 just because the transparency layer doesn't exist for the texture that is used. You could also try using any of the other channels for sampling.

    I'd suggest to also add an inspector variable to your script for the height_multiplier which would equal 1 and send it to the shader. It will be easier to debug by having control over all shader variable values inside the inspector.
     
    jister likes this.
  10. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,685
    @crysicle Awesome, your explanation makes total sense! Thanks! it even gave me a better insight on how Compute shaders work.