Search Unity

Terrain Adding instancing to a custom terrain shader

Discussion in 'World Building' started by shedworksdigital, Nov 7, 2018.

  1. shedworksdigital

    shedworksdigital

    Joined:
    Nov 7, 2016
    Posts:
    40
    Hello, I'm looking for some pointers on how to add instancing to my custom terrain shader. It was built a while ago, before the SRP was introduced. I've had a look through the TerrainLit shader of the HDRP but it's kind of hard to follow with everything buried inside multiple include files.

    I know that previously, just adding the multi_compile_instancing pragma and setting up the instance ID was enough to get something working, with UnityInstancing.cginc doing most of the work, but I noticed that the new TerrainLit shader isn't using any of the old .cginc files. I tried including "Packages/com.unity.render-pipelines.core/ShaderLibrary/UnityInstancing.hlsl" but that doesn't seem to be enough either.

    My game already has a fairly complex custom rendering system which is totally broken by the HDRP. If I'm not actually using the HDRP but I have the package installed, can I still make use of it's library of .hlsl helpers and get this working?
     
  2. zeroyao

    zeroyao

    Unity Technologies

    Joined:
    Mar 28, 2013
    Posts:
    166
    Hi there!

    From your post it seems that you are not using HDRP already. That probably means you are either using surface shader or custom shader targeting one of the built-in pipelines:

    - For surface shader, it is mostly enough to just have "#pragma multi_compile_instancing" line to enable instancing. All the work is done in the vertex modifier SplatmapVert and the common utility function SplatmapMix.

    - For custom vert/frag shader, you can again enable instancing, and take a reference to the above mentioned functions to know how to implement instancing.

    The source code for reference is in TerrainSplatmapCommon.cginc and Standard-FirstPass.shader. If you don't have them already, you should be able to find them in the download: http://beta.unity3d.com/download/a3...32.959193146.1541553286-1704479374.1533084849

    Cheers!
     
  3. shedworksdigital

    shedworksdigital

    Joined:
    Nov 7, 2016
    Posts:
    40
    After much fiddling I actually did get it working. I eventually tracked down the code that unpacks the per-patch data in the TerrainLit shader to TerrainLitDataMeshModification.hlsl and TerrainLitSplatCommon.hlsl in the HDRP package.

    I took the ApplyMeshModification method from there and reworked it to take an appdata_base input. For reference, the final thing looks something like this (this isn't the complete shader, just the important bits):

    Code (CSharp):
    1.  
    2.             #pragma multi_compile_instancing
    3.             #pragma instancing_options assumeuniformscaling nomatrices nolightprobe nolightmap
    4.             #include "UnityCG.cginc"
    5.        
    6.             sampler2D _TerrainHeightmapTexture;
    7.             sampler2D _TerrainNormalmapTexture;
    8.        
    9.             float4 _TerrainHeightmapRecipSize;   // float4(1.0f/width, 1.0f/height, 1.0f/(width-1), 1.0f/(height-1))
    10.             float4 _TerrainHeightmapScale;       // float4(hmScale.x, hmScale.y / (float)(kMaxHeight), hmScale.z, 0.0f)
    11.        
    12.             UNITY_INSTANCING_BUFFER_START(Terrain)
    13.                 UNITY_DEFINE_INSTANCED_PROP(float4, _TerrainPatchInstanceData)  // float4(xBase, yBase, skipScale, ~)
    14.             UNITY_INSTANCING_BUFFER_END(Terrain)
    15.        
    16.             appdata_base ApplyMeshModification(appdata_base input)
    17.             {
    18.             #ifdef UNITY_INSTANCING_ENABLED
    19.                 float2 patchVertex = input.vertex.xy;
    20.                 float4 instanceData = UNITY_ACCESS_INSTANCED_PROP(Terrain, _TerrainPatchInstanceData);
    21.            
    22.                 float2 sampleCoords = (patchVertex.xy + instanceData.xy) * instanceData.z; // (xy + float2(xBase,yBase)) * skipScale
    23.                 input.texcoord = float4(sampleCoords.xy * _TerrainHeightmapRecipSize.z, 0, 0);
    24.                 float height = UnpackHeightmap(tex2Dlod(_TerrainHeightmapTexture, input.texcoord));
    25.        
    26.                 input.vertex.xz = sampleCoords * _TerrainHeightmapScale.xz;
    27.                 input.vertex.y = height * _TerrainHeightmapScale.y;
    28.        
    29.                 input.normal= tex2Dlod(_TerrainNormalmapTexture, input.texcoord).rgb * 2 - 1;
    30.             #endif
    31.        
    32.                 return input;
    33.             }
    34.        
    35.             v2f vert (appdata_base v)
    36.             {
    37.                 UNITY_SETUP_INSTANCE_ID(v);
    38.                 v = ApplyMeshModification(v);
    39.                // rest of the shader continues from here...
    40.             }
    The important things to note:
    • If your shader has a _TerrainHeightmapTexture property, Unity will pass in the actual heightmap for the terrain to it. I don't know if this has always been the case or if that's new functionality in 2018.3 but it's pretty useful (the same goes for _TerrainNormalMapTexture and the normal map).
    • _TerrainHeightmapRecipSize and _TerrainHeightmapScale contain some information about the scale of the terrain that, again, Unity passes in for you. Stuff like heightmap resolution and max terrain height.
    • The ApplyMeshModification method gets called from the shader's vert function, and we need to read the _TerrainHeightmapTexture. the tex2D method of doing a texture lookup only works from the frag function, so we have to use tex2Dlod instead.
     
    Last edited: Nov 9, 2018
    mitaywalle likes this.
  4. shedworksdigital

    shedworksdigital

    Joined:
    Nov 7, 2016
    Posts:
    40
    Hello, I have an update on this, as well as a new problem. I tidied this instancing code up into a .cginc file that very closely follows the SplatmapVert function in TerrainSplatmapCommon, but just takes one appdata_base parameter, making it usable in a vert shader. It looks like this:

    Code (CSharp):
    1. #ifndef CUSTOM_TERRAIN_INSTANCING
    2. #define CUSTOM_TERRAIN_INSTANCING
    3.  
    4. #ifdef UNITY_INSTANCING_ENABLED
    5.     sampler2D _TerrainHeightmapTexture;
    6.     sampler2D _TerrainNormalmapTexture;
    7.  
    8.     float4 _TerrainHeightmapRecipSize;   // float4(1.0f/width, 1.0f/height, 1.0f/(width-1), 1.0f/(height-1))
    9.     float4 _TerrainHeightmapScale;       // float4(hmScale.x, hmScale.y / (float)(kMaxHeight), hmScale.z, 0.0f)
    10.  
    11.     UNITY_INSTANCING_BUFFER_START(Terrain)
    12.         UNITY_DEFINE_INSTANCED_PROP(float4, _TerrainPatchInstanceData)  // float4(xBase, yBase, skipScale, ~)
    13.     UNITY_INSTANCING_BUFFER_END(Terrain)
    14. #endif
    15.  
    16. void ApplyMeshModification(inout appdata_base v)
    17. {
    18. #ifdef UNITY_INSTANCING_ENABLED
    19.     float2 patchVertex = v.vertex.xy;
    20.     float4 instanceData = UNITY_ACCESS_INSTANCED_PROP(Terrain, _TerrainPatchInstanceData);
    21.  
    22.     float4 uvscale = instanceData.z * _TerrainHeightmapRecipSize;
    23.     float4 uvoffset = instanceData.xyxy * uvscale;
    24.     uvoffset.xy += 0.5f * _TerrainHeightmapRecipSize.xy;
    25.     float2 sampleCoords = (patchVertex.xy * uvscale.xy + uvoffset.xy);
    26.  
    27.     float hm = UnpackHeightmap(tex2Dlod(_TerrainHeightmapTexture, float4(sampleCoords, 0, 0)));
    28.     v.vertex.xz = (patchVertex.xy + instanceData.xy) * _TerrainHeightmapScale.xz * instanceData.z;  //(x + xBase) * hmScale.x * skipScale;
    29.     v.vertex.y = hm * _TerrainHeightmapScale.y;
    30.     v.vertex.w = 1.0f;
    31.  
    32.     v.texcoord.xy = (patchVertex.xy * uvscale.zw + uvoffset.zw);
    33.  
    34.     v.normal = tex2Dlod(_TerrainNormalmapTexture, float4(sampleCoords, 0, 0)).xyz * 2 - 1;
    35. #endif
    36. }
    37.  
    38. #endif // CUSTOM_TERRAIN_INSTANCING
    It works perfectly in the editor, and it does work in standalone Mac and Windows builds except for one thing - _TerrainNormalmapTexture is not being passed into the shader, so it ends up using the default grey texture instead. This doesn't stop it from rendering, but it does interfere with my lighting calculations and also results in artifacts in the shadowmap since the Shadowcaster pass uses normals for biasing to prevent shadow acne.

    Have I done something wrong here, or is this a bug with the renderer? I'm making builds with Unity 2018.3.0f1.

    EDIT:
    I should also mention that I'm using the procedural generation tool MapMagic to calculate and set the heightmap data at runtime. Might this have something to do with the problem?
     
    Last edited: Feb 25, 2019
    smonchdev0 likes this.
unityunity