Adding instancing to a custom terrain shader

Hello, I'm looking for some pointers on how to add instancing to my custom terrain shader. It was built a while ago, before the SRP was introduced. I've had a look through the TerrainLit shader of the HDRP but it's kind of hard to follow with everything buried inside multiple include files.

I know that previously, just adding the multi_compile_instancing pragma and setting up the instance ID was enough to get something working, with UnityInstancing.cginc doing most of the work, but I noticed that the new TerrainLit shader isn't using any of the old .cginc files. I tried including "Packages/com.unity.render-pipelines.core/ShaderLibrary/UnityInstancing.hlsl" but that doesn't seem to be enough either.

My game already has a fairly complex custom rendering system which is totally broken by the HDRP. If I'm not actually using the HDRP but I have the package installed, can I still make use of it's library of .hlsl helpers and get this working?

 

After much fiddling I actually did get it working. I eventually tracked down the code that unpacks the per-patch data in the TerrainLit shader to TerrainLitDataMeshModification.hlsl and TerrainLitSplatCommon.hlsl in the HDRP package.

I took the ApplyMeshModification method from there and reworked it to take an appdata_base input. For reference, the final thing looks something like this (this isn't the complete shader, just the important bits):

Code (CSharp):

1.  
2.             #pragma multi_compile_instancing
3.             #pragma instancing_options assumeuniformscaling nomatrices nolightprobe nolightmap
4.             #include "UnityCG.cginc"
5.        
6.             sampler2D _TerrainHeightmapTexture;
7.             sampler2D _TerrainNormalmapTexture;
8.        
9.             float4 _TerrainHeightmapRecipSize;   // float4(1.0f/width, 1.0f/height, 1.0f/(width-1), 1.0f/(height-1))
10.             float4 _TerrainHeightmapScale;       // float4(hmScale.x, hmScale.y / (float)(kMaxHeight), hmScale.z, 0.0f)
11.        
12.             UNITY_INSTANCING_BUFFER_START(Terrain)
13.                 UNITY_DEFINE_INSTANCED_PROP(float4, _TerrainPatchInstanceData)  // float4(xBase, yBase, skipScale, ~)
14.             UNITY_INSTANCING_BUFFER_END(Terrain)
15.        
16.             appdata_base ApplyMeshModification(appdata_base input)
17.             {
18.             #ifdef UNITY_INSTANCING_ENABLED
19.                 float2 patchVertex = input.vertex.xy;
20.                 float4 instanceData = UNITY_ACCESS_INSTANCED_PROP(Terrain, _TerrainPatchInstanceData);
21.            
22.                 float2 sampleCoords = (patchVertex.xy + instanceData.xy) * instanceData.z; // (xy + float2(xBase,yBase)) * skipScale
23.                 input.texcoord = float4(sampleCoords.xy * _TerrainHeightmapRecipSize.z, 0, 0);
24.                 float height = UnpackHeightmap(tex2Dlod(_TerrainHeightmapTexture, input.texcoord));
25.        
26.                 input.vertex.xz = sampleCoords * _TerrainHeightmapScale.xz;
27.                 input.vertex.y = height * _TerrainHeightmapScale.y;
28.        
29.                 input.normal= tex2Dlod(_TerrainNormalmapTexture, input.texcoord).rgb * 2 - 1;
30.             #endif
31.        
32.                 return input;
33.             }
34.        
35.             v2f vert (appdata_base v)
36.             {
37.                 UNITY_SETUP_INSTANCE_ID(v);
38.                 v = ApplyMeshModification(v);
39.                // rest of the shader continues from here...
40.             }

The important things to note:

• If your shader has a _TerrainHeightmapTexture property, Unity will pass in the actual heightmap for the terrain to it. I don't know if this has always been the case or if that's new functionality in 2018.3 but it's pretty useful (the same goes for _TerrainNormalMapTexture and the normal map).
• _TerrainHeightmapRecipSize and _TerrainHeightmapScale contain some information about the scale of the terrain that, again, Unity passes in for you. Stuff like heightmap resolution and max terrain height.
• The ApplyMeshModification method gets called from the shader's vert function, and we need to read the _TerrainHeightmapTexture. the tex2D method of doing a texture lookup only works from the frag function, so we have to use tex2Dlod instead.

 

Hello, I have an update on this, as well as a new problem. I tidied this instancing code up into a .cginc file that very closely follows the SplatmapVert function in TerrainSplatmapCommon, but just takes one appdata_base parameter, making it usable in a vert shader. It looks like this:

Code (CSharp):

1. #ifndef CUSTOM_TERRAIN_INSTANCING
2. #define CUSTOM_TERRAIN_INSTANCING
3.  
4. #ifdef UNITY_INSTANCING_ENABLED
5.     sampler2D _TerrainHeightmapTexture;
6.     sampler2D _TerrainNormalmapTexture;
7.  
8.     float4 _TerrainHeightmapRecipSize;   // float4(1.0f/width, 1.0f/height, 1.0f/(width-1), 1.0f/(height-1))
9.     float4 _TerrainHeightmapScale;       // float4(hmScale.x, hmScale.y / (float)(kMaxHeight), hmScale.z, 0.0f)
10.  
11.     UNITY_INSTANCING_BUFFER_START(Terrain)
12.         UNITY_DEFINE_INSTANCED_PROP(float4, _TerrainPatchInstanceData)  // float4(xBase, yBase, skipScale, ~)
13.     UNITY_INSTANCING_BUFFER_END(Terrain)
14. #endif
15.  
16. void ApplyMeshModification(inout appdata_base v)
17. {
18. #ifdef UNITY_INSTANCING_ENABLED
19.     float2 patchVertex = v.vertex.xy;
20.     float4 instanceData = UNITY_ACCESS_INSTANCED_PROP(Terrain, _TerrainPatchInstanceData);
21.  
22.     float4 uvscale = instanceData.z * _TerrainHeightmapRecipSize;
23.     float4 uvoffset = instanceData.xyxy * uvscale;
24.     uvoffset.xy += 0.5f * _TerrainHeightmapRecipSize.xy;
25.     float2 sampleCoords = (patchVertex.xy * uvscale.xy + uvoffset.xy);
26.  
27.     float hm = UnpackHeightmap(tex2Dlod(_TerrainHeightmapTexture, float4(sampleCoords, 0, 0)));
28.     v.vertex.xz = (patchVertex.xy + instanceData.xy) * _TerrainHeightmapScale.xz * instanceData.z;  //(x + xBase) * hmScale.x * skipScale;
29.     v.vertex.y = hm * _TerrainHeightmapScale.y;
30.     v.vertex.w = 1.0f;
31.  
32.     v.texcoord.xy = (patchVertex.xy * uvscale.zw + uvoffset.zw);
33.  
34.     v.normal = tex2Dlod(_TerrainNormalmapTexture, float4(sampleCoords, 0, 0)).xyz * 2 - 1;
35. #endif
36. }
37.  
38. #endif // CUSTOM_TERRAIN_INSTANCING

It works perfectly in the editor, and it does work in standalone Mac and Windows builds except for one thing - _TerrainNormalmapTexture is not being passed into the shader, so it ends up using the default grey texture instead. This doesn't stop it from rendering, but it does interfere with my lighting calculations and also results in artifacts in the shadowmap since the Shadowcaster pass uses normals for biasing to prevent shadow acne.

Have I done something wrong here, or is this a bug with the renderer? I'm making builds with Unity 2018.3.0f1.

EDIT:
I should also mention that I'm using the procedural generation tool MapMagic to calculate and set the heightmap data at runtime. Might this have something to do with the problem?

 

hello, I have faced the same problem. I wrote my own terrain shader, but not works in Draw Instancing mode.
I really have no idea where should I start to do. can you give me some threads? The mechanism or theory like that. Thanks!

 

Hey, I'm trying to implement terrain GPU Instancing in URP Shadergraph for a custom unlit shader.

I've implemented GPU instancing in shadergraph for foilage and normal objects before, but the terrain is giving me a bit of a headache.

I've copied the above code for use in shader graph,

Code (CSharp):

1. #ifndef CUSTOM_TERRAIN_INSTANCING
2. #define CUSTOM_TERRAIN_INSTANCING
3. #ifdef UNITY_INSTANCING_ENABLED
4.     sampler2D _TerrainHeightmapTexture;
5.     sampler2D _TerrainNormalmapTexture;
6.     float4 _TerrainHeightmapRecipSize;   // float4(1.0f/width, 1.0f/height, 1.0f/(width-1), 1.0f/(height-1))
7.     float4 _TerrainHeightmapScale;       // float4(hmScale.x, hmScale.y / (float)(kMaxHeight), hmScale.z, 0.0f)
8.     UNITY_INSTANCING_BUFFER_START(Terrain)
9.         UNITY_DEFINE_INSTANCED_PROP(float4, _TerrainPatchInstanceData)  // float4(xBase, yBase, skipScale, ~)
10.     UNITY_INSTANCING_BUFFER_END(Terrain)
11. #endif
12. void setupScale()
13. {
14. }
15.  
16. void ApplyMeshModification_float( float2 vertXY, out float3 vertex, out float2 Newtexcoord, out float3 normal )
17. {
18. #ifdef SHADERGRAPH_PREVIEW
19.     Newtexcoord=float2(0,0);
20.     normal=float3(0,0,0);
21.     vertex=float3(0,0,0);
22. #else
23. #ifdef UNITY_INSTANCING_ENABLED
24.     float2 patchVertex = vertXY;
25.     float4 instanceData = UNITY_ACCESS_INSTANCED_PROP(Terrain, _TerrainPatchInstanceData);
26.     float4 uvscale = instanceData.z * _TerrainHeightmapRecipSize;
27.     float4 uvoffset = instanceData.xyxy * uvscale;
28.     uvoffset.xy += 0.5f * _TerrainHeightmapRecipSize.xy;
29.     float2 sampleCoords = (patchVertex.xy * uvscale.xy + uvoffset.xy);
30.     float hm = UnpackHeightmap(tex2Dlod(_TerrainHeightmapTexture, float4(sampleCoords, 0, 0)));
31.     vertex.xz = (patchVertex.xy + instanceData.xy) * _TerrainHeightmapScale.xz * instanceData.z;  //(x + xBase) * hmScale.x * skipScale;
32.     vertex.y = hm * _TerrainHeightmapScale.y;
33.     Newtexcoord.xy = (patchVertex.xy * uvscale.zw + uvoffset.zw);
34.     normal = tex2Dlod(_TerrainNormalmapTexture, float4(sampleCoords, 0, 0)).xyz * 2 - 1;
35. #else
36.     Newtexcoord = float2(0, 0);
37.     normal = float3(0, 0, 0);
38.     vertex = float3(0, 0, 0);
39. #endif
40. #endif
41. }
42. #endif // CUSTOM_TERRAIN_INSTANCING

plugged it into to vertStage as follows:



and I'm getting the following error stack:



thoughts / tip / opinions on where to go from here?

 

Oh..yay...It's working. Sort of, with some trouble:

I've nicked a pragma directive from the URP terrain shader, and it did the job:



But...My DepthNormals texture (which I kinda need), is not being rendered. Can I get my depthnormals texture when instancing? I have a feeling it's something to do with the renderinglayer.
The original instancing option is norenderinglayer, but that seemed even worse?

EDIT: Bonus question, can I pass the new UV to the texcoord value in shadergraph, instead of using a custom interpolator?