[Tutorial] Adding a vertex shader to Lit

The tutorial is a few posts down in this thread:

https://forum.unity.com/threads/tutorial-adding-a-vertex-shader-to-lit.1037011/#post-6717088

---

Original "Help Wanted" post:

---

I'd like to add a custom vertex shader to the HDRP Lit shader.

My failed attempt:

I tried using the Lit base of Shader Graph to replace Lit, but the fragment shader of this base appears to be different than HDRP's Lit shader, at least in terms of default parameter values and the Inspector GUI. Someone else tried to recreate Lit in Shader Graph and also failed (forum thread link).



My second failed attempt:

I tried to copy-paste my vertex shader from the Shader Graph generated code into a duplicate of Lit.shader (found in PackageCache/{HDRP package}/Runtime/Material/Lit). Managing all the include statements, define statements, and other statements for each of the 14 passes overwhelmed me and I decided to make this thread. Also, the Lit shader has 14 passes in the first subshader while the Shader Graph Lit base has only 11. These differences added up and overwhelmed me.

-->

A third idea:

Use a compute shader to modify the vertices and send the results to the render pipeline directly instead of having to send them back to the CPU-side (into mesh.vertices) just to send them right back to the GPU-side.

Edit: It seems this isn't possible yet, at least not through GrapicsBuffer, because GraphicsBuffer.Target.Vertex isn't available yet. See this thread.

A fourth idea:

Try idea #2 again, but with more mental toughness (copy-pasting Shader Graph code to Lit.shader).

 

A fifth idea:

Inject the vertex shader code into the render pipeline in general and use #ifdef statements to activate it. This idea is based on this post. The post mentions modifying Runtime\RenderPipeline\ShaderPass\VertMesh.hlsl, but as you can see by the path it's not used exclusively by the Lit shader, which is where I think the #ifdef statements could be helpful. (Or maybe the person in the post forked the file and included it via #include statements) (Thanks for the idea @cubrman)

 

I copied all the files, renamed the shader, changed the includes to reference my copied files and modified the copied files.

 

Resolved!

Disclaimer: I'm not a game dev expert nor a Unity expert. This worked for my use case but may not work for yours.

Here are the steps I used to inject a vertex shader into the HDRP Lit shader:

1. Duplicate Lit.shader

Duplicate Lit.shader from the HDRP package cache, store it in your project, and rename it to something, e.g. "CustomLit.shader".

2. Duplicate LitProperties.hlsl

Duplicate LitProperties.shader from the HDRP package cache, store it in your project, and rename it to something, e.g. "CustomLitProperties.hlsl".

3. Add your custom properties

If you need to add a float, find a float that already exists and follow it through CustomLit.shader and CustomLitProperties.hlsl, adding lines with your custom float wherever the original float appears. If you need to add a Texture2D, find a Texture2D and follow it, and so on.

Note: I ignored the DOTS section of LitProperties.hlsl because I'm not using the DOTS HybridRenderer.

4. Decide which data you want from the mesh

Position is always provided. To declare which data you want, include any of the following define statements in CustomLit.shader. I put mine in the "Define" section which applies to all the passes of the shader, but you can define them on a pass-by-pass basis if you'd like.

Spoiler: Data options

#define ATTRIBUTES_NEED_NORMAL
#define ATTRIBUTES_NEED_TANGENT
#define ATTRIBUTES_NEED_TEXCOORD0
#define ATTRIBUTES_NEED_TEXCOORD1
#define ATTRIBUTES_NEED_TEXCOORD2
#define ATTRIBUTES_NEED_TEXCOORD3
#define ATTRIBUTES_NEED_COLOR

5. Define the ApplyMeshModification function

I did so in a new .hlsl file named "ApplyMeshModification.hlsl" so I could include it in each pass of the shader.

The following function will raise the mesh by 0.5 units.

Note: You must declare "#define HAVE_MESH_MODIFICATION" or the render pipeline won't apply this function to your mesh data.

Code (CSharp):

1. #define HAVE_MESH_MODIFICATION
2.  
3. AttributesMesh ApplyMeshModification(AttributesMesh input, float3 timeParameters)
4. {
5.     input.positionOS += float3(0.0f, 0.5f, 0.0f);
6.     return input;
7. }

Spoiler: AttributesMesh definition

Code (CSharp):

1.  
2. // Defined in com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/ShaderPass/VaryingMesh.hlsl
3. struct AttributesMesh
4. {
5.     float3 positionOS   : POSITION;
6. #ifdef ATTRIBUTES_NEED_NORMAL
7.     float3 normalOS     : NORMAL;
8. #endif
9. #ifdef ATTRIBUTES_NEED_TANGENT
10.     float4 tangentOS    : TANGENT; // Store sign in w
11. #endif
12. #ifdef ATTRIBUTES_NEED_TEXCOORD0
13.     float2 uv0          : TEXCOORD0;
14. #endif
15. #ifdef ATTRIBUTES_NEED_TEXCOORD1
16.     float2 uv1          : TEXCOORD1;
17. #endif
18. #ifdef ATTRIBUTES_NEED_TEXCOORD2
19.     float2 uv2          : TEXCOORD2;
20. #endif
21. #ifdef ATTRIBUTES_NEED_TEXCOORD3
22.     float2 uv3          : TEXCOORD3;
23. #endif
24. #ifdef ATTRIBUTES_NEED_COLOR
25.     float4 color        : COLOR;
26. #endif
27.  
28.     UNITY_VERTEX_INPUT_INSTANCE_ID
29. };

6. Include your new files into your new shader

In CustomLit.shader:

Replace LitProperties.hlsl with CustomLitProperties.hlsl:

Code (JavaScript):

1. // #include "Packages/com.unity.render-pipelines.high-definition/Runtime/Material/Lit/LitProperties.hlsl"
2. #include "Assets/CustomLit/CustomLitProperties.hlsl"

And include ApplyMeshModification.hlsl in each of the passes and immediately before the include statement for the main HLSL file for the pass (which is the last include statement for each pass):

#include "Assets/CustomLit/ApplyMeshModification.hlsl"

Here are some screenshot examples:

Forward pass:



Depth only pass:



There are 14 passes, but once adding this include statement to each pass you can edit ApplyMeshModification.hlsl to affect each pass.

Note: I ignored the second subshader, which contains passes for DirectX Raytracing (DXR).

 

Works for HDRP v.13.x