Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

[Tutorial] Adding a vertex shader to Lit

Discussion in 'High Definition Render Pipeline' started by GameDeveloper1111, Jan 11, 2021.

  1. GameDeveloper1111

    GameDeveloper1111

    Joined:
    Jul 24, 2020
    Posts:
    100
    The tutorial is a few posts down in this thread:

    https://forum.unity.com/threads/tutorial-adding-a-vertex-shader-to-lit.1037011/#post-6717088

    ---

    Original "Help Wanted" post:

    ---

    I'd like to add a custom vertex shader to the HDRP Lit shader.

    My failed attempt:

    I tried using the Lit base of Shader Graph to replace Lit, but the fragment shader of this base appears to be different than HDRP's Lit shader, at least in terms of default parameter values and the Inspector GUI. Someone else tried to recreate Lit in Shader Graph and also failed (forum thread link).

    upload_2021-1-11_10-51-24.png

    My second failed attempt:

    I tried to copy-paste my vertex shader from the Shader Graph generated code into a duplicate of Lit.shader (found in PackageCache/{HDRP package}/Runtime/Material/Lit). Managing all the include statements, define statements, and other statements for each of the 14 passes overwhelmed me and I decided to make this thread. Also, the Lit shader has 14 passes in the first subshader while the Shader Graph Lit base has only 11. These differences added up and overwhelmed me.

    upload_2021-1-11_10-53-22.png --> upload_2021-1-11_10-52-45.png

    A third idea:

    Use a compute shader to modify the vertices and send the results to the render pipeline directly instead of having to send them back to the CPU-side (into mesh.vertices) just to send them right back to the GPU-side.

    Edit: It seems this isn't possible yet, at least not through GrapicsBuffer, because GraphicsBuffer.Target.Vertex isn't available yet. See this thread.

    A fourth idea:

    Try idea #2 again, but with more mental toughness (copy-pasting Shader Graph code to Lit.shader).
     
    Last edited: Jan 12, 2021
  2. GameDeveloper1111

    GameDeveloper1111

    Joined:
    Jul 24, 2020
    Posts:
    100
    A fifth idea:

    Inject the vertex shader code into the render pipeline in general and use #ifdef statements to activate it. This idea is based on this post. The post mentions modifying Runtime\RenderPipeline\ShaderPass\VertMesh.hlsl, but as you can see by the path it's not used exclusively by the Lit shader, which is where I think the #ifdef statements could be helpful. (Or maybe the person in the post forked the file and included it via #include statements) (Thanks for the idea @cubrman)
     
    Last edited: Jan 11, 2021
  3. cubrman

    cubrman

    Joined:
    Jun 18, 2016
    Posts:
    361
    I copied all the files, renamed the shader, changed the includes to reference my copied files and modified the copied files.
     
    GameDeveloper1111 likes this.
  4. GameDeveloper1111

    GameDeveloper1111

    Joined:
    Jul 24, 2020
    Posts:
    100
    Resolved!

    Disclaimer: I'm not a game dev expert nor a Unity expert. This worked for my use case but may not work for yours.

    Here are the steps I used to inject a vertex shader into the HDRP Lit shader:

    1. Duplicate Lit.shader

    Duplicate Lit.shader from the HDRP package cache, store it in your project, and rename it to something, e.g. "CustomLit.shader".

    2. Duplicate LitProperties.hlsl

    Duplicate LitProperties.shader from the HDRP package cache, store it in your project, and rename it to something, e.g. "CustomLitProperties.hlsl".

    3. Add your custom properties

    If you need to add a float, find a float that already exists and follow it through CustomLit.shader and CustomLitProperties.hlsl, adding lines with your custom float wherever the original float appears. If you need to add a Texture2D, find a Texture2D and follow it, and so on.

    Note: I ignored the DOTS section of LitProperties.hlsl because I'm not using the DOTS HybridRenderer.

    4. Decide which data you want from the mesh


    Position is always provided. To declare which data you want, include any of the following define statements in CustomLit.shader. I put mine in the "Define" section which applies to all the passes of the shader, but you can define them on a pass-by-pass basis if you'd like.

    #define ATTRIBUTES_NEED_NORMAL
    #define ATTRIBUTES_NEED_TANGENT
    #define ATTRIBUTES_NEED_TEXCOORD0
    #define ATTRIBUTES_NEED_TEXCOORD1
    #define ATTRIBUTES_NEED_TEXCOORD2
    #define ATTRIBUTES_NEED_TEXCOORD3
    #define ATTRIBUTES_NEED_COLOR

    5. Define the ApplyMeshModification function

    I did so in a new .hlsl file named "ApplyMeshModification.hlsl" so I could include it in each pass of the shader.

    The following function will raise the mesh by 0.5 units.

    Note: You must declare "#define HAVE_MESH_MODIFICATION" or the render pipeline won't apply this function to your mesh data.

    Code (CSharp):
    1. #define HAVE_MESH_MODIFICATION
    2.  
    3. AttributesMesh ApplyMeshModification(AttributesMesh input, float3 timeParameters)
    4. {
    5.     input.positionOS += float3(0.0f, 0.5f, 0.0f);
    6.     return input;
    7. }
    Code (CSharp):
    1.  
    2. // Defined in com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/ShaderPass/VaryingMesh.hlsl
    3. struct AttributesMesh
    4. {
    5.     float3 positionOS   : POSITION;
    6. #ifdef ATTRIBUTES_NEED_NORMAL
    7.     float3 normalOS     : NORMAL;
    8. #endif
    9. #ifdef ATTRIBUTES_NEED_TANGENT
    10.     float4 tangentOS    : TANGENT; // Store sign in w
    11. #endif
    12. #ifdef ATTRIBUTES_NEED_TEXCOORD0
    13.     float2 uv0          : TEXCOORD0;
    14. #endif
    15. #ifdef ATTRIBUTES_NEED_TEXCOORD1
    16.     float2 uv1          : TEXCOORD1;
    17. #endif
    18. #ifdef ATTRIBUTES_NEED_TEXCOORD2
    19.     float2 uv2          : TEXCOORD2;
    20. #endif
    21. #ifdef ATTRIBUTES_NEED_TEXCOORD3
    22.     float2 uv3          : TEXCOORD3;
    23. #endif
    24. #ifdef ATTRIBUTES_NEED_COLOR
    25.     float4 color        : COLOR;
    26. #endif
    27.  
    28.     UNITY_VERTEX_INPUT_INSTANCE_ID
    29. };

    6. Include your new files into your new shader

    In CustomLit.shader:

    Replace LitProperties.hlsl with CustomLitProperties.hlsl:

    Code (JavaScript):
    1. // #include "Packages/com.unity.render-pipelines.high-definition/Runtime/Material/Lit/LitProperties.hlsl"
    2. #include "Assets/CustomLit/CustomLitProperties.hlsl"
    And include ApplyMeshModification.hlsl in each of the passes and immediately before the include statement for the main HLSL file for the pass (which is the last include statement for each pass):

    #include "Assets/CustomLit/ApplyMeshModification.hlsl"


    Here are some screenshot examples:

    Forward pass:

    upload_2021-1-12_11-18-5.png

    Depth only pass:

    upload_2021-1-12_11-18-30.png

    There are 14 passes, but once adding this include statement to each pass you can edit ApplyMeshModification.hlsl to affect each pass.

    Note: I ignored the second subshader, which contains passes for DirectX Raytracing (DXR).
     
    Last edited: Jan 12, 2021
  5. tomekkie2

    tomekkie2

    Joined:
    Jul 6, 2012
    Posts:
    890
    Works for HDRP v.13.x
     
    Last edited: Jul 28, 2022