Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Tutorial: How to dynamically tessellate a planar mesh

Discussion in 'Shaders' started by Mr_Admirals, Dec 28, 2018.

  1. Mr_Admirals


    May 13, 2017

    Hello! I've been hard at work on my snow deformation shader, and this forum has been a huge help to me in getting it working. By far the trickiest part of getting what I have so far was the dynamic tessellation, and with so few tessellation tutorials out there, I figured I'd try and give back to the community by sharing how to dynamically tessellate an object. So, without further ado, let's get into it!

    What is Tessellation?

    Tessellation in relation to graphics, is the process of subdividing shapes into smaller versions of that shape. This can be used to enhance detail through increased geometry, or to make distortions and modifications to the mesh look more appealing or believable. The reason I say shapes, is because DX11 allows for three distinct shapes: triangles, quads, or isolines. For this tutorial, we'll be working with triangles.

    How does the Tessellation Pipeline function?

    The tessellation pipeline is separated into three stages: hull, tessellation, and domain.

    Hull Program

    Invoked once per patch (polygon), the hull stage takes one input and produces two output. Its input are the control points that make up a patch. Control points are essentially the original vertices that define the shape. For the output, it returns the control points as well as calculated tessellation factors. To return the two outputs, the hull program is actually made up of two separate functions that operate in parallel.

    Tessellation Program

    Thankfully, what would be the most challenging part of tessellation is handled for us internally. This stage takes in the control points and tessellation factors produced by the hull program and slices up the patch according to the defined domain (triangle, quad, isoline).

    Domain Program

    The purpose of the domain program is to take the subdivided points the tessellation program has produced, and turn them into vertices that can be rendered. To do this, it takes in the control points from the hull program, its tessellation factors, and, last but not least, the tessellation points.

    Part 0 - Shader Set Up

    We'll start from a standard unlit shader. We're going to leave most of the things the same. So begin by removing all fog related code, and adding in "#pragma target 4.6" with the other "#pragma"'s. While we're at it, change the name of the vertex and fragment programs to "VertexProgram" and "FragmentProgram" and update their pragma's accordingly.

    Part 1 - Vertex and Fragment

    Let's start with the vertex and fragment programs. The tessellation pipeline is actually sandwiched between these two programs, so they mostly stay the same. However, because we end up with more vertices than we started with, the vertex program will need to be called twice. To do this, we'll define two separate vertex programs. First however, let's define our structs. And, for the purpose of this tutorial, we'll keep them as simple as possible.

    Code (csharp):
    1. struct appdata
    2. {
    3.    float4 vertex : POSITION;
    4.    float2 uv : TEXCOORD0;
    5. };
    7. struct v2f
    8. {
    9.    float2 uv : TEXCOORD0;
    10.    float4 vertex : SV_POSITION;
    11. };
    Next up will be the vertex and fragment programs.

    Code (csharp):
    1. v2f VertexProgram (appdata v)
    2. {
    3.    v2f o;
    4.    o.vertex = UnityObjectToClipPos(v.vertex);
    5.    o.uv = TRANSFORM_TEX(v.uv, _MainTex);
    6.    return o;
    7. }
    9. fixed4 FragmentProgram (v2f i) : SV_Target
    10. {
    11.    fixed4 col = tex2D(_MainTex, i.uv);
    12.    return col;
    13. }
    Very basic so far. Now, if you'll recall, I mentioned that we'll need two vertex programs. We'll need to create a new struct, a new function, and change the vert pragma. First, lets change the pragma.

    Code (csharp):
    1. #pragma vert TessellationVertexProgram
    Okay, with that out of the way let's move onto the struct. This struct will represent the control points and will be fed into the hull program. It will be almost identical to appdata.

    Code (csharp):
    1. struct ControlPoint
    2. {
    3.    float4 vertex : INTERNALTESSPOS;
    4.    float2 uv : TEXCOORD0;
    5. };
    Now for our new vertex program. Because we're using this vertex program to pass along the vertices as control points to the hull program, nothing needs to change between the two, so we'll simply copy over their values.

    Code (csharp):
    1. ControlPoint TessellationVertexProgram(appdata v)
    2. {
    3.    ControlPoint p;
    4.    p.vertex = v.vertex;
    5.    p.uv = v.uv;
    6.    return p;
    7. }
    Part 2 - Hull

    So, we have our pre-tessellation vertex program and post-tessellation vertex program set up as well as our fragment program. Let's dive into the nitty-gritty of tessellation! If you recall, I had mentioned that the hull program was split up into two separate functions executed in parallel. The reason for this is because the hull program has two outputs: the control points, and the tessellation factors. Let's get the control point output out of the way.

    Before sharing the code, I want to mention a few things. The hull program takes inputs as a patch. This means multiple control points (vertices). We'll need to define how many control points we're expecting, as well as the domain (triangle, quad, isoline). The last obvious thing we'll need to define, is what the other half of the hull program is called so they both can be executed in parallel. To define all these, we'll use attributes before the function definition.

    Code (csharp):
    1. [UNITY_domain("tri")]
    2. [UNITY_outputcontrolpoints(3)]
    3. [UNITY_outputtopology("triangle_cw")]
    4. [UNITY_partitioning("integer")]
    5. [UNITY_patchconstantfunc("PatchConstantFunction")]
    6. ControlPoint HullProgram(InputPatch<ControlPoint, 3> patch, uint id : SV_OutputControlPointID)
    7. {
    8.    return patch[id];
    9. }
    Several things to note. There are two attributes I did not mention. output topology refers to the winding of the triangles. In our case, because we want the faces to face us, they will wind clockwise. The partitioning refers to how the tessellation factors will be interpreted. In our case, we want the full 1 to 64 range, so we'll partition them as integers. Lastly, because we are only returning one control point at a time, we need to specify which one in the patch we are working with, so we also take in an ID as well as a patch.

    So, now we need to implement our other half of the hull program; this is referred to as the patch constant function. As you'll note, that's also what we named it. As this function will be returning the tessellation factors, let's talk about those. Because we're dealing with triangles we need to define 4 separate factors: one for each edge, and one for the inside. As these numbers vary, so do the patterns and complexity. We will create a struct for the tessellation factors so the tessellation program can get them as a single package.

    Code (csharp):
    1. struct TessellationFactors
    2. {
    3.    float edge[3] : SV_TessFactor;
    4.    float inside : SV_InsideTessFactor;
    5. };
    Before implementing the patch constant function, we should do a few more things to prepare. For the simpleness of this tutorial, we'll be working with uniform tessellation factors. So we'll want a slider so we can control the level of tessellation we're working with. Add this line to the properties section of the shader.

    Code (csharp):
    1. _Uniform ("Uniform Tessellation", Range(1, 64)) = 1
    Next, we want to get set for dynamic tessellation. We'll be dynamically tessellating based off of a texture. But you can alternatively tessellate based off of vertex color, or even vertex position. Anyway, for the texture, we'll need an additional texture slot in the properties.

    Code (csharp):
    1. _TessMap ("Tessellation Map", 2D) = "black" {}
    Let's go ahead and initialize those two properties within the subshader too.

    Code (csharp):
    1. sampler2D _TessMap;
    2. float _Uniform;
    Okay, now we have everything we need to implement the patch constant function! This time I'll post the code first and then explain afterwards. Be sure to put this code before the HullProgram function.

    Code (csharp):
    1. TessellationFactors PatchConstantFunction(InputPatch<ControlPoint, 3> patch)
    2. {
    3.    float p0factor = tex2Dlod(_TessMap, float4(patch[0].uv.x, patch[0].uv.y, 0, 0)).r;
    4.    float p1factor = tex2Dlod(_TessMap, float4(patch[1].uv.x, patch[1].uv.y, 0, 0)).r;
    5.    float p2factor = tex2Dlod(_TessMap, float4(patch[2].uv.x, patch[2].uv.y, 0, 0)).r;
    6.    float factor = (p0factor + p1factor + p2factor);
    7.    TessellationFactors f;
    8.    f.edge[0] = factor > 0.0 ? _Uniform : 1.0;
    9.    f.edge[1] = factor > 0.0 ? _Uniform : 1.0;
    10.    f.edge[2] = factor > 0.0 ? _Uniform : 1.0;
    11.    f.inside = factor > 0.0 ? _Uniform : 1.0;
    12.    return f;
    13. }
    As we can see, the function takes in a patch. From the patch, we use the control point (vertex) UV coordinates to sample our tessellation map. For this tutorial, we'll be sampling the red values of the texture to determine if we should tessellate. If any of the red values have a value greater than 0 (meaning there is some red at one of the three UV coordinates), then all 4 factors are given our uniform tessellation value. Otherwise, they are given a value of 1.0, telling the tessellation program not to tessellate.

    Part 3 - Domain

    We are now on the final piece of the puzzle: the domain program. Because the domain program produces more vertices than we started with, we'll need to process these new vertices through our vertex program. And that means constructing appdata objects ourselves. In our case, we'll need the vertex positions and UV coordinates for all our new points.

    Unfortunately, what we get from the tessellation program is just barycentric coordinates for each new point. So we'll have to do a bit of leg work. Thankfully it's not too bad. All we need to do is use the barycentric coordinates as weights to interpolate our original control points. Doing this for all the control point fields gives us a complete appdata object that we can now feed into our vertex program.

    Code (csharp):
    1. [UNITY_domain("tri")]
    2. v2f DomainProgram(TessellationFactors factors,
    3.          OutputPatch<ControlPoint, 3> patch,
    4.          float3 barycentricCoordinates : SV_DomainLocation)
    5. {
    6.    appdata data;
    8.    data.vertex = patch[0].vertex * barycentricCoordinates.x +
    9.          patch[1].vertex * barycentricCoordinates.y +
    10.          patch[2].vertex * barycentricCoordinates.z;
    11.    data.uv = patch[0].uv * barycentricCoordinates.x +
    12.          patch[1].uv * barycentricCoordinates.y +
    13.          patch[2].uv * barycentricCoordinates.z;
    15.    return VertexProgram(data);
    16. }
    With everything now in place, all that's left is to get out pragmas in order.

    Code (csharp):
    1. #pragma vertex TessellationVertexProgram
    2. #pragma fragment FragmentProgram
    3. #pragma hull HullProgram
    4. #pragma domain DomainProgram
    5. #pragma target 4.6
    Part 4 - Using the Shader

    Now that our shader is done, you're probably going to want to test it out! To do that we need to do several things. First off, let's create a plane and put it in your scene (I recommend you use a higher poly plane - that way you can get more definition out of the tessellation). Next we need a material. Create one and select the shader to be the one we just created. You should now see two slots for textures that are appropriately named. Fill in the "Texture" slot with whatever you like. Next, you'll need a tessellation map. Go into your favorite image editing program and create a black image with some red areas (or use my tessellation map I created in the example section). Plug that into the "Tessellation Map" slot. All that's left is to apply the material to the plane and crank up that "Uniform Tessellation"!


    This was a pretty basic implementation of dynamic tessellation, but hopefully it gets you started so you can modify it to suit your needs. This was my first tutorial I've ever done, so if something wasn't explained particularly well or you have questions, please let me know!



    CatlikeCoding Tessellation Tutorial

    Tessellation Stages

    Unity Surface Shader Tessellation

    Barycentric Coordinates
  2. Gloubi88


    Mar 30, 2015
    Great tutorial, thanks! I'm saving this for later use :)
  3. Mr_Admirals


    May 13, 2017
    Glad I could help out!
  4. alexanderameye


    Nov 27, 2013
    Very useful!