Need help creating a shader that "curls" polygons around a mesh

I have this monster creature that is covered in quads that serve as its fur. I am trying to create a shader that can control how "curled up" the hair gets. The hair polygons are vertex colored so that the tip is colored and the base is not. I've tried 2 methods so far: multiplying the vertex color with the normal vector, then adding it to the object position (this does curl it up, but it also stretches it far too much) and playing with the rotation node (no result close to what I want when using this method). What else can I try?

 

If your texture UVs are nicely aligned on these quads to be vertical or horizontal, and all quads are exactly the same length, you can abuse the mesh's tangent vector to infer the quad's "top" and rotate them.

Lets say you've got a quad with vertically aligned UVs. The mesh's tangent vector will be aligned to the uv's horizontal axis (pointing "right") in mesh space. Calculating the bitangent with cross(v.normal, v.tangent.xyz) * v.tangent.w gives you a vector that's aligned with the uv's vertical axis (pointing "up"). That bitangent * v.color.r * quadLength gets you the top vertex position for pivoting around.

So, that's the "easy" part. Next is rotating around that point.

Code (csharp):

1. float3 normal = normalize(v.normal);
2. float3 tangent = normalize(v.tangent.xyz);
3. float3 bitangent = cross(normal, tangent) * v.tangent.w;
4.  
5. float distFromPivot = v.color.r * _QuadLength;
6. float3 pivotOffset = bitangent * distFromPivot;
7.  
8. float angle = _Curl * UNITY_PI / 180; // degrees to radians
9. float s, c;
10. sincos(angle , s, c);
11.  
12. v.vertex.xyz += pivotOffset;
13. v.vertex.xyz -= (normal * s + tangent * c) * distFromPivot;

That's totally untested, but I think it should work?

If your quads are variable in length, then the vertex color could encode the length as:
length / max length * 255 = vertex color (for a 0 - 255 color value), or stored in an extra UV map as the actual length to remove the extra material property.

edit: giving it some more thought, it's wrong, but I'd have to think about it more to work it out exactly. Might be as simple as swapping the s and c on line 12, or removing line 13.
edit 2: fixed the example code

 

Don't have the time right now to process what you explained here mathematically, but just by giving it a quick look, I want to ask if those mathematical functions and calculations could be reproduced in shadergraph

 

Yep, all of that can be done using Shader Graph without any problems. Also, I updated the example to be correct, it was close before.

 

Ok, and if the UV's are not laid out nicely and in a uniform matter? The solution I found is specified in the original post, where I would multiply the vertex color with the normal vector and then add it to the position (controlled by a strength variable). This does raise the polygons, but it also increases their length. I tried clamping the multiplied normal vector with certain values, but this doesn't solve the problem, as the vertices get pushed down. What I need is a way to "cut" them off, while keeping their orientation. Is there any way I can achieve this?

 

I'm using the tangents to get a guess at the orientation of the quads, find a pivot position, and the axis to rotate around. Without that you'd have to encode that data into your mesh some other way, either baked into the vertex color (which has very poor precision) or into UVs (which has to be spread across multiple UVs if you're assigning the values in another program, or modify the mesh with a c# script in the engine after import).

Basically, the normal value gets one vector needed for the "curl", but you still need to know a second one so that you're rotating the vertex position around some point rather than just offset it along the normal, which will always lead to it getting longer.

 

That's the thing: finding a pivot to rotate around. All of my hair polygons are combined into 1 mesh. How do I find the right pivot for each and every one of them? And how do I know on which axis to rotate each of them?

 

With a script that goes over every triangle & vertex to figure that out, or calculated and set manually quad by quad.

How I'd do it:
Iterate over every vertex to find those with the vertex color you've used to mark the moving edges.
Find all triangles that reference those vertices, and match them up to find the quads.
Find the average position difference between the vertex colored vertices and non vertex colored vertices in each quad, and store that offset in an extra UV set.