[SOLVED] Height to Nomal in shader

So I'm using noise, that i combine in ways to generate a dynamic noise pattern. I use tesselation for the displacement based on this generated noise, but displacement only gets me only so far. That's why i would like to convert my 2d noise into a normal Map in shader, to complement this random noise.

I found this node in shadergraph which exactly does this, but I have to write this in a surface shader. The code has a problem with the Position Parameters and the TangentMatrix though, when i paste it in my shadercode.
Does somebody has an idea, of what i could do, to make it work?

Code (CSharp):

1. void Unity_NormalFromHeight_Tangent(float In, out float3 Out)
2. {
3.     float3 worldDirivativeX = ddx(Position * 100);
4.     float3 worldDirivativeY = ddy(Position * 100);
5.     float3 crossX = cross(TangentMatrix[2].xyz, worldDirivativeX);
6.     float3 crossY = cross(TangentMatrix[2].xyz, worldDirivativeY);
7.     float3 d = abs(dot(crossY, worldDirivativeX));
8.     float3 inToNormal = ((((In + ddx(In)) - In) * crossY) + (((In + ddy(In)) - In) * crossX)) * sign(d);
9.     inToNormal.y *= -1.0;
10.     Out = normalize((d * TangentMatrix[2].xyz) - inToNormal);
11.     Out = TransformWorldToTangent(Out, TangentMatrix);
12. }

https://docs.unity3d.com/Packages/com.unity.shadergraph@5.3/manual/Normal-From-Height-Node.html

(I couldn't combine normal maps in a similar way like noise, thats why i would have to calculate this conversion in shader)

 

Ok, so i checked the code of a generated shadergraph shader and bascially it say this:

• Position = IN.WorldSpacePosition (so worldPos i guess)

• TangentMatrix = float3x3(IN.WorldSpaceTangent, IN.WorldSpaceBiTangent, IN.WorldSpaceNormal

Getting the worldPos and the worldNormal is pretty straightforward, but the other two?

 

Another Update: Seems like the Position and the Tangent are provided in the appdata. When i put in everything in, i spits out error though...

 

And that error would be? And you're using appdata_tan or a custom appdata?

 

Surface shaders in Unity automatically handle tangent-space transformations when you set o.Normal to a value, so it would be easier to abuse that fact and simply figure out the tangent space normals from the height texture alone.

Something along the lines of;

Code (CSharp):

1. //Don't forget to also define _HeightMap_TexelSize as a float4 (Unity will automatically fill this with your texture's dimensions)
2. float3 ts = float3 (_HeightMap_TexelSize.xy, 0);
3.  
4. float2 uv = IN.uv_HeightMap;
5. float2 uv0 = uv + ts.xz;
6. float2 uv1 = uv + ts.zy;
7. float h = tex2D (_HeightMap, uv).r;
8. float h0 = tex2D (_HeightMap, uv0).r;
9. float h1 = tex2D (_HeightMap, uv1).r;
10.  
11. float3 p0 = float3 (ts.xz, h0 - h);
12. float3 p1 = float3 (ts.zy, h1 - h);
13.  
14. o.Normal = normalize (cross (p0, p1));

 

Surface shaders don't necessarily provide all of the data you need in an easy to access form. Plus, if you're modifying an appdata struct someplace for a Surface Shader, unless you're hand modifying the generated code, you're probably not going to get the results you want. The only place you have direct access to the appdata struct is with a custom vertex function, but you can't do derivatives in a vertex function. You also should only ever be using appdata_full for custom vertex functions as the actual vertex shader is using that as the input. Understand the custom vertex function is just a function the actual generated vertex shader is calling, and the appdata struct for that function does not define the vertex stage's input.

First, let's talk about that TangentMatrix[2].xyz. That is the world normal, as you correctly surmised already. In a Surface Shader it would be accessed by putting float3 worldNormal; in the Input struct. Or at least it would be if that hasn't been broken for years for any shader that sets o.Normal. Instead along with including worldNormal and INTERNAL_DATA in the Input struct you need to use this bit of code in the surf function to get the world normal:

Code (csharp):

1. IN.worldNormal = WorldNormalVector(IN, float3(0,0,1));

Next is world to tangent transformation. Technically the world to tangent matrix is passed to the surf function, that's what the INTERNAL_DATA macro is adding to the struct. But due to some peculiarities with how Unity handles Surface Shader generation, if you try to actually directly use that data you'll get shader compiler errors. So instead you'll need to use the same WorldNormalVector() function to extract the other two elements of the matrix. My triplanar Surface Shader has an example implementation:
https://github.com/bgolus/Normal-Ma...der/blob/master/TriplanarSurfaceShader.shader

Code (csharp):

1. float3 WorldToTangentNormalVector(Input IN, float3 normal) {
2.     float3 t2w0 = WorldNormalVector(IN, float3(1,0,0));
3.     float3 t2w1 = WorldNormalVector(IN, float3(0,1,0));
4.     float3 t2w2 = WorldNormalVector(IN, float3(0,0,1));
5.     float3x3 t2w = float3x3(t2w0, t2w1, t2w2);
6.     return normalize(mul(t2w, normal));
7. }

Those WorldNormalVector() functions are dot products against the actual matrix, which feels like it should be way less performant than if you directly access the matrix, but shader compilers are smart. Since the input values to the dot product is a hard coded value, in the actual compiled shader the above code becomes identical to accessing the matrix directly!

I would normally agree with this, and it can also usually produce higher quality results than using derivatives, it depends on how expensive your noise function is. For a texture, I'd absolutely say use @Namey5 's example code, or something similar. For a noise function it may end up being significantly more expensive to have to calculate the noise multiple times. Also, ideally you want to use 4 offset samples instead of the center and 2 offset to derive the normals, otherwise you get a slight bias in your normals to the side and up, but it all depends on what level of quality you need / can find acceptable. I'd even maybe try both derivatives and calculating the noise multiple times and seeing which you prefer, and if either is significantly worse for performance in your use case.

 

@Invertex @Namey5 @bgolus Thank you guys for helping me out here! Just got a bit of freetime to test around again.
The code snippet from Namey works pretty well and the i understand the node from Unity way better now (this also fixed the error i was seeing before). Thank you for this in depth explanation. Bgolus!

The WorldToTangentNormalVector conversion doesn't work for me sadly, but heres a quirky screenshot anyways of the normal before until the point, were the tangentconversion should happen(It looks already quite wrong kinda, the input was a normal tex2d texture, but atleast something is happening)

Ah and here's the code, thats creating this.

Code (CSharp):

1. float noise2 = tex2D(_DispTex, IN.uv_MainTex);
2.  
3. IN.worldNormal = WorldNormalVector(IN.worldNormal, float3(0,0,1));          
4.  
5. float3 worldDirivativeX = ddx(IN.worldPos * 100);
6. float3 worldDirivativeY = ddy(IN.worldPos * 100);
7. float3 crossX = cross(IN.worldNormal.xyz, worldDirivativeX);
8. float3 crossY = cross(IN.worldNormal.xyz, worldDirivativeY);
9. float3 d = abs(dot(crossY, worldDirivativeX));
10. float3 inToNormal = ((((noise2 + ddx(noise2)) - noise2) * crossY) + (((noise2 + ddy(noise2)) - noise2) * crossX)) * sign(d);
11. inToNormal.y *= -1.0;
12. float3 nor = normalize((d * IN.worldNormal.xyz) - inToNormal);

The method of Namey5 would be promising, if it didn't require a tex2d input- I'm creating the noise by layering multiple textures on top of each other, but this already requires using tex2d before even coming to the normal conversion step. That's why the second method, were i just input a single float, would be better suited.

 

WorldNormalVector doesn't take IN.worldNormal as the first property, just IN. If that's the code you have, I'm impressed it compiles.

Otherwise, that looks correct.

 

Code (CSharp):

1. Shader "Custom/HeightToNormalSurfaceShader" {
2.     Properties {
3.         _Color ("Color", Color) = (1,1,1,1)
4.         _MainTex ("Albedo (RGB)", 2D) = "white" {}
5.         [NoScaleOffset] _HeightMap ("Height Map (R)", 2D) = "black" {}
6.         _HeightScale ("Height Scale", Float) = 0.1
7.         _Glossiness ("Smoothness", Range(0,1)) = 0.5
8.         [Gamma] _Metallic ("Metallic", Range(0,1)) = 0.0
9.     }
10.     SubShader {
11.         Tags { "RenderType"="Opaque" }
12.         LOD 200
13.  
14.         CGPROGRAM
15.         #pragma surface surf Standard fullforwardshadows
16.         #pragma target 3.0
17.  
18.         sampler2D _MainTex;
19.         sampler2D _HeightMap;
20.  
21.         struct Input {
22.             float2 uv_MainTex;
23.             float3 worldPos;
24.             float3 worldNormal;
25.             INTERNAL_DATA
26.         };
27.  
28.         half _Glossiness;
29.         half _Metallic;
30.         fixed4 _Color;
31.         half _HeightScale;
32.  
33.         float3 HeightToNormal(float height, float3 normal, float3 pos)
34.         {
35.             float3 worldDirivativeX = ddx(pos);
36.             float3 worldDirivativeY = ddy(pos);
37.             float3 crossX = cross(normal, worldDirivativeX);
38.             float3 crossY = cross(normal, worldDirivativeY);
39.             float3 d = abs(dot(crossY, worldDirivativeX));
40.             float3 inToNormal = ((((height + ddx(height)) - height) * crossY) + (((height + ddy(height)) - height) * crossX)) * sign(d);
41.             inToNormal.y *= -1.0;
42.             return normalize((d * normal) - inToNormal);
43.         }
44.  
45.         float3 WorldToTangentNormalVector(Input IN, float3 normal) {
46.             float3 t2w0 = WorldNormalVector(IN, float3(1,0,0));
47.             float3 t2w1 = WorldNormalVector(IN, float3(0,1,0));
48.             float3 t2w2 = WorldNormalVector(IN, float3(0,0,1));
49.             float3x3 t2w = float3x3(t2w0, t2w1, t2w2);
50.             return normalize(mul(t2w, normal));
51.         }
52.  
53.         void surf (Input IN, inout SurfaceOutputStandard o) {
54.             fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
55.             o.Albedo = c.rgb;
56.  
57.             half h = tex2D(_HeightMap, IN.uv_MainTex).r * _HeightScale;
58.             IN.worldNormal = WorldNormalVector(IN, float3(0,0,1));
59.             float3 worldNormal = HeightToNormal(h, IN.worldNormal, IN.worldPos);
60.  
61.             o.Normal = WorldToTangentNormalVector(IN, worldNormal);
62.  
63.             o.Metallic = _Metallic;
64.             o.Smoothness = _Glossiness;
65.             o.Alpha = c.a;
66.         }
67.         ENDCG
68.     }
69.     FallBack "Diffuse"
70. }

 

Yes!! It works! Thanks @bgolus for helping me out with this example script! Really appreciate it!!
I'm currently jumping a bit between different projects, but I'm gonna upload my results here if I'm finished!

 

Normal from height map seem make point filter effect,not smooth.how to fixed?

 

Yep. The quality of the normals that function produces is going to be fairly poor.

It's also not that it's point filtered, it's that it's bilinearly filtered. Bilinear filtering is a two way (hence "bi") linear blend from one pixel color to the next. The normal is the direction of the slope, and since the blend is linear between each pixel, the slope is a flat-ish surface between each pixel.

See this page for an explanation.
https://www.iquilezles.org/www/articles/texture/texture.htm

You can also look at doing more advanced (and plausibly more accurate) things like bicubic sampling of the texture which can give you a softer curve to work with.

 

Hey peeps, I got some questions if you don't mind...

float3 d = abs(dot(crossY, worldDirivativeX));

float3 inToNormal = ((((height + ddx(height)) - height) * crossY) + (((height + ddy(height)) - height) * crossX)) * sign(d);

((height + ddx(height)) - height) * crossY
What's the idea of adding the value just to remove it right after? Couldn't it be just ddx(height) * crossY?

sign(d)
Why get the sign of d if it is assigned to an absolute value in the line above? Wouldn't it be always positive?

Edit: Sorry to tag but.. @bgolus, curiosity is killing me