Ideas on how to accomplish 2d terrain lighting with only a texture and a height map

Hey folks, I’m very new to shader design – so if the answer is buying a package or.. that it simply can’t be done – I understand.

I’ve created some c# code to attempt what I’m about to describe, but it’s so slow it’s unusable, and it only works in very specific light directions. I feel like a shader is the way to go here, but how to accomplish it, I am not sure. Any tips would be most helpful.

I have a texture (1024x1024), and I have a height map, which is also (1024x1024). By supplying the vector the light is coming from, how can I create a shader that will fill in AO, or shadows where the terrain would be casting shadows over lower terrain? Think a mountain shadow.

I got the idea from a program called World Machine. (AWESOME APP, TRY IT!) and wanted to play around with it in my own work. And that got me to wondering how one of their features work.

Here is a simple GIF that attempts to describe what I’m going for. Notice the shadows moving, as if the mouse cursor was the light source.

 

Here's a basic shader that will do this;

Code (CSharp):

1. Shader "Custom/Heightmap Lighting"
2. {
3.     Properties
4.     {
5.         _Color ("Color", Color) = (1,1,1,1)
6.         _MainTex ("Albedo (RGB)", 2D) = "white" {}
7.         _HeightTex ("Heightmap (R)", 2D) = "grey" {}
8.         _Height ("Height", Float) = 1.0
9.         _LightDir ("Light Direction (XY)", Vector) = (0.5,0.5,0.0,0.0)
10.         _LightDirect ("Direct Light Strength", Float) = 1.0
11.         _LightAmbient ("Ambient Light Strength", Float) = 0.05
12.     }
13.     SubShader
14.     {
15.         Tags { "RenderType"="Opaque" }
16.         LOD 200
17.  
18.         Pass
19.         {
20.             CGPROGRAM
21.             #pragma vertex vert
22.             #pragma fragment frag
23.  
24.             #include "UnityCG.cginc"
25.  
26.             sampler2D _MainTex;
27.             sampler2D _HeightTex;
28.             float4 _HeightTex_TexelSize;
29.  
30.             float2 _LightDir;
31.  
32.             half _Height;
33.             half _LightDirect;
34.             half _LightAmbient;
35.  
36.             half4 _Color;
37.  
38.             struct appdata
39.             {
40.                 float4 vertex : POSITION;
41.                 float2 uv : TEXCOORD0;
42.             };
43.  
44.             struct v2f
45.             {
46.                 float4 pos : SV_POSITION;
47.                 float2 uv : TEXCOORD0;
48.             };
49.  
50.             v2f vert (appdata v)
51.             {
52.                 v2f o;
53.                 //Vertex displacement for fun :)
54.                 //v.vertex.y += tex2Dlod (_HeightTex, float4 (v.uv, 0, 0)).r * _Height * 50.0;
55.                 o.pos = UnityObjectToClipPos (v.vertex);
56.                 o.uv = v.uv;
57.                 return o;
58.             }
59.  
60.             //Returns the largest component of a vector2
61.             float max2 (float2 a)
62.             {
63.                 return max (a.x, a.y);
64.             }
65.  
66.             half4 frag (v2f i) : SV_Target
67.             {
68.                 //Find per-pixel texture offsets
69.                 static const float4 ts = _HeightTex_TexelSize;
70.                 static const float4 duv = float4 (ts.xy, -ts.x, 0);
71.  
72.                 //Figure out a 3D light direction from the 2D input vector (also flip xz direction here because it's easier)
73.                 static const float3 lightDir = normalize (float3 (-_LightDir.x, 1.0 - saturate (max2 (abs (_LightDir))), -_LightDir.y));
74.  
75.                 //Sample the base height
76.                 float height = tex2D (_HeightTex, i.uv).r;
77.  
78.                 //Adjacent pixel UVs
79.                 float2 uv0 = i.uv + duv.wy;
80.                 float2 uv1 = i.uv - duv.wy;
81.                 float2 uv2 = i.uv + duv.xw;
82.                 float2 uv3 = i.uv - duv.xw;
83.  
84.                 //Sample heights of adjacent pixels
85.                 float h0 = tex2D (_HeightTex, uv0).r;
86.                 float h1 = tex2D (_HeightTex, uv1).r;
87.                 float h2 = tex2D (_HeightTex, uv2).r;
88.                 float h3 = tex2D (_HeightTex, uv3).r;
89.  
90.                 //Construct virtual 3D positions of adjacent pixels
91.                 float3 p0 = float3 (uv0.x, h0 * _Height, uv0.y);
92.                 float3 p1 = float3 (uv1.x, h1 * _Height, uv1.y);
93.                 float3 p2 = float3 (uv2.x, h2 * _Height, uv2.y);
94.                 float3 p3 = float3 (uv3.x, h3 * _Height, uv3.y);
95.  
96.                 //Calculate surface normals by taking position deltas
97.                 float3 normal = normalize (cross (p1 - p0, p3 - p2));
98.  
99.                 //Calculate basic surface lighting
100.                 float diff = saturate (dot (normal, lightDir));
101.                
102.                 //Multiply lighting by base height for AO and depth so that it makes a bit more sense
103.                 return tex2D (_MainTex, i.uv) * _Color * lerp (_LightAmbient, _LightDirect, diff * height);
104.             }
105.             ENDCG
106.         }
107.     }
108.     FallBack "Diffuse"
109. }

In general, the idea is to figure out surface normals from the heightmap and then calculate lighting as if it were a 3D object. Ideally you would use a real normal map, but this works by taking heights from adjacent pixels and reconstructing their position to calculate normals directly in the shader. This can look pretty bad with low resolution textures, but it gets the idea across.

 

oh my goodness! I will study this. I wasn't expecting such a good response and so fast. My original c# code was trying to detect a shadow cast by a taller "pixel" from the height map in the direction toward the light. It never occurred to me to think about it like a 3d object! Of course. Lighting can be calculated with the normals, and the normals can be construed by looking at surrounding objects. This doesn't seem like shadows at all, but instead lessening the mount of light for normals that are facing away from the light direction. brilliant!

Thank you for this body of work. I'll learn a lot from this.
I have also been watching this https://www.udemy.com/course/coding-in-unity-introduction-to-shaders which is helping me get a handle on shaders in general.

I wish I could pay you for helping me.

 

Wanted to share the result of your awesome work. I made one small change to add a "sealevel" slider to the shader. I don't apply the lighting to the water.

Still work to do, but this is progress!

 

Just in case anyone else stumbles upon this great answer from Namey5, I wanted to provide a quick, important, alteration I had to make to the shader to get it to work correctly in UI stuff.

I was using a raw image to display the results, and instead of darkening areas, it was lightening them! I found out that the final lighting calculation in the shader was also affecting the alpha. Before returning the final color, you'll need to reset the alpha back to 1.

Code (CSharp):

1.                    
2. // Multiply lighting by base height for AO and depth so that it makes a bit more sense
3. half4 col = tex2D(_MainTex, i.uv) * _Color * lerp(_LightAmbient, _LightDirect, diff * height);
4.  
5. // Reset the alpha
6. col.a = 1;
7.                
8. return col;
9.