Question The best way to render smooth foliage?

I'm trying to make an okay-looking stylized foliage tree, and, once again, I am running into a lighting-related issue that has been haunting me for years.
I was wondering if the professionals among you have any suggestions on how to approach this in Unity.

First of all, this is the model I was given by one of our artists. The foliage is just a bunch of intersecting planes with custom normals to smooth out the shading a bit.


Now, in Unity the thing looks like this. And here's the problem: The planes are casting some really ugly-looking shadows onto one another which completely ruins the illusion.


Turning off "Receive Shadows" is not the solution to this, because the tree still needs to be able to receive shadows from mountains, buildings and other large structures.
Turning "Cast Shadows" off is even less of an option because, obviously, the tree still needs to cast a shadow onto the ground below.
With these two options ruled out, how is one supposed to get to something as nice-looking as the trees in other games like, for instance, Breath of the Wild? (I am aware that there's some translucency going on in Nintendo's foliage shaders, but that alone won't fix the issue in Unity.)


Thanks in advance, everyone, for looking into this.

 

Genshin Impact would be a good example of a Unity game that has successfully overcome this problem.





It seems that the foliage planes are still casting shadows onto one another, only they are significantly less pronounced, or blurrier, than on other objects.

 

Reminds me of the The Witness game. There was a thread about its foliage some time ago and had some interesting points:
https://forum.unity.com/threads/beg...st-to-do-the-foliage-from-the-witness.518842/

Maybe it's worth a read for you.

 

Probably use a proxy mesh, sperate foliage from trunk, but I don't see the foliage casting onto itself in those example, I see a gradient gong from internal darker to regular color, then mask by penumbra. It seems the effect is artistic not technique.

 

Ah yes, thanks. I did read that. They were never really discussing shadows though. Mostly view-dependent alpha blending and how to store two sets of normals. (The latter was kind of interesting but also rather wasteful. I'd rather use my uv channels for other stuff.)

You can see slight self-shadowing of the foliage in some pictures, with Genshin Impact especially, but it's really blurry. Either way I can't turn off any of the shadow options as I explained above. Trees still need to be able to cast shadows.

Foliage and trunk are using the same material btw to save on drawcalls, so my approach to lighting can't be too extraordinary.

 

I'm saying it's not from the shadow texture, the change in quality should have tipped you. I explain they probably do artistic manipulation.

 

You don't want to use shadow to give trees volume, you want to render it with a special foliage shader that takes light direction and world normal into account. Furthermore, you want to use AO to give your trees the inner shadow.

Give it a search on the asset store and you know what I mean.

 

Another trick I didn't mention, because that mean you must implement your own custom shadow player, is that when resolving shadow for foliage from a shadowmap, is that you scale the lighting by distance to the shadow occlusion, ie foliage just below occlusion are still bright, but the further along you go, the darker it is. That's very advance stuff to do in unity.

 

You could make the foliage shader's shadow pass add an extra depth offset to the vertices, so from the light POV the geometry is "further away", reducing self-shadowing on leaves but still allowing them to receive and cast shadows from/to distant surfaces.

 

I'm not looking to give the tree any more volume. The goal is to get rid of the overly crisp self-shadowing on the foliage while making sure that it is still somewhat respecting the scene lighting. Putting a gradient on the foliage should be trivial, but we can't have a glowing forest in the pitch-black shadow of a mountain just because the sun is still shining on the other side of it.

Sounds really interesting, but wouldn't this negatively affect the trunk which is part of the mesh, and possibly also the shadow that the tree is casting onto the terrain?

 

If the trunk is a separate material, no. If it's the same material, you could add per-vertex information to control how much offset will be applied at each vertex (using vertex colors or UVs). The shadow on the ground would look the same: what would happen is that, for shadow rendering purposes, the geometry would be "pushed away" from the light by a specified amount. As long as the offset isn't large enough to push the leaves into the ground, it should look the same.

 

You could also try uniforming your vertex normals in your 3d editor. I bake lighting in alot of foliage and to make sure I get a more uniform look on my foliage I make sure all my vertex normals on all alpha planes follow the same direction in my 3d program before bringing the meshes into unity like this. https://simonschreibt.de/gat/airborn-trees/ Give it a try you can implement a better style by following your own look just in the normals before any shaders get applied.

 

I'd also note that the shadow artifacts the op is looking to avoid, and referenced Genshin Impact as an example of a game that doesn't have them ... has the exact same artifacts they're complaining about.

Here's a screen grab from a gameplay video. Same issue.


Foliage in the distance appear to be falling back to simple cards, probably with a depth offset to prevent self shadowing, or blob meshes which won't show the same issues. As others have mentioned, the shadows are also softer. They also appear to be using some baked in shadows so interiors of the leaves go to shadow to hide more of the problems.

 

Sounds very good, but I doubt that that 's possible in Amplify Shader Editor, which is the tool I'm bound to use to make shaders.

Well, that sucks.

So as a conclusion I believe my takeaway from this thread is that manipulating the shadows on foliage simply isn't possible without a whole lot of ugly trickery that causes issues on other areas... and that the Offset values in a shader can be used to manipulate shadow casting. I had no idea.
I wasn't able to find a comprehensible explanation (layman's terms) of what the Factor and Unit values actually do, so I just started experimenting. With Factor = 0 and Units = 3000, this is the result I'm getting:

Granted that I need to use an additional shadow caster for the trunk, because it would otherwise be cut off about 1 unit above the ground, the result is still pretty good.
So unless anyone has anything to add to this, I believe that's it for this thread. Thanks for your help, everyone.

 

Don't use

Offset Factor, Units

. It's fine for proving out something, or doing small offsets, but once you get into large offsets of hundreds or thousands of units ... here be dragons this way.

Offset isn't consistently implemented between devices. One GPU's "3000" might equate to another's "30" or "300000".

It's better to do the offset in the shader itself by hand. This has the added benefit of allowing you to use different offsets for different parts of the mesh if you are so inclined, or more easily control it from the material (though you can modify the offset parameters using material properties too).

Code (csharp):

1.         Pass {
2.             Name "ShadowCaster"
3.             Tags { "LightMode" = "ShadowCaster" }
4.  
5.             CGPROGRAM
6.             #pragma vertex vert
7.             #pragma fragment frag
8.             #pragma target 2.0
9.             #pragma multi_compile_shadowcaster
10.             #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
11.             #include "UnityCG.cginc"
12.  
13.             struct v2f {
14.                 V2F_SHADOW_CASTER;
15.                 float2  uv : TEXCOORD1;
16.                 UNITY_VERTEX_OUTPUT_STEREO
17.             };
18.  
19.             uniform float4 _MainTex_ST;
20.             float _ShadowOffset;
21.  
22.             v2f vert( appdata_base v )
23.             {
24.                 v2f o;
25.                 UNITY_SETUP_INSTANCE_ID(v);
26.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
27.                 TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
28.                 o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
29.  
30.                 // only does offset for directional or spot light shadows
31.                 #if !defined(SHADOWS_CUBE)
32.                 if (unity_LightShadowBias.z != 0.0)
33.                 {
34.                     float4 worldPos = mul(unity_ObjectToWorld, v.vertex);
35.                     worldPos.xyz -= UNITY_MATRIX_V._m20_m21_m22 * _ShadowOffset;
36.                     float4 clipPos = UnityWorldToClipPos(worldPos);
37.                     float newZ = clipPos.z / clipPos.w * o.pos.w;
38.                     #if defined(UNITY_REVERSED_Z)
39.                     o.pos.z = min(o.pos.z, clipPos.z / clipPos.w * o.pos.w);
40.                     #else
41.                     o.pos.z = max(o.pos.z, clipPos.z / clipPos.w * o.pos.w);
42.                     #endif
43.                 }
44.                 #endif
45.  
46.                 return o;
47.             }
48.  
49.             uniform sampler2D _MainTex;
50.             uniform fixed _Cutoff;
51.             uniform fixed4 _Color;
52.  
53.             float4 frag( v2f i ) : SV_Target
54.             {
55.                 fixed4 texcol = tex2D( _MainTex, i.uv );
56.                 clip( texcol.a*_Color.a - _Cutoff );
57.    
58.                 SHADOW_CASTER_FRAGMENT(i)
59.             }
60.             ENDCG
61.         }

 

Check out this article!

https://simonschreibt.de/gat/airborn-trees/

https://forums.unrealengine.com/com...rical-Normals-for-Trees-(Blender)=#post751757