Shadow Problem

I have a problem with my character's shadow. In the attached image, I used the same model with different fbx import settings. On the left I set Normals:Import, on the right I set Normals:None. When I set Normals:Calculate and increase Smoothing Angle I have the same shadow with the right character, but character's geometry becomes smooth instead of flat. what is the solution for this?

 

This is caused by Unity's Normal Bias on lights. When rendering shadow maps, the mesh is slightly shrunk using the vertex normals to avoid shadow acne problems. The side effect of this any hard concave edges may show gaps, since the vertex normals on either side of the edge will open up those seams.

Solutions:

1. Live with it and pretend it doesn't exist.
2. Turn down the Normal Bias setting on the light (though you'll likely have to increase the Bias to avoid shadow artifacts, which also increases the chance of peter panning / disconnected shadow issues). Least amount of work, but worst looking option.
3. Use two separate meshes for the visible mesh and shadow caster, one with the hard edges for what you see set to not cost a sahdow, another with smooth normals (which you can do with the Normals:Calculate option & Smoothing Angle set to as big as the slider will go) set to shadow only on the mesh renderer component. This is a bit of a pain to manage since you need to have two copies of the mesh, and isn't great for performance if you have a lot of characters since you're now rendering two skinned meshes for every character. It is easy to implement though.
4. Use a mesh with smooth normals, and use a shader to calculate the flat triangle normals. Works great if you want a completely facetted look, and the resulting flat normals are technically more correct than those on a skinned mesh normally. But if you only want some edges to be hard, there's no way to make a shader do that. There's also no way to get this to work on OpenGLES 2.0 devices as it lacks either of the shader features required to calculate flat normals per triangle.
5. Use a mesh with smooth normals, and use a normal map to get the flat sides / hard edges you want. Flat sides won't be as perfectly flat as the vertex normal based approach. And you'll need a relatively high resolution normal map for it to look sharp. Splitting up the UVs for each area like you would for a light map with some padding between can help get sharper edges, and most modelling tools have automatic UV layout tools for that.
6. Use a hard edged mesh, but encode the smooth normals into the mesh using either abusing the vertex tangent's, or using the vertex color / extra UV set to store smoothed tangent space normals. Use the smoothed normal in a custom shadow caster shader. Toony Colors Pro does this already for doing mesh outlines on skinned meshes, but I'm not sure if it fixes the shadows or not (I would hope it does, but I've never looked). You can't just store the smooth normal in the vertex color or UV in object space since skinned meshes only update the vertex position, normal, and tangent, so when they're deformed by their skinning any normal not in the vertex normal or tangent won't get touched.

 

@bgolus
So this is some sort of bug? Or is it just that way in unity?

 

Not a bug per se, but a least bad solution to a difficult problem.

Shadow acne from real time shadows are a really hard thing to avoid. The main three solutions are linear biasing, normal biasing, and receiver plane biasing.

Linear biasing is the oldest solution. This just pushes the shadow surfaces away from the light. But this causes peter panning. And it requires quite a high amount of bias to avoid acne on any surface nearly parallel with the light direction.

Normal biasing works better than linear biasing at removing acne, but causes a different set of artifacts. Mainly it causes seams to open up on hard edges like above. Most AAA games use smooth meshes with normal maps even on hard edged objects in large part for this very reason. Even then there are still issues with normal biasing causing an object's shadow to shrink, and especially thin objects it might invert itself meaning it suddenly gets a thicker shadow or even self shadows when it shouldn't.

So most games use some combination of linear biasing and normal biasing, trying to use the minimum amount of both to produce the least amount of visible artifacts. Any artifacts that do show up often objects are just subtly rotated to try to make it less bad, or they give up trying to fix it. With the advent of TAA, some games jitter the shadow maps projections and make use of a lot of noise to hide it.

Receiver plane biasing is a little different. The above two are applied while the shadow map itself is being rendered, so the data being sampled is already offset when it's read by the shadow receiver. Receiver plane biasing does the biasing on the receiver. This has the benefit of not needing to do any biasing in the shadow map passes, meaning (potentially) no peter panning and no shrinking / inverting shadows. However it can be harder to implement properly as it's still doing the same linear and normal biasing calculations, just using the surface normal to offset where the object's shader samples the shadow map. This can cause the inverse of peter panning where shadows end up looking shorter than they're supposed to, and on hard edges the shadow edge may also become broken. But at least it's not thinner, and there won't be cracks. It's also hard for deferred rendering because you don't have the original mesh surface normal anymore meaning the surface's normal map can cause the received shadow edge to warp, or it has to calculate the normal from the depth which will have issues on object edges. Forward renderers usually have the advantage of knowing the actual mesh surface normal, but Unity's built in forward renderer does the main directional light's shadows as a deferred pass with the depth texture, so it has the same problem as deferred rendering would!


Receiver plane biasing is supposed to solve the issues that normal biasing has and remove the need for smoothed normals, but ultimately even under ideal situations it still works best with smoothed mesh normals.

 

Don't know if you solved this, but a solution that worked perfect for me was going into my imported mesh's "Skinned Mesh Renderer" component and turning on "Two Sided" for the cast shadows option. This worked best for me and gave the best shadows without messing with the normals. Hope it works for you