Help understanding how to implement soft shadow volumes

Hello everyone,
I'm currently making the lighting system for my game, its fairly low poly and with cel shading. I've tried to use standard unity shadows, but they really clash with the style even when soft. So I looked into shadow volumes, and they worked pretty well and i thought that i could apply a sort of posterized-ish penumbra the further away the shadow was from the light source. While I was working on that, I found this paper specifically: http://fileadmin.cs.lth.se/cs/Personal/Tomas_Akenine-Moller/pubs/soft_egwr2002.pdf .
I read it and understand the basic algorithm but I don't know how to implement it in a shader, I need help at around 4.2.

From a shader I have no idea how to get the direction of the penumbra wedge sides, or the "true" shadow volume's normal. Any help or a point in the right direction would be appreciated.

 

I remember seeing a lot of experiments in using stencil based shadow volumes to produce soft shadows like this years ago, it certainly brings back memories ... however these are all considered dead ends today. These techniques worked when much of the triangle processing was still happening on the CPU, and the model poly counts were very low. Basic, sharp shadow volumes can still work fairly well, but the soft versions either required rendering the whole volume multiple times with a jittered offset, or for this style of soft shadow volume rendering each “edge” as it’s own shadow volume. In the early days of low poly models, that made sense as there weren’t going to be too many edges. Today, depending on your model, the brute force approach of rendering the shadow volume multiple times may actually be faster!

 

Btw if you search for “penumbra wedges” you’ll find later papers and presentations that show more of what they’re doing.

 

I had planned on baking my penumbras/volumes before into game objects for the main directional light, and getting the other info from a shader(I think I still may be able to get it from a geometry shader, but that seems to have downsides too). But if i have to get it from cpu it's definitely not worth it.

I think I'm better off baking the shadow volumes into multiple layers per mesh, like an onion, and simply overwriting lighter shades on the stencil when i render them. The shadows wont be able to merge together very well, especially with point lights, but it should work fine for the art style.

 

This isn't something you can do with geometry shaders unfortunately. Unity's geometry shaders lack the necessary information to calculate accurate silhouette edges as they don't have access to adjacency data that you might see academic papers on rendering techniques often use. Plus the way this soft shadow technique works requires each edge is rendered in 3 unique passes, and these have to be done in order for each individual edge. This means for even a simple cube this technique adds a minimum of 12 extra draw calls per light. For more modern polycount meshes it can mean hundreds or even thousands of additional draw calls per object. It's also a high fillrate limited technique as it requires writing to the stencil buffer with every pass, and many of these will cover a not insignificant potion of the screen. This is why I commented that this technique is considered a dead end.

If your light positions and geometry are static you can absolutely build the volumes before hand and reduce runtime costs. If you wanted to go with more traditional hard edged stencil shadow volumes, or even the brute forced multiple jittered shadow volumes, this might be a good option. Depending on how blurry you want your shadows, you may not even need that many jittered volumes to get acceptable results.

 

It's still pretty interesting. I might revisit this in the future.

Thanks bgolus!