How to get access to light attenuation in a deferred surface shader

Hey everyone. I'm fairly new to shaders, and I'm trying to figure out how to make a deferred vegetation shader. I know of a lot of ways to do SSS/light wrapping/translucency, but all of them require access to the shadows. It seems possible to do in deferred, because other assets/engines do it. Can anyone give me any tips to point me in the right direction?

 

Other engines do it with one (or more) of a few ways.

1. Store all information about the surface's translucency into the gbuffers to use later when rendering the deferred lights.
2. During the gbuffer pass also allow access to light information (often just the main light) so translucency / sss can be calculated immediately.
3. Render translucency / sss as a separate forward pass that gets added later / injected into the accumulation buffer pre-lighting.
4. Anything that uses translucency / sss gets rendered entirely as a forward pass like transparent objects.

I believe most of the assets on the store work like option 4, and do work "with deferred rendering", but don't actually get rendered using the deferred path. Alloy and Uber have custom gbuffer layouts which allow the, to work like option 1. Unreal Engine 3 worked like option 4 as well, and I think Unreal Engine 4 is a combination of 1 and 3 where cheap translucency is done in the gbuffer, but more complex sss is an added forward pass on top.

I seem to remember some older game engines using option 2, but it's not common. I seem to remember some foliage shaders doing this for Unity, but it means no shadowed translucent lighting.

Anything not option 1 or 4 in Unity is difficult as they don't give easy access to the shadows or even lighting data needed. For people who do completely custom lighting and shadow systems for Unity it's easier, but of course that's because they're not relying on Unity to do it.

 

@bgolus thanks very much for this information! If I wanted to do option 1, what file would I have to change? Would it be Internal-DeferredShading.shader?

 

Yes, though it depends a bit on how you want to implement your stuff. There's not a lot of free room in the existing gbuffer layout that Unity has, just a single channel with 2bits of precision (can store the values 0.0, 0.33, 0.67, and 1.0). I don't know exactly what Alloy or Uber do, but I suspect they use that channel as a switch to let the shader know to handle the gbuffer differently. Then of course you need to have your custom shaders wire to the gbuffers in your custom way.

 

@bgolus well I'm ferociously attacking this still and trying to figure it out. Thanks again for your help, you really set me on the right track! At the moment I'm having quite a hard time trying to figure out how to write to that little bit of unused gbuffer in RT2. I'm looking at the generated code from a surface shader but I'm unable to identify the relevant part in actually writing to the different gbuffer channels. Could you give me a hint? Will I be able to write to it in my surface shader file?

 

If you're using surface shader you'd need to use a custom lighting model and declare a deferred function.
https://docs.unity3d.com/Manual/SL-SurfaceShaderLighting.html

You should be able to set the outNormal.a to the value you want. You'll have to look around for examples of how to set the other values properly.

 

@bgolus You're an angel. I had just found the outNormal.a in the generated shader and was prepared to use that vert frag as my shader lol. Didn't know about deferred lighting functions in surface shaders! This is so much better. I'm really hot on the scent now! You have know idea how helpful you've been. Much of the knowledge I've gained in the last two days is from your posts around the forums.

 

I DID IT! I'm seriously freaking out right now because I've been trying to make a fully deferred vegetation unsuccessfully shader for 3 years now. I'm such a state of shock and disbelief that I'm checking the render paths debug view to make sure its actually the deferred shader every 5 min. I have a vegetation-heavy game and not only does ambient occlusion not give artifacts now (because forward shaders not writing to the DepthNormals) but my performance is 40% better across the board! This is a wonderful day! Thanks again @bgolus! Seriously could not have done it without you! Here it is in action:

 

Looks great! Glad to be of help.

 

I'm glad you were successful I know what it's like to work on shader for a long time.
BTW what range was you target application, high end PC or mid range.

 

I don't exactly know what's considered mid/high, but I'm aiming for 60 fps on a PS4 level video card for my medium setting, and at least 30 on mid-level laptop GPUs like GTX 960M for my low setting.

 

GTX 960M might be upper mid range now. I'm running a Quadro NVS 290 on a work station PC. It's low end but better than dedicated graphics.