Various skybox-fog related shader questions (deferred rendering)

I could use a few suggestions on how to best achieve a number of fog related effects for my game. It has a sort of RTS-ish perspective with freely movable camera that can be put into a position where the Horizon would be seen. The levels are square terrain meshes, something like this:



I want to hide the edges of the terrain without drawing tons of landscape outside of the active play area. The accepted solution for this kind of thing in RTS games seems to be fog. This is what I have made:



It is "sort of ok", but I'd prefer it to look "good".

What I would like is to have the fog fade to the skybox, because that generally seems to give a more realistic impression, based on what I've seen. I've briefly tried Kejiro's fog effect, which doesn't seem to work anymore for some reason (my Unity version might be incompatible).
The fog effect I've implemented so far is a mix of fading the custom deferred terrain shader to the fog-color in the emissive channel and enclosing everything in a cube with inwards facing normals and a custom depth-biased transparent shader that also usese the fog-color.
The rest of the fog in my scene currently comes from the fog effect in the Post Processing Stack v1 and I'm using Unity 5.6.

My thoughts on how to possibly tackle this are so far:
- use a grabpass directly after drawing the skybox
- use that pass in the terrain shader to fade the terrain edges towards hat pass
- use that pass in a modified version of the global fog image effect (I'd rather hack just the fog than a part of the Post Stack)

But... what do I do with particle shaders? I'm relying quite heavily on having fancy particle effects that also blend nicely into the fog. Currently they do that just fine. E.g. I'm using @jack-riddell's excellent lit smoke and fire 2 ( https://www.assetstore.unity3d.com/en/#!/content/79694 ) for fire and smoke. What are my options if I don't want to break all those shaders and make them look out of place?

Is there a way to manipulate the fog-related parts in some .cginc file somewhere to all shaders using the fog macros sample the color from the grabpass in screenspace, or would that require manual editing of all particle shaders?

All thoughts or suggestions on this are welcome, thanks.

I should mention my own custom shaders are all done in ShaderForge.

 

The deferred passes render before the skybox. A grab pass after the skybox would be after the terrain was already rendered so there wouldn't be any sky to blend to.

You could render the sky with another camera to a render texture. Or if you're using a cubemap skybox you can read directly from that texture. That's what the KinoSky did.

When doing fog for deferred it has to be done as a post process to look correct. This is how the post processing stack fog works or how Global Fog and KinoFog used to work. However all forward rending (specially anything transparent) it needs to be done as part of the shader. By default Unity supports only the Fog modes listed in the lighting panel, which are just different kinds of falloff curves. If you want to do something like fading to a skybox you do have to customize any transparent shader you use. That includes particle shaders.

There is a work around of sorts. Most shaders are going to use the built in fog macros for calculating and rendering fog. It’s possible to override the file that define those macros within the Unity editor folders. You’ll need to force all shaders to recompile once you do though, so you’ll want to put the modified file in your project’s asset folder along side a simple custom particle shader to test with. That likely won’t affect the fog post process though, so that too will need to be modified manually.

 

Thanks a lot for your reply @bgolus! I didn't know sky gets drawn after terrain in deferred, that info surely saved me some time. The "clear flag: skybox" setting in the camera inspector made me think skybox comes first.

I think what I will try to do is use a hand-made skybox textures instead of the default procedural skybox, so that I have better control about what color goes where, and then hack one of the fog post effects (either in the stack or adapt one that is separate) to use my skybox texture. For the particle shaders I'll likely not mess with them and instead try to author the skybox in a way that I can manually set the fog color to something that reasonably averages the color of my skybox in places where particles can typically occur.

Thanks again for your help!

 

Because in Unity 4 and prior, it did. Unity 5's procedural skybox is expensive enough to render that there were likely clear performance savings in having it not render full screen every frame if some or all of the screen was covered with other stuff.

 

Just wanted to give you an update on how it's going. I ended up adding a second camera that renders the skybox to a rendertexture and added a second material to the terrain that just draws a transparent shader with the edge of the terrain fading from fully opaque skybox rendertexture to fully transparent. A neutral fog color works well enough for the regular fog image effect and the particle shaders. Since the transparent shader gets drawn after deferred fog, it's not an issue that the deferred fog has a slightly different color. There still are some angles from where I can see the terrain's edge in front of the skybox, but since there's no incentive for players to go looking for that it should be ok like this.
Thanks again for your help. It's amazing how much knowledge you share with this forum in general!