Shadow Fadeoff approaching far clip plane

Hi,

So, this is an issue that has peeved me for a good while now, I reckon now is as good a time as any to ask about it.

The problem is as follows:

My game world (KSP) is too large for a single camera to render accurately. I have very close foreground objects (ships and such) and very distant objects (terrain). A single camera can't render the near objects without causing severe z fighting issues on the terrain (especially along shorelines), and to render terrain stably, I have to push the near clip so far it clips off the player's ship.

So, the solution is to use two cameras (on this layer. There are more cameras, but these are the ones that draw shadows, so they're the ones that matter). A Near camera starts off at a very close near plane and far clips at about 1km. Then a far camera picks up from there and draws the world from 1km to 80km (past which the terrain is replaced by the scaled layers, which are beyond the scope of the issue).

The problem I have is in the transition between the near and far cameras. They stack up nicely for geometry, but there is a noticeable artifact with shadows. You can see it plainly here:



That hard edge is where the far camera's frustum begins. If you look closely, you'll notice that the edge is caused by the near camera fading off shadows as they approach the far end of its frustum.

I've looked far and wide, and so far haven't had much luck finding anything about this... Admittedly, a multi-camera rendering stack isn't exactly common, and for most single-camera rendering setups, this is actually a feature, as shadows gradually fade from view as they get farther away.

I've tinkered with the shadow settings quite a bit too. In the KSC scene as seen above, the shadow distance is set to 5000m, well past the far clip of the near camera. The problem is that the shadows will fade both when approaching teh shadow distance, or when approaching the far clip plane, and there seems to be no way to adjust this behaviour.



Just for reference, I'm in Unity 5.2.4 now, but this has been here since U4.x and possibly even earlier. I have never not had this behaviour. It also seems to be consistent across platforms and rendering APIs too.

If anyone has useful knowledge to share about this, it will be hugely appreaciated!

As always, thanks in advance!!

Cheers

 

Same exact problem here, and it is very frustrating and wreaks graphic havoc with our two-camera setup. Any workaround or fix? Thanks in advance!

 

I reckon this could be worked around by writing custom shaders which implement a new shadow receiving pass, but it seems like an awful amount of reinventing the wheel for something which could be a simple tweak on the other side of things.

Unless there is a way we can do this tweak ourselves... Is there a way to change the default shadow receiver behaviour for all shaders?

Cheers

 

Added a schematic drawing for clarity. Current version is 5.2.4, and this is still the same (not that I expected a fix in any case).

I did however, find that the shader cgincludes are all open-source and modifiable, so it seems like it might be possible to fix this on our end. However, those files are a maze of nested include directives and legacy support code. I couldn't for the life of me find where this fade against the far clip is implemented. (In our case, for fwd-rendered soft-shadowed shaders)

If anyone with more experience digging through those shader sources is able to offer some ideas at this point, it would be immensely appreciated.

Again, thanks in advance!

Cheers

 

We're in deferred mode, but might take a stab at it later. Anyways, can't really understand why this does not have any Unity attention. Seems like it would be a *very* quick fix for the graphics team? Just make the shadow fade optional per camera, problem solved!

 

I struggle with this frequently. In open world games or games with far view, the unity shadow system is entirely inadequate and ugly, frankly. Unity said to me that it's part of their plans to take a stab at shadows at some point, but this is very vague.

Another consideration is the performance loss from redundant multiple camera renders. Ideally Unity would support something like reversed depth (which would also help Rust a lot too) http://outerra.blogspot.co.uk/2012/11/maximizing-depth-buffer-range-and.html

This would probably fix up shadows, except shadows afaik don't have the same precision available due to stencil or some such, and therefore needs a new technique regardless similar to what UE4 does. But I'm not too clear on what that would entail.

If Unity implemented this, it should fix up pretty much all the problems, and be just as fast with the vertex shader notes in the above link for optimisations.

Keeping a finger on this subject.

 

Thanks for that link. I've actually done a similar thing here a while back to z-sort ksp's map icons. Those draw to a fixed-depth canvas, so implementing a custom depth curve (mine isn't logarithmic, but it's similar in intent) was quite simple.

After I had that working, I was wondering if this couldn't be applied to terrain/scenery shaders to offset the verts along z in the vert program, so they're spaced more optimally along the depth range. That article answers a lot of questions I was mulling over about that. Sliding verts along z in projection space does seem to work, but it does create new issues that have to be solved afterwards. It seems they've solved quite a few already.

Trouble is though, in our case this would be a pretty massive restructuring of the game's rendering setup. It's definitely not a small fix... And considering that on the flip side, our dual camera setup only suffers from this shadow artifact (and a few other more minor glitches), the amount of effort needed to pull off the shadow fix probably makes it the more attractive option right now.

I do dream of one day being able to do away with the dual-camera setup entirely.

Cheers

 

I had an idea about this... on the last unity version there was a fix for some shadowing artifacts (thanks for that one, btw, we really needed it). I still have the previous versions here... My plan is to run a folder diff between the shader include srcs on both, and see what was changed, if anything.

A bit of a shot in the dark, but it might give an insight as to how/where shadows are set up. Those sources are impossible to navigate without a much more intelligent IDE than notepad++...

BTW, if anyone has suggestions of a good IDE for shader code editing, please share. I know VS has some new features, but I'd love to be able to see the shader code in a proper (C#-like) development env.

Cheers

 

Hi. So tell me, did you resolve the problem with the fading shadows in the far clipping plane? Or did you change your approach?
I am facing the same problem (with a 3 camera setup) and can't seem to find any reliable solution within Unity...

 

I guess the only way to go would be to implement your own shadows... I can help in this if needed, as I've done my own shadow mapping before. (That, and such a solution could work out nicely when rendering cascades per-camera. )

 

I have to bump this.

 

What should I do if I don't have any "screen space shadow shader"?
I can barely make a diffuse shader if I follow a -good- tutorial.
Google didn't turn up anything either

 

@Tesrym: If you need help getting @Jordi Bonastre's solution to work, I can help. I've done simple mods for the built-in shaders before.

Speaking of which...

@Jordi Bonastre Is there a place, like maybe, a "suggestion box" where you can put in suggestions? (I think Unity Feedback has it... but my situation's a little different) I made a small modification of the procedural skybox a while ago that I wanted to share, and wouldn't mind seeing it included with the default shaders.

 

Hi Unity, make it a per camera check box! thaaaaaanks.

 

Yes I would like some help @FuzzyQuills
Jordi lost me.
I got some shader error about needing another pass (???) and UnityEngine.Rendering.BuiltinShaderType.ScreenSpaceShadows does not exist (is this a 5.4 beta thing?)

 

Hmm... I'll have to check that API thing against the docs. (Or Jordi could enlighten us! )
Whats the error actually say?

Also, try importing UnityEngine.Rendering into your script. If that doesn't work either... what version are you actually using? pre-5.3 probably didn't have that...

 

Hi Jordi!
Thanks for that documentation!

Is there anyway to access the cascaded data in code? Like the world2Shadow matrices and the splitSpheres?

Cheerz!

 

I simply dont have it.

Not... Oh dog damnit I discovered it is in the beta http://docs.unity3d.com/540/Documen...ing.BuiltinShaderType.ScreenSpaceShadows.html
Aight. let me ... try a little beta then.

@Jordi Bonastre, you must change the article line "This article applies to Unity version 5.3" to 5.4

 

This is where I get stuck.
I havent got the faintest idea on where to insert "shadow += GET_SHADOW_FADE(wpos, vpos.z);".
There are already two of them in there.
http://pastebin.com/ajn13cA9

I don't have dyslexia, but shader code really evokes one.
Will keep guessing/playing, but I would not mind some extra help

 

Even I can replicate that. Good job @Jordi Bonastre.
So it kind of "works" in forward rendering.
Cant see any difference in deferred.

I wont be using this for anything, but I hope someone can take this idea further.
Good luck everyone

 

Hi. Since unity 5.6 works differently, where should I look to get this fix working in unity 5.6 ? Is it even still possible ? (I made a more precise answer on the Jordi-Bonastre's knowledge base article (https://support.unity3d.com/hc/en-us/articles/208504286))

 

How i implemet this in unity 2017

 

Sorry to bump this thread but I also facing this issue with Unity 2017.

Is there a way to disable this ******* fading on the far clip plane ?

 

Bump ?

 

Hello, we really need to get this working. Could someone at Unity please share the information on how to get around this issue in unity 2017? Solution doesn't work anymore...

 

Bump again.
Please I (we) really need to solve this.

 

Also having issues with this

 

Yes please a fix would be awesome

 

Waiting for a solution too !

 

Bump. Same issue.

 

Bump ...
Come on , how a checkbox to disable this stupid feature can be so hard to add !

 

So, I can kind of get around this in 2018.2.8 by editing the built-in UnityShadowLibrary.cginc file in the Unity install directory. If you go to UnityInstallDirectory/Editor/Data/CGIncludes folder you will find the UnityShadowLibrary.cginc file. Go to Line 322 and replace the UnityComputeShadowFade function with the following :

Code (CSharp):

1.  
2. half UnityComputeShadowFade(float fadeDist)
3. {  
4.     return 0.0;
5.     //return saturate(fadeDist * _LightShadowData.z + _LightShadowData.w);
6. }
7.  

This is obviously not the best fix as it will effect all of your cameras and maybe other things I'm not aware of too? Also, it's annoying since I have to do this on every install of Unity and anyone else sharing a project with me has to also make this change to see the same results as me. But for my particular needs this works for me for now. Someone who knows more about shaders or has a little bit more time than me may be able to find a better solution though.

 

Please make the shadow fade a render setting or something. Trying to make cloud shadows and it is a nightmare because of this.

 

Have run into this issue again
Unity guys, PLEASE. Its incredibly annoying and ruining working with multiple cameras

 

Thank you for this solution. It is not perfect as there are still some artefacts when the shadow reach the edges of screen on specific distances. But this a good way to work around.

I still hope Unity will finally add a checkbox to deactivate this feature.

 

More likely scenario is, they will implement another shadowing technology before they'll circle back to this issue. I'd suggest to bend your needs. Unity will not bend, we can establish that from our experience.

 

So, I've spent most of the day here looking at this issue again, starting from jnoffke's suggestion of poking at the UnityComputeShadowFade method. It's VERY hard to investigate any of this, because any change requires a unity restart to take effect, and being shader code, you can't use Visual Studio's navigation tools to find out how the code is tied together.

Anyhow, I've made a few discoveries, which means good and bad news.

The good news is that the computeshadowfade method is very much the right place to tweak this. The bad news, is that the fading against the far plane isn't something that was explicitly coded, at least not in the shaders. It seems to be pre-factored in with the shadow distance values that are exposed to the shader.

So that means there isn't anything that can be simply commented out to disable this. It's something that unity is doing before it passes the shadow distance values to the shaders.

So that sucks, but not all is lost. It is still possible to replace the shadow fade method with something that doesn't use the _LightShadowData values. It means we will have to keep track of the shadow draw distance ourselves, but as far as I can see, it's the only way to get around this problem.

What I'm going to try next is setting up external values that I can control with Shader.SetFloat maybe, and drive those from script... Let's see how that goes.

Cheers

 

So, here is a follow up:

The problem with just returning zero from that method is that while it does fix the fading near the camera clip plane, it also removes the smooth fading at the actual shadow distance. The catch is the the smooth fading values are pre-multiplied with the far plane fading, so there's no simple way to un-multiply that from those values.

After some thinking though, I realized that we don't have to do that. This whole issue only manifests itself in a stacked-camera situation, so it stands to reason that in such cases, the 'near' camera will have a fairly small far-clip distance.

If we assume then that the near camera will always draw shadow up to the far plane, we can then just override the fading altogether (by returning zero) if the distance of the frag being drawn is less than the near camera's far plane:

Code (CSharp):

1. half UnityComputeShadowFade(float fadeDist)
2. {
3.     if (fadeDist < 100)
4.         return 0;
5.     else
6.         return saturate(fadeDist * _LightShadowData.z + _LightShadowData.w);
7. }

My version of the shadow fade method then looks like this. At the lowest quality setting that still has shadows, the shadow distance is 150, which is more than the near camera range, at 100. So for any distances shorter than that, we bypass the fading.

This seems to have worked, at least in my project here (not KSP anymore). Hopefully it works for others too.

Cheers

 

Did you make it work on 2019.1 ? It doesn't seem like it works here
I tried returning 0 on both methods:

// ------------------------------------------------------------------

Code (CSharp):

1. float UnityComputeShadowFadeDistance(float3 wpos, float z)
2. {
3.     float sphereDist = distance(wpos, unity_ShadowFadeCenterAndType.xyz);
4.     return 0.0;// lerp(z, sphereDist, unity_ShadowFadeCenterAndType.w);
5. }
6.  
7. // ------------------------------------------------------------------
8. half UnityComputeShadowFade(float fadeDist)
9. {
10. return 0.0;
11.    // if(fadeDist<116)
12.    // return 0;
13.    // else
14.    // return saturate(fadeDist * _LightShadowData.z + _LightShadowData.w);
15. }

but It doesn't work, any other workaround?

 

Adding an option to enable or disable this can't be more than a one-liner for the folks at Unity, right?

 

You can't imagine how many times I thought that about some particular thing related to Unity

Anyway, SOMEONE? is there a workaround for this? Does anybody know why making the changes suggested nothing changes on my project? I will be eternally grateful for some guidance.

 

After you made changes to the .cginc file did you restart the editor? Also, this hack only really works for Close Fit Shadow Projection option set in Quality Settings. Stable Fit shadow projection will still have a large gap in the shadows as it looks like it's using a sphere to compute the shadow fade distance. Even with this hack on in close fit, you'll still likely get a little seam where the near camera's far clip plane and far camera's near clip plane meet.

 

I don't understand, It doesn't work.
Yes, I did restart the editor and yes Im using Close Fit shadows.
Right now im overwriting almost every method in the .cginc files related to shadows to see if I find the right one.
Let's see if I find out something.
Thanks!!

 

I'm on 2018.3 here. Not sure if this solution will work on later versions. The built-in shader code is very sensitive to version changes (it's not meant to be poked at, but here we are).

But yes, make sure you restart the editor after making any changes to those files. Unity will only load them in once at startup. (which makes working with these files all that more pleasant )