Unity 5.4 : (Case 818448) Incorrect cascaded shadows data when used during LightEvent.AfterShadowMap

Hi!

When I try to use the cascaded shadow globals (unity_ShadowSplitSqRadii, _LightSplitsNear, _LightSplitsFar, _LightShadowData, unity_WorldToShadow ...) for a custom effect in a shader using CommandBuffer.Blit (during the AfterShadowMap light event), I get three observable phenomena :

1. One light (seemingly the last rendered one) in the GameView has its cascaded shadows data switched with one in the SceneView. This only happens if lighting is enabled in the SceneView, else data in the SceneView are the same as in the GameView.
2. Data are not the ones corresponding to the light linked.
3. When changing the amount of shadow casting directional lights, the data of the others change.

Case is opened with number 818448.
Anyone experiencing related problems?

 

He bien comment ca va depuis 10tacles

It seems the open source light volume shader thread reported a similar problem you might check to see how they solve it. Will seek it later if nobody else answer.

 

Salut Timmy! erm... I mean... Hi NeoShaman!

Indeed they came across the same problem but they only worked around by disabling the cascaded shadows, which is not an acceptable solution as I need them and also, because it is clearly a bug.

I just got an email from Unity saying that they were able to reproduce the bug and that it was passed to the rendering team for fixing.
I'll keep this thread updated, and will also post the outcome in the Volumetric Light thread.

 

I also came into this problem when I was trying to draw something into cascaded shadow maps. Just in the same situation (LightEvent.AfterShadowmap, unity_WorldToShadow[])
It's good to hear that this is a bug...

 

I don't think this is a bug.

The Issues

1) The CommandBuffer bound to AfterShadowMap is executed whenever a shadowmap is rendered for that light, regardless of which camera is currently rendering it. The shader properties bound are correct for Camera.current at the time of the rendering, which may or may not be the camera you'd like it to be (it could be the scene camera, for instance). If you are rendering something to an external buffer in AfterShadowMap, then it may be wrong in the editor because AfterShadowMap will be called twice, once for your MainCamera, and then a second time for the Scene View. The second call would overwrite the data from the first call, and you'd get "the wrong data".
If this were "fixed", and an exception were made to prevent the Scene Camera from executing custom CommandBuffers, then it would be impossible to preview custom lighting effects in the Scene View, and would also probably break all lighting in the Scene View in general, assuming they run through the same rendering pipeline. Perhaps the real issue here is that the Scene Camera is not shown in the Frame Debugger.

2) This is extremely poorly documented, but seems to be intended as well. The shadow maps for Directional Lights usually aren't used (or set up to be used) in AfterShadowMap. Directional Light shadows are typically rendered in two passes. First, a light-space shadow map is computed, then second that light-space map is sampled and reprojected into a screen-space buffer, which the deferred shading pass samples to calculate shadows. Parameters such as "unity_WorldToShadow" may not be set because they aren't used in AfterShadowMap, and are only bound in BeforeScreenspaceMask, when the rendering pipeline is preparing to reproject the light-space map into screen-space.

3) This is again an issue with the ordering of commands in the render pipeline. The variables containing the shadow strength and properties of a Directional Light are unused in AfterShadowMap, and are only bound when they are needed, later in BeforeScreenspaceMask.

The Solutions

Problems 2 & 3 :

The solution here is to bind two CommandBuffers to your light. First, bind one to AfterShadowMap which captures the shadow buffer (the current active render target in this pass), and binds it to a global shader property.

Code (CSharp):

1. _AfterShadowMapCommandBuffer.SetGlobalTexture( "_CascadedShadowMap",  BuiltinRenderTextureType.CurrentActive  );
2.  

Then attach a second CommandBuffer to BeforeScreenspaceMask to do whatever processing requires info about your camera. At this stage in the render pipeline, things like "unity_WorldToShadow", "_LightShadowData", etc. should be set up. By including a sampler in your shader called "_CascadedShadowMap" you should be able to sample the shadowmap you bound in the first pass..

Problem 1 :

Attach a callback to Camera.onPreRender and Camera.onPostRender, or use the MonoBehaviour auto-magic functions to properly set up and clean up your camera! OnPreRender and OnPostRender are called sequentially for each camera. so the order will be as follows.

Code (CSharp):

1. Camera1.OnPreRender()
2. Camera1.OnPostRender()
3. Camera2.OnPreRender()
4. Camera2.OnPostRender()
5. SceneCamera.OnPreRender()
6. SceneCamera.OnPostRender()
7. ...

When OnPreRender is called for whatever camera you want to be rendering your effect, you need to initialize the AfterShadowMap / BeforeScreenspaceMask CommandBuffers for all the lights you'd like to be working with. You don't have to reallocate the buffers, but you will need to write all of the commands you'd like them to execute here, rather than in Start or something.

Code (CSharp):

1. void OnPreRender () {
2.     _lightScript.afterShadowMap.SetRenderTarget( .... )
3.     _lightScript.afterShadowMap.Blit( ... )
4.  
5.     _lightScript.beforeScreenspaceMask.SetRenderTarget(...)
6. }

Then, when OnPostRender is called, clear both of those CommandBuffers using "CommandBuffer.Clear()". This will ensure that other cameras rendering your lights won't do anything when they execute your CommandBuffers elsewhere.

Code (CSharp):

1. void OnPostRender () {
2.     _afterShadowMap.Clear();
3.     _beforeScreenSpaceMask.Clear();
4. }

* Alternatively, you could use AddCommandBuffer and RemoveCommandBuffer, rather than rebuilding and clearing them, but in my situation, I needed to adjust some shader values every frame anyway.

What Should Be Done ( This one's for you, Unity )

I would (and in fact, do) argue that this is not a bug. However, it's extremely confusing due to poor documentation, and things like the Scene Camera being hidden in the frame debugger. I would like to see these changes.

1. A toggle in the Frame Debugger to show/hide the render stack of the Scene Camera. This would go a long way to eliminate confusion when it comes to data appearing "in the wrong places".
2. Updates to the documentation for each LightEvent and CameraEvent listing all (relevant) bound shader properties at that time. What's the current active target? Did the lighting pass set "unity_worldToCameraMatrix"? etc..
   
3. Bind Directional Light properties and matrices in AfterShadowMaps if possible. I won't pretend to know exactly what goes on under the hood beyond what I've been able to infer through debugging and probing, but if at all possible, I would like to see light data bound one stage earlier in the shadowing pipeline than it currently is. As far as I can tell, it wouldn't interfere with anything, and would eliminate the need for a semi-redundant CommandBuffer on Directional Lights if developers would like to perform operations on shadowmaps.
   

I spend about five work-days reading through the builtin shader source, stepping through the frame debugger, reading all (of what little) documentation I could find on CommandBuffers, and experimenting with minimal test buffers and this is about what I've sussed out. I hope this helps others understand some of the oddities of working with CommandBuffers and LightEvents! Let me know if I can clarify something, or if I'm just plain wrong

Cheers,
-Andrew