Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[SOLVED] Shadow Mask in a PostProcess Shader?

Discussion in 'Shaders' started by flogelz, Jul 9, 2019.

  1. flogelz

    flogelz

    Joined:
    Aug 10, 2018
    Posts:
    141
    I would like to mask an outline postprocess shader by the shadow map on screen, so that the outline only appears on lit parts of the scene. Is there a macro or a trick to get this texture? I read about commandbuffers, but really have no starting point, of how to get there, if its even possible-
     
  2. flogelz

    flogelz

    Joined:
    Aug 10, 2018
    Posts:
    141
    I just found the solution! So I was also searching for a way to sample the screen space shadow texture with a commandbuffer and noticed, that they use this technique in the 3D GameKit from Unity! And it just seems to work straight out of the box this way! I saw many questions about this and nobody ever posted his working code, so here ya go!

    Code that should be attached to the main directional light:
    Code (CSharp):
    1. using UnityEditor;
    2. using UnityEngine;
    3. using UnityEngine.Rendering;
    4.  
    5. namespace Gamekit3D
    6. {
    7.     [ExecuteInEditMode]
    8.     public class CopyShadowMap : MonoBehaviour
    9.     {
    10.         CommandBuffer cb = null;
    11.  
    12.         void OnEnable()
    13.         {
    14.             var light = GetComponent<Light>();
    15.             if (light)
    16.             {
    17.                 cb = new CommandBuffer();
    18.                 cb.name = "CopyShadowMap";
    19.                 cb.SetGlobalTexture("_DirectionalShadowMask", new RenderTargetIdentifier(BuiltinRenderTextureType.CurrentActive));
    20.                 light.AddCommandBuffer(UnityEngine.Rendering.LightEvent.AfterScreenspaceMask, cb);
    21.             }
    22.         }
    23.  
    24.         void OnDisable()
    25.         {
    26.             var light = GetComponent<Light>();
    27.             if (light)
    28.             {
    29.                 light.RemoveCommandBuffer(UnityEngine.Rendering.LightEvent.AfterScreenspaceMask, cb);
    30.             }
    31.         }
    32.     }
    33. }
    And in the shader, access the map likes this:
    Code (CSharp):
    1. sampler2D _DirectionalShadowMask;
    and in the surf/fragment shader:
    Code (CSharp):
    1. float shadowmask = tex2D(_DirectionalShadowMask,screenUV).b;
    Theres also this Macro written there, but I'm not sure for what it's used:
    Code (CSharp):
    1. UNITY_DECLARE_SHADOWMAP(_DirectionalShadowMap);
    (I just didnt used it and it still worked) Also thanks to @bgolus for clearing this up:
    That's for defining a shadow map texture object ... which technically the screen space shadow texture is not so can be ignored. You've already defined it as a sampler2D, which is what it is.

    Anyways, I hope this helps someone!
     
    Last edited: Jul 22, 2019
    artofbeni and Arnage_nl like this.
  3. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    809
    Bumping this because I have an issue.

    The solution above works, except if I set:

    Code (CSharp):
    1. camera.depthTextureMode = DepthTextureMode.DepthNormals;
    In that case, it seems like the current active render target in AfterScreenspaceMask is no longer the screenspace shadowmask, but instead _CameraDepthNormalsTexture.

    I have a feeling this may be a bug because the documentation states:
    But I'm no shader expert.

    (tested in Unity 2018.4.14f1 & 2019.3.10f1)

    @bgolus Do you know if this is a known limitation?
     
    Last edited: Apr 23, 2020
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Sounds like a bug to me, yeah. The depth normal texture should only be the current target when using CameraEvent.AfterDepthNormalsTexture on a camera command buffer. Report it with a repro.
     
  5. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    809
    Submitted. Case 1245576.

    The conditions for it to happen seem to be:
    • Only one camera in the scene
    • Rendering path set to Deferred
    • DepthNormals texture enabled via script
     
  6. VirtusH

    VirtusH

    Joined:
    Aug 18, 2015
    Posts:
    95
    Any update on this? I can't find the bug report or anything!