Search Unity

Feedback How do you reverse-engineer built-in shader parameters?

Discussion in 'General Discussion' started by bitinn, Oct 25, 2020.

  1. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    One very annoying thing about Unity, is that tons of its built-in shader parameters are not explained. Neither in shader library comment, nor in documentation, nor in C# code (as they are C++).

    For example:

    _LightShadowData? People have been guessing it for years.
    unity_LightShadowBias? Again, same story.

    Developers have been using built-in shader parameters without knowing its full formula for a decade now.
    And that's why this kind of legacy problem remains in URP.

    Why do I care?

    When we need to reproduce the same input, but via SRP to make something backward compatible.

    Without a source code access license, I can only *try* to reverse-engineer some of them, and this often involves tons of testing, to see which parameters affect the result.

    What can Unity do?

    With URP and HDRP we gained understanding on how these built-in shader parameters are calculated.

    But Built-in pipeline parameters remains a mystery, so porting them remains a challenge. I do not believe there are any good technical or intellectual property reasons to not share how they are calculated.

    So can Unity do us a favor, please.
     
  2. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    13,562
    Nope.
    Shaders are HLSL and not C++. Their code is available for download from unity's website.

    https://unity3d.com/get-unity/download/archive

    unity_LightShadowBias being used:
    Code (csharp):
    1.  
    2. float4 UnityClipSpaceShadowCasterPos(float4 vertex, float3 normal)
    3. {
    4.     float4 wPos = mul(unity_ObjectToWorld, vertex);
    5.     if (unity_LightShadowBias.z != 0.0)
    6.     {
    7.         float3 wNormal = UnityObjectToWorldNormal(normal);
    8.         float3 wLight = normalize(UnityWorldSpaceLightDir(wPos.xyz));
    9.         // apply normal offset bias (inset position along the normal)
    10.         // bias needs to be scaled by sine between normal and light direction
    11.         // (http://the-witness.net/news/2013/09/shadow-mapping-summary-part-1/)
    12.         //
    13.         // unity_LightShadowBias.z contains user-specified normal offset amount
    14.         // scaled by world space texel size.
    15.         float shadowCos = dot(wNormal, wLight);
    16.         float shadowSine = sqrt(1-shadowCos*shadowCos);
    17.         float normalBias = unity_LightShadowBias.z * shadowSine;
    18.         wPos.xyz -= wNormal * normalBias;
    19.     }
    20.     return mul(UNITY_MATRIX_VP, wPos);
    21. }
    22.  
    23.  
    _LightShadowData:
    Code (csharp):
    1.  
    2. // ---- Screen space direction light shadows helpers (any version)
    3. #if defined (SHADOWS_SCREEN)
    4.     #if defined(UNITY_NO_SCREENSPACE_SHADOWS)
    5.         UNITY_DECLARE_SHADOWMAP(_ShadowMapTexture);
    6.         #define TRANSFER_SHADOW(a) a._ShadowCoord = mul( unity_WorldToShadow[0], mul( unity_ObjectToWorld, v.vertex ) );
    7.         inline fixed unitySampleShadow (unityShadowCoord4 shadowCoord)
    8.         {
    9.             #if defined(SHADOWS_NATIVE)
    10.                 fixed shadow = UNITY_SAMPLE_SHADOW(_ShadowMapTexture, shadowCoord.xyz);
    11.                 shadow = _LightShadowData.r + shadow * (1-_LightShadowData.r);
    12.                 return shadow;
    13.             #else
    14.                 unityShadowCoord dist = SAMPLE_DEPTH_TEXTURE(_ShadowMapTexture, shadowCoord.xy);
    15.                 // tegra is confused if we use _LightShadowData.x directly
    16.                 // with "ambiguous overloaded function reference max(mediump float, float)"
    17.                 unityShadowCoord lightShadowDataX = _LightShadowData.x;
    18.                 unityShadowCoord threshold = shadowCoord.z;
    19.                 return max(dist > threshold, lightShadowDataX);
    20.             #endif
    21.         }
    22.     #else // UNITY_NO_SCREENSPACE_SHADOWS
    23.         UNITY_DECLARE_SCREENSPACE_SHADOWMAP(_ShadowMapTexture);
    24.         #define TRANSFER_SHADOW(a) a._ShadowCoord = ComputeScreenPos(a.pos);
    25.         inline fixed unitySampleShadow (unityShadowCoord4 shadowCoord)
    26.         {
    27.             fixed shadow = UNITY_SAMPLE_SCREEN_SHADOW(_ShadowMapTexture, shadowCoord);
    28.             return shadow;
    29.         }
    30.     #endif
    31.     #define SHADOW_COORDS(idx1) unityShadowCoord4 _ShadowCoord : TEXCOORD##idx1;
    32.     #define SHADOW_ATTENUATION(a) unitySampleShadow(a._ShadowCoord)
    33. #endif
    34.  
    35.  
    The code can be read and modified. Example of modifier shader is here:
    https://neginfinity.bitbucket.io/20...-primitives-in-unity3d-part-5-distance-fields .
     
  3. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    You misunderstood my point: I knew these shadow code, I understood how unity_LightShadowBias are used, I have customized shader library and developed my SRP at this point.

    I am talking the part where we pass unity_LightShadowBias into shader, from CPU. We need to compute unity_LightShadowBias somehow. And they are not what you may assume them to be.
     
    INeatFreak likes this.
  4. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    basically, deduce how these values are obtained:

    Screen Shot 2020-10-25 at 16.22.40.png

    Screen Shot 2020-10-25 at 16.23.48.png

    Screen Shot 2020-10-25 at 16.24.50.png
    Screen Shot 2020-10-25 at 16.25.07.png

    If you read linked thread, I have already figured out some of them, but I cannot be certain if they are the whole picture, that's the annoying part.
     
    INeatFreak likes this.
  5. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    13,562
    .... excuse me, but this is a very standard stuff across all of game engines.

    "WorldToCamera" is camera view matrix.
    "CameraToWorld" is inverse of camera view matrix.
    Coincedentally, "worldToCamera" is equialent to Camera.gameObject.transform.worldToLocalMatrix.
    And "CameraToWorld" is equivalent to Camera.gameObject.transform.localToWorldMatrix.

    https://docs.unity3d.com/ScriptReference/Transform-localToWorldMatrix.html
    https://docs.unity3d.com/ScriptReference/Transform-worldToLocalMatrix.html

    So, basically, for all cameras, their view matrix is an equivalent of inverse of world matrix of their gameobject.

    What's more, because vectors of object matrices are perpendicular, rather than doing full inverse, you can create fast inverse, via decomposition of camera matrix into 4 vectors, transposition of rotational component, and recalculating its movement vector.

    In my opinion, this kind of stuff are things you are supposed to just know....

    Of course, there's a caveat and that's when camera requires flipping zaxis, or camera forward vector does not match object forward vector but all are easy to derive.
    --------------
    In your example, you have a matrix in, I believe, column notation (?).

    Where frist column corresponds to x vector of camera matrix, second one corresponds to y vector, third is z vector, and the last one is position offset.

    The two matrices are identical with exception of z vector which points in opposite direction. Which would mean that either one system is running on DirectX and another on OpenGL, or you're performing something that requires rendered scene to bee mirrored along camera z axis.
     
    Last edited: Oct 25, 2020
  6. JohnnyA

    JohnnyA

    Joined:
    Apr 9, 2010
    Posts:
    5,041
    @neginfinity the OP is not looking for some general description of concepts but exact details of these parameters. You can see from the linked code in the original post that Unity themselves made an error in this regard, so it is not as simple as you make it seem to be.

    You also didn't address the more pressing examples: if you know definitively how to calculate the _LightShadowData input please 'enlighten' us :)
     
    INeatFreak and OCASM like this.
  7. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    13,562
    Please describe the problem you're trying to solve with it.
     
  8. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    13,562
    Judging by the code snippet I provided, its coefficient whose sole purpose is to ensure that unitySampleShadow never returns value smaller than it.

    Now... unitySampleShadow literally returns how much light is visible at this point. See the code here, in particular "getShadowAttenuation":
    https://neginfinity.bitbucket.io/2016/04/raytraced-primitives-in-unity3d-part-41-code-cleanup.html

    Specifically:
    Code (csharp):
    1.  
    2. float getShadowAttenuation(float3 worldPos){
    3. #if defined(SHADOWS_CUBE)
    4.    {
    5.        unityShadowCoord3 shadowCoord = worldPos - _LightPositionRange.xyz;
    6.        float result = UnitySampleShadowmap(shadowCoord);
    7.        return result;
    8.    }
    9. #elif defined(SHADOWS_SCREEN)
    10.    {
    11.    #ifdef UNITY_NO_SCREENSPACE_SHADOWS
    12.        unityShadowCoord4 shadowCoord = mul( unity_World2Shadow[0], worldPos);
    13.    #else
    14.        unityShadowCoord4 shadowCoord = ComputeScreenPos(mul(UNITY_MATRIX_VP, float4(worldPos, 1.0)));
    15.    #endif
    16.        float result = unitySampleShadow(shadowCoord);
    17.        return result;
    18.    }      
    19. #elif defined(SHADOWS_DEPTH) && defined(SPOT)
    20.    {      
    21.        unityShadowCoord4 shadowCoord = mul(unity_World2Shadow[0], float4(worldPos, 1.0));
    22.        float result = UnitySampleShadowmap(shadowCoord);
    23.        return result;
    24.    }
    25. #else
    26.    return 1.0;
    27. #endif
    28. }
    29.  
    And how it is used:
    Code (csharp):
    1.  
    2. float shadowAtten = getShadowAttenuation(worldPos);
    3.    float lightAtten = getLightAttenuation(worldPos);
    4.    float lightDot = dot(lightDir, sphereContact.n);
    5.    float lightFactor = max(0.0, lightDot);
    6.    col = col + baseColor * lightFactor * _LightColor0 * shadowAtten * lightAtten;
    7.  
    So, 1.0 returned by unitySampleShadows means all light made it through, while 0.0 returns no light made it through.

    Which means with value of 0.0, you'll get doom 3 style shadow, and with values above 0 light will pass through the object and still affect shadowed area. While at 1.0 the shadow will have no effect.

    By the way. That's just based on the snippet I posted, and the work I did earlier. If you dig further through the cginc files, you'll arrive at this:
    Code (csharp):
    1.  
    2. inline half3 SubtractMainLightWithRealtimeAttenuationFromLightmap (half3 lightmap, half attenuation, half4 bakedColorTex, half3 normalWorld)
    3. {
    4.     // Let's try to make realtime shadows work on a surface, which already contains
    5.     // baked lighting and shadowing from the main sun light.
    6.     half3 shadowColor = unity_ShadowColor.rgb;
    7.     half shadowStrength = _LightShadowData.x;
    8.     // Summary:
    9.     // 1) Calculate possible value in the shadow by subtracting estimated light contribution from the places occluded by realtime shadow:
    10.     //      a) preserves other baked lights and light bounces
    11.     //      b) eliminates shadows on the geometry facing away from the light
    12.     // 2) Clamp against user defined ShadowColor.
    13.     // 3) Pick original lightmap value, if it is the darkest one.
    14.     // 1) Gives good estimate of illumination as if light would've been shadowed during the bake.
    15.     //    Preserves bounce and other baked lights
    16.     //    No shadows on the geometry facing away from the light
    17.     half ndotl = LambertTerm (normalWorld, _WorldSpaceLightPos0.xyz);
    18.     half3 estimatedLightContributionMaskedByInverseOfShadow = ndotl * (1- attenuation) * _LightColor0.rgb;
    19.     half3 subtractedLightmap = lightmap - estimatedLightContributionMaskedByInverseOfShadow;
    20.     // 2) Allows user to define overall ambient of the scene and control situation when realtime shadow becomes too dark.
    21.     half3 realtimeShadow = max(subtractedLightmap, shadowColor);
    22.     realtimeShadow = lerp(realtimeShadow, lightmap, shadowStrength);
    23.     // 3) Pick darkest color
    24.     return min(lightmap, realtimeShadow);
    25. }
    26.  
    And this:
    Code (csharp):
    1.  
    2. inline fixed UnitySampleShadowmap (float4 shadowCoord)
    3. {
    4.     #if defined (SHADOWS_SOFT)
    5.         half shadow = 1;
    6.         // No hardware comparison sampler (ie some mobile + xbox360) : simple 4 tap PCF
    7.         #if !defined (SHADOWS_NATIVE)
    8.             float3 coord = shadowCoord.xyz / shadowCoord.w;
    9.             float4 shadowVals;
    10.             shadowVals.x = SAMPLE_DEPTH_TEXTURE(_ShadowMapTexture, coord + _ShadowOffsets[0].xy);
    11.             shadowVals.y = SAMPLE_DEPTH_TEXTURE(_ShadowMapTexture, coord + _ShadowOffsets[1].xy);
    12.             shadowVals.z = SAMPLE_DEPTH_TEXTURE(_ShadowMapTexture, coord + _ShadowOffsets[2].xy);
    13.             shadowVals.w = SAMPLE_DEPTH_TEXTURE(_ShadowMapTexture, coord + _ShadowOffsets[3].xy);
    14.             half4 shadows = (shadowVals < coord.zzzz) ? _LightShadowData.rrrr : 1.0f;
    15.             shadow = dot(shadows, 0.25f);
    16.         #else
    17.             // Mobile with comparison sampler : 4-tap linear comparison filter
    18.             #if defined(SHADER_API_MOBILE)
    19.                 float3 coord = shadowCoord.xyz / shadowCoord.w;
    20.                 half4 shadows;
    21.                 shadows.x = UNITY_SAMPLE_SHADOW(_ShadowMapTexture, coord + _ShadowOffsets[0]);
    22.                 shadows.y = UNITY_SAMPLE_SHADOW(_ShadowMapTexture, coord + _ShadowOffsets[1]);
    23.                 shadows.z = UNITY_SAMPLE_SHADOW(_ShadowMapTexture, coord + _ShadowOffsets[2]);
    24.                 shadows.w = UNITY_SAMPLE_SHADOW(_ShadowMapTexture, coord + _ShadowOffsets[3]);
    25.                 shadow = dot(shadows, 0.25f);
    26.             // Everything else
    27.             #else
    28.                 float3 coord = shadowCoord.xyz / shadowCoord.w;
    29.                 float3 receiverPlaneDepthBias = UnityGetReceiverPlaneDepthBias(coord, 1.0f);
    30.                 shadow = UnitySampleShadowmap_PCF3x3(float4(coord, 1), receiverPlaneDepthBias);
    31.             #endif
    32.         shadow = lerp(_LightShadowData.r, 1.0f, shadow);
    33.         #endif
    34.     #else
    35.         // 1-tap shadows
    36.         #if defined (SHADOWS_NATIVE)
    37.             half shadow = UNITY_SAMPLE_SHADOW_PROJ(_ShadowMapTexture, shadowCoord);
    38.             shadow = lerp(_LightShadowData.r, 1.0f, shadow);
    39.         #else
    40.             half shadow = SAMPLE_DEPTH_TEXTURE_PROJ(_ShadowMapTexture, UNITY_PROJ_COORD(shadowCoord)) < (shadowCoord.z / shadowCoord.w) ? _LightShadowData.r : 1.0;
    41.         #endif
    42.     #endif
    43.     return shadow;
    44. }
    45.  
    The meaning of the value is "Shadow Strength".

    The use of _LightShadowData's z and w component is deprecated. Their meaning can be found in UnityCG.cginc, and they define how fast the shadows decay based on distance from camera in some specific case. Where .z defines fade speed, and .w -additional offset.

    (opinion)The reason why there's no documentation is because the shader code is not an API. It is in state of flux and changes. And to figure out what the code does, the code acts as documentation itself.
     
  9. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    @neginfinity so to clarify, I know how they are generally calculated and used, but I need to know how *exactly* they are calculated. because I am porting some built-in pipeline features and I need the value *exactly* as before.

    It might not be a requirement for your project, but it is on mine :)
     
    INeatFreak likes this.
  10. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    Or put it in another way, the shadow implementation on Built-in, URP and HDRP are all different. We are making a SRP that works with Built-in, without changing the meaning of Shadow Bias settings on our Light settings. (unlike URP and HDRP).
     
  11. INeatFreak

    INeatFreak

    Joined:
    Sep 12, 2018
    Posts:
    46
    @bitinn Hey. Have you ever figured it out these parameters?

    I've found _LightShadowData and _LightProjectionParams for 2019.1 version. But struggling with unity_WorldToShadow matrix. Do you know it by any chance?