Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  3. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Camera Depth Texture sampling with 2018.3 and HDRP 4.X (mip map issue)

Discussion in 'Graphics Experimental Previews' started by stephero, Dec 6, 2018.

  1. stephero

    stephero

    Joined:
    Feb 8, 2016
    Posts:
    123
    Hi,
    I am using Unity 2018.3.0b12 with HDRP, and I want to sample the camera depth texture.
    Here how I do it, it's pretty regular:
    Code (CSharp):
    1. UNITY_DECLARE_DEPTH_TEXTURE(_CameraDepthTexture);
    2.  
    3. // ...
    4.  
    5. float4 projPos = ComputeScreenPos(vertex_out);
    6. projPos.z = -UnityObjectToViewPos(vertex_in).z;
    7.  
    8. // ...
    9.  
    10. return LinearEyeDepth(SAMPLE_DEPTH_TEXTURE_PROJ(_CameraDepthTexture, UNITY_PROJ_COORD(projPos))) * _ProjectionParams.w;
    11.  
    With the package HDRP 3.X (from 3.0.0 to 3.3.0), it used to work fine. Here is the normal rendering and the depth capture.
    Screenshot_19.png
    Screenshot_20.png

    But with the HDRP package 4.X (from 4.0.0 to the latest 4.3.0), it's completely broken. It looks like the _CameraDepthTexture stores mipmaps of the depth.
    Screenshot_21.png

    What should I do to properly sample the depth map texture with HDRP 4.X, while still working with HDRP 3.X and the built-in render pipeline?

    Thanks
     
    Jick87 likes this.
  2. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    936
    Hi,

    We have change the depth texture to encode a full depth pyramid (so all mip are in the mip0 side by side). To correctly sample the depth buffer you should use LOAD_TEXTURE2D (with screen absolute coordinate) instead of SAMPLE.

    In ShaderVariables.hlsl there is 2 helper function:

    // Note: To sample camera depth in HDRP we provide these utils functions because the way we store the depth mips can change
    // Currently it's an atlas and it's layout can be found at ComputePackedMipChainInfo in HDUtils.cs
    float SampleCameraDepth(uint2 pixelCoords)
    {
    return LOAD_TEXTURE2D_LOD(_CameraDepthTexture, pixelCoords, 0).r;
    }

    float SampleCameraDepth(float2 uv)
    {
    return SampleCameraDepth(uint2(uv * _ScreenSize.xy));
    }
     
  3. stephero

    stephero

    Joined:
    Feb 8, 2016
    Posts:
    123
    Thanks for the reply. I played with it a bit and it seems to work.

    But once I include the mandatory files to use the HDRP API ("Packages/com.unity.render-pipelines.core/ShaderLibrary/Common.hlsl"), it's really complicated to adapt a legacy shader which used to call a lot of "UnityCG.cginc" functions...

    I am making Unity Assets and aim to make shaders compatible with both legacy renderer and SRP. Until HDRP 3.3.0, it was fine because for my case, I didn't have to include SRP or HDRP specific files. But with this new way of sampling the depth texture, I have to include "ShaderLibrary/Common.hlsl", but it doesn't look like it's really possible to use both the SRP API and the good old "UnityCG.cginc".

    Is there any good practices about that, or are we supposed to have 2 completely separate shaders: one including "UnityCG.cginc" for the legacy pipeline, and one including the SRP API for HDRP/LWRP support?

    Thanks
     
  4. Tartiflette

    Tartiflette

    Joined:
    Apr 10, 2015
    Posts:
    84
    On a mostly related note, the Scene Depth shader graph node also wrongly samples the whole mipmap chain.
     
    sewy, P_Jong and Jick87 like this.
  5. darkydoodle

    darkydoodle

    Joined:
    Oct 27, 2018
    Posts:
    64
    I'm using a very simple customRender derivation from the HDRP on second camera.
    ==> I have some fragments (As) overwriting other fragments (Bs) althought they (As) are farther away and hidden behind (Bs).
    I tried different Ztest, nothing changes and anyway not all fragments are overwritten so it's not something that's happening in all cases there's just a logic to it (the bigger the distance between As and Bs, the more it happens).
    I looked at the _CameraDepthTexture and the depth of (As) is also overwriting the depth of (Bs)
    This is not happening on the main HDRP camera

    Anyway, in relation to this topic, I decided to compare the depth of the fragments to the depth texture. I did this previously in the legacy version (for water effects) and it worked well (just to say that it's not the first time i'm doing this kind of stuff).
    First, the _CameraDepthTexture is not an atlas of mipmap when you do a derivation, like with the HDRP main camera.
    Second, the encoding of the depth there is very strange. I tried to reconstruct the fragment depth by lots of different means using
    LinearEye
    Linear01
    "nothing"
    on
    -TransformWorldToView(o.vertex.xyz).z * _ProjectionParams.w;
    LOAD_TEXTURE2D_LOD(_CameraDepthTexture, i.screenPos.xy, 0).r; (with o.screenPos = ComputeScreenPos(o.vertex))
    SampleCameraDepth(i.screenPos.xy); (which is not working because it transforms my coordinates)
    etc...

    I can see that there's depth values in the depth texture "at the good place" for all of my tests, but impossible to compare it with the fragment depth like I did before. The value ranges are completly different, even when I manage to have a 0..1 range by using unity shader functions.
    Do someone know how the depth is encoded in the _CameraDepthTexture ?

    EDIT : found out about my depth problem : forgot I put 0 for depth buffer in the render texture. I'll see if it changes the fact that the depth texture values are strange. And anyway I need to compare depth for other effects.
     
    Last edited: Dec 18, 2018
  6. sewy

    sewy

    Joined:
    Oct 11, 2015
    Posts:
    150
    @SebLagarde any feedback on this one please?
     
    nhf75 likes this.
  7. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
  8. Tartiflette

    Tartiflette

    Joined:
    Apr 10, 2015
    Posts:
    84
    sewy and elettrozero like this.
  9. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    936
    fixed in 4.8.0
     
    sewy likes this.
  10. Tartiflette

    Tartiflette

    Joined:
    Apr 10, 2015
    Posts:
    84
    elettrozero likes this.
  11. Jason-Michael

    Jason-Michael

    Joined:
    Jul 6, 2015
    Posts:
    20
    I got the error : undeclared identifier 'SampleCameraDepth' when I call this method. I changed
    CGPROGRAM/ENDCG to
    HLSLPROGRAM/ENDHLSL and added

    Code (CSharp):
    1. HLSLINCLUDE
    2.     #include "UnityCG.cginc"
    3.     #include "HLSLSupport.cginc"
    4.  
    5.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/Common.hlsl"
    6.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/API/D3D11.hlsl"
    7.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/Macros.hlsl"
    8.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/UnityInstancing.hlsl"
    9.     #include "Packages/com.unity.render-pipelines.high-definition/Runtime/ShaderLibrary/ShaderVariables.hlsl"  
    10.     ENDHLSL
    11.  
    12.  
    And in frag function
    Code (CSharp):
    1. float depth = SampleCameraDepth(i.uv);
    here is the error:
    upload_2019-1-21_17-20-46.png

    I can not figure it out. Any advice is a big help, thanks in advance.
     
  12. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
    After doing what did you receive the error?
    If is after upgrading the package to 4.8, you must remove and reinstall the package.
     
  13. Jason-Michael

    Jason-Michael

    Joined:
    Jul 6, 2015
    Posts:
    20
    @elettrozero thanks buddy, my hdrp version is 4.8.0, I removed it and add it back again , still have the same error.
    Here is my shader code :

    Code (CSharp):
    1. Shader "MJ/ForwardDecal"
    2. {
    3.     Properties
    4.     {
    5.         _MainTex ("Decal Texture", 2D) = "white" {}
    6.     }
    7.  
    8.     HLSLINCLUDE
    9.     #include "UnityCG.cginc"
    10.     #include "HLSLSupport.cginc"
    11.  
    12.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/Common.hlsl"
    13.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/API/D3D11.hlsl"
    14.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/Macros.hlsl"
    15.     #include "Packages/com.unity.render-pipelines.core/ShaderLibrary/UnityInstancing.hlsl"
    16.     #include "Packages/com.unity.render-pipelines.high-definition/Runtime/ShaderLibrary/ShaderVariables.hlsl"
    17.     ENDHLSL
    18.  
    19.     CGINCLUDE
    20.     #include "UnityCG.cginc"
    21.     #include "HLSLSupport.cginc"
    22.     ENDCG
    23.  
    24.     SubShader
    25.     {
    26.         Tags{ "Queue"="Geometry+1" }
    27.  
    28.         Pass
    29.         {
    30.             ZWrite Off
    31.             Blend SrcAlpha OneMinusSrcAlpha
    32.  
    33.             HLSLPROGRAM
    34.             #pragma target 3.0
    35.             #pragma vertex vert
    36.             #pragma fragment frag
    37.  
    38.             struct v2f
    39.             {
    40.                 float4 pos : SV_POSITION;
    41.                 float4 screenUV : TEXCOORD0;
    42.                 float3 ray : TEXCOORD1;
    43.             };
    44.            
    45.             v2f vert (appdata_base v)
    46.             {
    47.                 v2f o;
    48.                 o.pos = UnityObjectToClipPos (v.vertex);
    49.                 o.screenUV = ComputeScreenPos (o.pos);
    50.                 o.ray = UnityObjectToViewPos(v.vertex).xyz * float3(-1,-1,1);
    51.                 return o;
    52.             }
    53.  
    54.             sampler2D _MainTex;
    55.             sampler2D _CameraDepthTexture;
    56.             float4 frag(v2f i) : SV_Target
    57.             {
    58.                 i.ray = i.ray * (_ProjectionParams.z / i.ray.z);
    59.                 float2 uv = i.screenUV.xy / i.screenUV.w;
    60.                
    61.                 float depth = LOAD_TEXTURE2D(_CameraDepthTexture, uv).x;
    62.                
    63.                 depth = Linear01Depth (depth);
    64.                
    65.                 float4 vpos = float4(i.ray * depth,1);
    66.                 float3 wpos = mul (unity_CameraToWorld, vpos).xyz;
    67.                 float3 opos = mul (unity_WorldToObject, float4(wpos,1)).xyz;
    68.                 clip (float3(0.5,0.5,0.5) - abs(opos.xyz));
    69.                
    70.                 float2 texUV = opos.xz + 0.5;
    71.  
    72.                 float4 col = tex2D (_MainTex, texUV);
    73.                 return col;
    74.             }
    75.             ENDHLSL
    76.         }
    77.     }
    78.  
    79.     Fallback Off
    80. }
    What is the right way to get depth :mad:
     
  14. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
  15. Jason-Michael

    Jason-Michael

    Joined:
    Jul 6, 2015
    Posts:
    20
    @elettrozero Shader Graph can certainly use the right depth value, but I just wanna to write a custom shader in which use the right depth value to implement a decal effect. Still , thanks buddy, I should go on Googling.
     
  16. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
    Try with this method SHADERGRAPH_SAMPLE_SCENE_DEPTH passing screen position .xy / .w
    I assume you're on HDRP, therefore you cannot access the _CameraDepthTexture directly.
     
  17. Jason-Michael

    Jason-Michael

    Joined:
    Jul 6, 2015
    Posts:
    20
    @elettrozero I just looked this function and use it in my shader code, it does't work ,bug you do enlighten me ;), I searched
    LOAD_TEXTURE2D and
    LOAD_TEXTURE2D_LOD, found the right way to get depth, 4 ways can do this.
    1. float depth = LOAD_TEXTURE2D_LOD(_CameraDepthTexture, screenPos, 0).r;
    2. float depth = LOAD_TEXTURE2D(_CameraDepthTexture, screenPos);
    3. float depth = LOAD_TEXTURE2D_LOD(_DepthPyramidTexture, TexCoordStereoOffset(screenPos), 0).r;
    4. float depth = LOAD_TEXTURE2D(_DepthPyramidTexture, screenPos).r;

    basicly use LOAD_TEXTURE2D, LOAD_TEXTURE2D_LOD and _CameraDepthTexture, _DepthPyramidTexture.
    I found that _CameraDepthTexture and _DepthPyramidTexture seem to look the same, andf from the buffer name I guess that they are the same texture in fact.
    CameraDepth.png
    DepthPyramid.png
     
    Last edited: Jan 22, 2019
  18. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
    They said they put the whole pyramid in the depth buffer texture and you should load the LOD you want but can you access one of the two variables in HDRP?
     
  19. Jason-Michael

    Jason-Michael

    Joined:
    Jul 6, 2015
    Posts:
    20
    yes , before access these variables we should declare the texture as TEXTURE2D, like:
    TEXTURE2D(_CameraDepthTexture);
    TEXTURE2D(_DepthPyramidTexture);

    not like in CG, which is
    sampler2D _CameraDepthTexture
    sampler2D _DepthPyramidTexture, this is the difference which got me stock.
     
    elettrozero likes this.
  20. jjxtra

    jjxtra

    Joined:
    Aug 30, 2013
    Posts:
    1,467
    Won’t this cause a problem for high resolution? Hypothetically if I was rendering at 6 or 8 k the depth texture would exceed the 8k texture limit?
     
  21. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,749
    Can't get this to work :(
    if i declare
    Code (CSharp):
    1. TEXTURE2D(_CameraDepthTexture);
    i get the error
    Code (CSharp):
    1. redefinition of '_CameraDepthTexture'
    if i don't I can use
    Code (CSharp):
    1. uint2 positionSS = input.texcoord * _ScreenSize.xy;
    2. float depth = LOAD_TEXTURE2D(_CameraDepthTexture, positionSS);
    and get error
    Code (CSharp):
    1.  'Load': no matching 1 parameter intrinsic method
    and then there is the matter of how to use
    Code (CSharp):
    1. Linear01Depth(rawDepth);
    Which now asks for one additional param
    Code (CSharp):
    1.  'Linear01Depth': no matching 1 parameter

    EDIT:
    Found this which works for 2019.3.0f6
    Code (CSharp):
    1. float4 CustomPostProcess(Varyings input) : SV_Target
    2. {  
    3.         float depth = LoadCameraDepth(input.positionCS.xy);  
    4.         return depth;
    5. }
     
    Last edited: Feb 3, 2020
    Propagant likes this.
  22. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,481
    Has _DepthPyramidTexture been removed from HDRP in Unity 2019.3.5f1 and HDRP 7.1.8 ?
     
  23. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Sorry, for necroing this thread, but is there a way to sample mip-maps of the depth texture inside shader graph?
     
  24. fenderrex

    fenderrex

    Joined:
    Sep 9, 2015
    Posts:
    4
    so ummm is it me or would having the depth map and a map of volumes blend difference in 3d space like solve like all problems associated with two players not having the same visual environment? like if I'm hiding in fog/dark someone outside the fog can see me easier than i can see them... like I'm pissed at unity really think unreal would be better than this hdrp junk