Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question How to get world coordinates of pixel in fragment shader.

Discussion in 'Shaders' started by SHARKFEN, Aug 15, 2021.

  1. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    So a big part of this game im making is going to be the oceans and ocean fog. I made a shader with fog and everything already, but I can not find any answers online on finding exact world space coordinates of each pixel in the fragment shader. Its a very simple concept. Im going to get the pixel coordinate, and see if its below or above the water surface transform. If its below, I execute my ocean fog, if its above the water surface, i dont execute the fog. I have already gotten an effect with the shader below thats very close to what i need, but it works awkwardly. Im going for that seamless transition of the camera above and below the water surface (like in subnautica or sea of thieves). I am using the universal render pipeline if that matters at all. The shader is being applied through a blit custom render feature which works great.

    Any help is really appreciated.



    Here is my image effect script:

    Code (CSharp):
    1. Shader "Custom/ScreenFog"
    2. {
    3.     Properties
    4.     {
    5.         _MainTex("Base (RGB)", 2D) = "white" {}
    6.         _DepthLevel("Depth Level", Range(1, 3)) = 2
    7.         _FogColor("FogColor", Color) = (1, 0, 0, 1)
    8.         _Cutoff("Cutoff", Float) = 0.5
    9.         _Falloff("Falloff", Float) = 0.5
    10.         //fake bool, 1 true, 0 false;
    11.         _isUnderwater("is Underwater", Float) = 1
    12.     }
    13.     SubShader
    14.     {
    15.         Pass
    16.         {
    17.  
    18.             CGPROGRAM
    19.  
    20.             #pragma target 3.0
    21.             #pragma vertex vert
    22.             #pragma fragment frag
    23.             #include "UnityCG.cginc"
    24.  
    25.             uniform sampler2D_float _CameraDepthTexture;
    26.             uniform fixed _DepthLevel;
    27.             uniform half4 _MainTex_TexelSize;
    28.  
    29.             struct uinput
    30.             {
    31.                 float4 vertex : POSITION;
    32.                 float2 uv : TEXCOORD0;
    33.             };
    34.  
    35.             //cbuffer cbmat : register(b0)
    36.             //{
    37.             //    float4x4 tW; //World transform
    38.             //    float4x4 tWVP;//World * View * Projection
    39.             //};
    40.  
    41.             struct v2f
    42.             {
    43.                 float4 pos  : SV_POSITION;
    44.                 float2 uv   : TEXCOORD0;
    45.                 float4 wpos : TEXCOORD1;
    46.             };
    47.  
    48.             sampler2D _MainTex;
    49.             float _Cutoff;
    50.             float _Falloff;
    51.             fixed4 _FogColor;
    52.             float _isUnderwater;
    53.             float4 _MainTex_ST;
    54.  
    55.             v2f vert(uinput v)
    56.             {
    57.                 v2f o;
    58.                 o.pos = UnityObjectToClipPos(v.vertex);
    59.                 o.uv = TRANSFORM_TEX(v.uv, _MainTex);
    60.  
    61.                 float3 viewVector = mul(unity_CameraInvProjection, float4(v.uv.xy * 5 - 1, 0, 1));
    62.                 o.wpos = mul(unity_CameraToWorld, float4(viewVector, 0));
    63.                 return o;
    64.             }
    65.  
    66.             fixed4 frag(v2f o) : SV_TARGET
    67.             {
    68.                 if (o.wpos.y > 0) {
    69.                     _isUnderwater = 1;
    70.                 }
    71.                 else {
    72.                     _isUnderwater = 0;
    73.                 }
    74.                 if (_isUnderwater == 1) {
    75.                     float depth = UNITY_SAMPLE_DEPTH(tex2D(_CameraDepthTexture, o.uv));
    76.                     depth = pow(Linear01Depth(depth), _DepthLevel * (_DepthLevel * _DepthLevel / 2));
    77.                     depth = smoothstep(_Cutoff - _Falloff, _Cutoff, depth);
    78.  
    79.                     fixed4 fogOnly = _FogColor;
    80.  
    81.                     fixed4 col = tex2D(_MainTex, o.uv);
    82.                     col = lerp(col, fogOnly, depth);
    83.                     return 1 - col;
    84.                 }
    85.                 else if (_isUnderwater == 0) {
    86.                     float depth = UNITY_SAMPLE_DEPTH(tex2D(_CameraDepthTexture, o.uv));
    87.                     depth = pow(Linear01Depth(depth), _DepthLevel * (_DepthLevel * _DepthLevel / 2));
    88.                     depth = smoothstep(_Cutoff - _Falloff, _Cutoff, depth);
    89.  
    90.                     fixed4 fogOnly = _FogColor;
    91.  
    92.                     fixed4 col = tex2D(_MainTex, o.uv);
    93.                     col = lerp(col, fogOnly, depth);
    94.                     return col;
    95.                 }
    96.                 else {
    97.                     return _FogColor;
    98.                 }
    99.             }
    100.             ENDCG
    101.         }
    102.     }
    103. }
    104.  
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    The depth in the camera depth texture is the depth from the near plane in view space. The depth you're presumably trying to compare against is your water's depth in world space.

    Here's an example shader for getting the world position from the depth buffer, assuming you're using a perspective camera.
    Code (csharp):
    1. Shader "Unlit/WorldPosFromDepth"
    2. {
    3.     Properties
    4.     {
    5.     }
    6.     SubShader
    7.     {
    8.         Tags { "Queue"="Transparent" "RenderType"="Transparent" "IgnoreProjector"="True" }
    9.         LOD 100
    10.  
    11.         Pass
    12.         {
    13.             CGPROGRAM
    14.             #pragma vertex vert
    15.             #pragma fragment frag
    16.  
    17.             #include "UnityCG.cginc"
    18.  
    19.             struct appdata
    20.             {
    21.                 float4 vertex : POSITION;
    22.             };
    23.  
    24.             struct v2f
    25.             {
    26.                 float4 pos : SV_POSITION;
    27.                 float4 projPos : TEXCOORD0;
    28.                 float3 camRelativeWorldPos : TEXCOORD1;
    29.             };
    30.  
    31.             UNITY_DECLARE_DEPTH_TEXTURE(_CameraDepthTexture);
    32.  
    33.             v2f vert (appdata v)
    34.             {
    35.                 v2f o;
    36.                 o.pos = UnityObjectToClipPos(v.vertex);
    37.                 o.projPos = ComputeScreenPos(o.pos);
    38.                 o.camRelativeWorldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz - _WorldSpaceCameraPos;
    39.                 return o;
    40.             }
    41.  
    42.             bool depthIsNotSky(float depth)
    43.             {
    44.                 #if defined(UNITY_REVERSED_Z)
    45.                 return (depth > 0.0);
    46.                 #else
    47.                 return (depth < 1.0);
    48.                 #endif
    49.             }
    50.  
    51.             half4 frag (v2f i) : SV_Target
    52.             {
    53.                 float2 screenUV = i.projPos.xy / i.projPos.w;
    54.  
    55.                 // sample depth texture
    56.                 float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, screenUV);
    57.  
    58.                 // get linear depth from the depth
    59.                 float sceneZ = LinearEyeDepth(depth);
    60.  
    61.                 // calculate the view plane vector
    62.                 // note: Something like normalize(i.camRelativeWorldPos.xyz) is what you'll see other
    63.                 // examples do, but that is wrong! You need a vector that at a 1 unit view depth, not
    64.                 // a 1 unit magnitude.
    65.                 float3 viewPlane = i.camRelativeWorldPos.xyz / dot(i.camRelativeWorldPos.xyz, unity_WorldToCamera._m20_m21_m22);
    66.  
    67.                 // calculate the world position
    68.                 // multiply the view plane by the linear depth to get the camera relative world space position
    69.                 // add the world space camera position to get the world space position from the depth texture
    70.                 float3 worldPos = viewPlane * sceneZ + _WorldSpaceCameraPos;
    71.                 worldPos = mul(unity_CameraToWorld, float4(worldPos, 1.0));
    72.  
    73.                 half4 col = 0;
    74.  
    75.                 // draw grid where it's not the sky
    76.                 if (depthIsNotSky(depth))
    77.                     col.rgb = saturate(2.0 - abs(frac(worldPos) * 2.0 - 1.0) * 100.0);
    78.  
    79.                 return col;
    80.             }
    81.             ENDCG
    82.         }
    83.     }
    84. }
    If you need to support orthographic cameras, you need to do it a little differently. You might want to look at Unity's own Internal-ScreenSpaceShadows.shader to see how they do it:
    https://github.com/TwoTailsGames/Un...sExtra/Internal-ScreenSpaceShadows.shader#L63
     
  3. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    So this is definitely on the right track. I have access to the camera's world space position and im able to check if the camera is above or below water to draw water fog or not draw my water fog. But what i wonder is if there is any way that i can have the effect of showing both on screen at the same time.

    Like in this image where the fog of water is shown underneath the water line and there is no fog above the water, so just air. I assume the way to do this is doing exactly what the script does now, except instead of check the y value of the camera in world space, check the pixels y value. I know that this would be incredibly intensive to do and impractical. But ive always wondered if this can be done.



    But either way thank you so much for your reply, i can still get an incredible effect with this. What I think im going to do is create a script that shoots a vertical ray from the camera and when it hits the water that frame, it checks the y value of the waves at that current spot. Then i can pass that y value into my shader and compare it against the y value of the cam world position, and if its below i fog it up, and if its above i do some kind of atmospheric fog or something. Thanks!

    New Script:

    Code (CSharp):
    1. Shader "Custom/ScreenFog"
    2. {
    3.     Properties
    4.     {
    5.         _MainTex("Base (RGB)", 2D) = "white" {}
    6.         _DepthLevel("Depth Level", Range(1, 3)) = 2
    7.         _FogColor("FogColor", Color) = (1, 0, 0, 1)
    8.         _Cutoff("Cutoff", Float) = 0.5
    9.         _Falloff("Falloff", Float) = 0.5
    10.         //fake bool, 1 true, 0 false;
    11.         _isUnderwater("is Underwater", Float) = 1
    12.     }
    13.     SubShader
    14.     {
    15.         Pass
    16.         {
    17.  
    18.             CGPROGRAM
    19.  
    20.             #pragma target 3.0
    21.             #pragma vertex vert
    22.             #pragma fragment frag
    23.             #include "UnityCG.cginc"
    24.  
    25.             uniform sampler2D_float _CameraDepthTexture;
    26.             uniform fixed _DepthLevel;
    27.             uniform half4 _MainTex_TexelSize;
    28.  
    29.             struct uinput
    30.             {
    31.                 float4 vertex : POSITION;
    32.                 float2 uv : TEXCOORD3;
    33.             };
    34.  
    35.             //cbuffer cbmat : register(b0)
    36.             //{
    37.             //    float4x4 tW; //World transform
    38.             //    float4x4 tWVP;//World * View * Projection
    39.             //};
    40.  
    41.             struct v2f
    42.             {
    43.                 float4 pos  : SV_POSITION;
    44.                 float2 uv   : TEXCOORD3;
    45.                 float4 projPos : TEXCOORD0;
    46.                 float3 camRelativeWorldPos : TEXCOORD1;
    47.             };
    48.  
    49.             sampler2D _MainTex;
    50.             float _Cutoff;
    51.             float _Falloff;
    52.             fixed4 _FogColor;
    53.             float _isUnderwater;
    54.             float4 _MainTex_ST;
    55.  
    56.             v2f vert(uinput v)
    57.             {
    58.                 v2f o;
    59.                 o.pos = UnityObjectToClipPos(v.vertex);
    60.                 o.uv = TRANSFORM_TEX(v.uv, _MainTex);
    61.                 o.projPos = ComputeScreenPos(o.pos);
    62.                 o.camRelativeWorldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz - _WorldSpaceCameraPos;
    63.                 return o;
    64.             }
    65.  
    66.             fixed4 frag(v2f o) : SV_TARGET
    67.             {
    68.                 float depth = UNITY_SAMPLE_DEPTH(tex2D(_CameraDepthTexture, o.uv));
    69.                 depth = pow(Linear01Depth(depth), _DepthLevel * (_DepthLevel * _DepthLevel / 2));
    70.                 depth = smoothstep(_Cutoff - _Falloff, _Cutoff, depth);
    71.  
    72.                 float3 viewPlane = o.camRelativeWorldPos.xyz / dot(o.camRelativeWorldPos.xyz, unity_WorldToCamera._m20_m21_m22);
    73.  
    74.                 fixed4 fogOnly = _FogColor;
    75.  
    76.                 float3 worldPos = viewPlane * depth + _WorldSpaceCameraPos;
    77.                 worldPos = mul(unity_CameraToWorld, float4(worldPos, 1.0));
    78.  
    79.                 if (worldPos.y > 0) {
    80.                     fixed4 col = tex2D(_MainTex, o.uv);
    81.                     col = lerp(col, fogOnly, depth);
    82.                     return col;
    83.                 }
    84.                 else {
    85.                     fixed4 col = tex2D(_MainTex, o.uv);
    86.                     col = lerp(col, fogOnly, depth);
    87.                     return 1 - col;
    88.                 }
    89.             }
    90.             ENDCG
    91.         }
    92.     }
    93. }
    94.  
     
  4. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    Oh yeah! Not sure if its worth mentioning at all but the crest ocean asset in the unity store pulls it off flawlessly. So it is possible, and honestly they might even do it a completely different way that im imagining they do.


    This is just a screenshot from their asset page
    upload_2021-8-16_6-3-9.png
     
  5. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Ah, you're trying to handle the case of the water line crossing the camera.

    Yeah, that's not done "in shader", not exactly. There are a couple of ways to accomplish this, but every system I've seen either renders a "curtain" of geometry at the camera's near plane that matches or hides the water geometry surface intersecting with the camera plane, or renders a full screen mask of some kind. Then uses that to render the underwater area differently than above water. But basically it requires some additional c# side support to accomplish.

    For Crest you can actually look at how they're doing it since the non-URP/HDRP version is on github.
    https://github.com/wave-harmonic/crest
     
    jhocking likes this.
  6. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    Thanks for letting me know, i had no idea that this was done outside of the shader.

    So I took a bit of time to look through the crest documentation and scripts to see how they do it. For BIRP and URP they use an object called curtain and a meniscus object, which child to the main camera, and I think it is an object that draws fog behind it using its material, and the script creates the geometry curtain from the bottom of the screen upwards. But where i still get lost is how the shader knows where the line of the water is. I have some shader and script knowledge but this is far beyond my understanding of c#. Its really interesting to learn about it and im willing to spend a couple months looking into this and how they do it. I have a separate unity project with their working ocean so ill definitely delve into that some more.
     
  7. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    Ive been working at it some more and using a geometry curtain that renders underwater fog is just not working out for me, and there are also 0 references online to do so. And the crest documentation is for the BIRP and not URP so it doesnt translate well. I was wondering, do you think its possible to do the water effect with renderer features. Like maybe masking the water and above somehow and then using the mask for a image effect render feature that renders fog?
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    It should absolutely be possible as a render feature … if you can render a mask which is a whole extra can of worms that usually involves something like creating a curtain mesh still. * shrug *

    And yeah, URP stuff is a huge pain. Unity has put out official tutorials on how to do stuff for the URP that they then made obsolete in updates to the URP weeks, or sometimes only days later. There’s a reason why even “free” assets are now paid assets on the store when they’ve made URP and HDRP compatible versions. It’s a full time job just keeping them working, assuming Unity doesn’t break something else so bad they stop working at all.
     
  9. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    Hm yeah pretty interesting stuff. I at least managed out a mask that renders black and white depth onto just my water plane and black as the background. I did all this with render features and custom blits. Now the big issue is how can i essentially get back my normally rendered scene with colors and gameObjects, while saving this mask info, and using the mask to render or not render fog on the black or white pixels in a sort of image effect. upload_2021-8-19_0-9-29.png
     
  10. SHARKFEN

    SHARKFEN

    Joined:
    May 13, 2018
    Posts:
    7
    Note: All my dreams could come true with a second camera rendering the mask to a render texture. Except ive been told to avoid this due to it having a heavy impact on performance. If there were a way i could just send the rendered data to a mask then clear all the effects on screen. It feels so simple from here but im not sure what to do.