Resolved Convert object UV to screen space UV to apply distortion in a specific area

Hello community,

I'm stuck trying to implement the following effect:

• I have a quad on the scene with material and shader assigned
• in the shader I stretch and distort UVs in the object space (0,1 range)
• then I need to grab screen color behind the quad and apply these UV distortion to this pixels (taking into account position, rotation and scale on the quad)

I'm using Shader Graph and "Screen Color" node to get the pixel color from the background. If I plug my distorted UVs directly to this node then (obviously) I have the whole screen rendered in this quad (normalized screen space UVs directly mapped to normalized object space UVs).



I guess I need somehow convert or remap my distorted Object UVs to the screen space UVs and then plug them to Screen Color node. But I cannot figure out how to do that. Please help!

 

First, this is a Shader Graph question, and there's a Shader Graph specific forum you should ask these questions in.

Ignoring that, you cannot use the UVs of an object for screen space distortion. Using the object's UVs as an input to the Scene Color node just means you'll see the contents of the screen displayed on that object. You need to use the Screen Position node, and distort those. If you're using the object's UVs to derive the distortion, that might still work out okay as long as the object (and the camera) aren't rotated.

 

thanks bgolus! Sorry for shader graph, I just used it to quickly try different approaches. For me this is more general issue not related to shader graph. It would be the same if I'd write shader in text editor.

I thought that I could multiply UVs by MVP matrix to convert it from object to Clip space but apparently it doesn't work.

I tried to use Screen Position as you suggested and the background renders correctly, but distortion is applied in screen space instead of object space. Basically it is the same problem, but I this case need to convert distortion from screen to object space...

This what I'm trying to achieve:

 

You actually kind of had the right idea. The only thing you missed is you're not transforming a "UV" from object to clip space, but a synthetic local object space to clip space.

Code (csharp):

1. Shader "Unlit/ScreenSpaceUVDistortion"
2. {
3.     Properties
4.     {
5.         _Distortion ("Texture", 2D) = "white" {}
6.         _DistortionStrength ("Strength", Range(0,1)) = 0.5
7.     }
8.     SubShader
9.     {
10.         Tags { "Queue"="Transparent" "RenderType"="Transparent" }
11.         LOD 100
12.  
13.         GrabPass {}
14.  
15.         Pass
16.         {
17.             CGPROGRAM
18.             #pragma vertex vert
19.             #pragma fragment frag
20.  
21.             #include "UnityCG.cginc"
22.  
23.             struct appdata
24.             {
25.                 float4 vertex : POSITION;
26.                 float2 uv : TEXCOORD0;
27.             };
28.  
29.             struct v2f
30.             {
31.                 float4 pos : SV_POSITION;
32.                 float2 uv : TEXCOORD0;
33.                 float4 grabPos : TEXCOORD1;
34.             };
35.  
36.             sampler2D _GrabTexture;
37.  
38.             sampler2D _Distortion;
39.             float _DistortionStrength;
40.  
41.             v2f vert (appdata v)
42.             {
43.                 v2f o;
44.                 o.pos = UnityObjectToClipPos(v.vertex);
45.                 o.uv = v.uv;
46.                 o.grabPos = ComputeGrabScreenPos(o.pos);
47.                 return o;
48.             }
49.  
50.             half4 frag (v2f i) : SV_Target
51.             {
52.                 float4 distortionPos = float4(tex2D(_Distortion, i.uv).xy - 0.5, 0.0, 1.0);
53.                 float4 clipSpaceDistortion = UnityObjectToClipPos(distortionPos);
54.                 float4 grabDistortion = ComputeGrabScreenPos(clipSpaceDistortion);
55.                 grabDistortion.xy /= grabDistortion.w;
56.  
57.                 float2 grabUV = i.grabPos.xy / i.grabPos.w;
58.                 grabUV = lerp(grabUV, grabDistortion, _DistortionStrength);
59.  
60.                 half4 col = tex2D(_GrabTexture, grabUV);
61.                 return col;
62.             }
63.             ENDCG
64.         }
65.     }
66. }

The "UV texture" should be thought of as an encoded object space texture. The code assumes you're using the default quad mesh, which is aligned to the xy coordinates in a -0.5 to 0.5 range. Since the texture is a 0.0 to 1.0 value, we subtract 0.5 and it's now an "object space" position.

 

Its hard to explain how grateful I am
Thank you so much bgolus! You are the best. As always

 

@bgolus any thoughts on why this would behave differently on android? I'm using the shader graph implementation. I see that there are differences between clip space for dirextx and opengl, but I would assumed the shader compiler would spit out proper versions.

 

Yeah, there's some weirdness with screen space UVs between DirectX and OpenGL. The BIRP version should work properly between both APIs, but the shader graph might not since I'm doing the "ComputeGrabScreenPos" function manually, and without the additional API checks. See if it looks correct on OpenGL with that multiply by 1.0, 1.0 removed.

If it does, make a new Vector2 that uses 1.0 for x, and the Camera Node's Z Buffer Sign as the Y and multiply by that.

 

@bgolus that was it! Thanks so much!