Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Bump offset/parallax mapping window shader

Discussion in 'Shaders' started by Piggyback, May 27, 2016.

  1. Piggyback

    Piggyback

    Joined:
    May 22, 2016
    Posts:
    7
    Hi, I'm trying to write a window shader that offsets an interior texture based on viewing angle.
    Here is an example of how to do this in unreal:

    The offset is done by a node called bump offset which is given the viewing direction and the texture coordinates. This is what is called parallax mapping, but with a static height instead of a hightmap.

    How would I go about writing this function in cg?
    I've tried this to translate an open gl function I found here: http://www.learnopengl.com/#!Advanced-Lighting/Parallax-Mapping

    like this:
    Code (CSharp):
    1.  float2 ParallaxMapping(float2 texCoords, float3 viewDir)
    2.  {
    3.  float height = -1;
    4.  float2 p = viewDir.xy * (height * 0.1);
    5.  return texCoords - p;
    6.  }


    and then in the vertex program set the view dir to:
    Code (CSharp):
    1.  
    2.  mul(modelMatrix, input.vertex).xyz - _WorldSpaceCameraPos;
    3.  
    and then in the fragment program do this to the uv:
    Code (CSharp):
    1.  
    2.  uv = ParallaxMapping(input.tex.xy, input.viewDir);
    3.  
    and it kind of works, looks great when looking up and down but moves inverted to how I want it when looking from left to right, and it moves unpredictably when the window is rotated. I think I need to take the normal of the windowpane into account somehow.

    If you have done this before I'd love to get some tips to point me in the right direction! :)
     
  2. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,473
    This is an interesting technique! It kept me thinking the whole Sunday, I hate when that happens :)

    Now about 6 hours later, I put together a shader with three different approaches which I came up with. You can find a download link to the project at the bottom of this post.

    Here is how the result looks:


    Code (CSharp):
    1. //
    2. // download from http://www.console-dev.de/bin/Unity_Shader_Fake_Interior.zip
    3. //
    4. // this shader is part of an answer to the following forum post:
    5. // http://forum.unity3d.com/threads/bump-offset-parallax-mapping-window-shader.407091/
    6. //
    7. // I believe this approach is rather unconventional.
    8. // My idea was that I could use the clip space coordinates to offset the mesh texture coordinates.
    9. // clip space is in range -1 to +1
    10. // if the vertex is at the left screen edge, x would be -1 in clip space
    11. // if the vertex is at the right screen edge, x would be +1 in clip space
    12. // the sample principle applies for the y coordinate.
    13. //
    14. // if the interior quad is also mapped in this fashion (top-left = 0,0 and bottom-right=1,1)
    15. // it should be possible to simply add the clip-space coordinates to the texture coordinates
    16. // and we have the parallax effect already already working.
    17. //
    18. // however, if we look to the left we actually want to reveal more of the left part of the texture.
    19. // just adding the clip-space coordinates would simple move the texture to the left, revealing more of
    20. // the right texture part. so I simply changed the sign (subtract rather than add) to scroll it into
    21. // the opposite direction.
    22. //
    23. // since we just modify texture coordinates, we can also do just everything in the vertex shader \o/
    24. //
    25. // in this file you can find 3 different tests, showing my hackery.
    26. //
    27.  
    28. Shader "Custom/Interior"
    29. {
    30. Properties {
    31.     _Color ("Main Color", Color) = (1,1,1,1)
    32.     _MainTex ("Base (RGB)", 2D) = "white" {}
    33.  
    34.     // The maximum parallax value should not be greater than
    35.     // the uv border in the mesh. Usually you would uv map a quad from:
    36.     // top-left=0,0 to bottom-right=1,1
    37.     // however, the shader moves the uv's by the specified _Parallax value.
    38.     // in order to never go outside the 0,0 .. 1,1 range, you "add a border to the uvs in the mesh".
    39.     // To properly support a maximum parallax value of 0.25, you would uv map the quad as:
    40.     // top-left=0.25,0.25 to bottom-right=0.75,0.75
    41.     // if the shader adds the _Parallax value to the uvs, we have the full uv range again.
    42.     _Parallax ("Parallax", Range (0, 1)) = 0.25
    43. }
    44.  
    45. CGINCLUDE
    46. sampler2D _MainTex;
    47. float4 _MainTex_ST;
    48. fixed4 _Color;
    49. float _Parallax;
    50.  
    51. struct Input
    52. {
    53.     float2 st_MainTex;
    54. };
    55.  
    56. // test 2 seems to work well and I understand why it is working
    57. #define TEST_2
    58.  
    59. #if defined(TEST_1)
    60. inline float2 InteriorUV(float4 vertex, float3 normal, float2 texcoord)
    61. {
    62.     float4 clipSpace = mul(UNITY_MATRIX_MVP, vertex);
    63.     clipSpace.y = -clipSpace.y;
    64.     clipSpace = normalize(clipSpace);
    65.  
    66.     // applys texture scaling or displacement using the screen center as origin.
    67.     // this results in undesired scretching at the screen borders.
    68.     // you can still compensate by adjusting the texture tiling and offset in the material.
    69.     // to get a _Parallax value of 0.25 to apply the scaling from the quad center, you would need
    70.     // to modify the material settings as followed:
    71.     //   Tiling X = 1.5    Y = 1.5
    72.     //   Offset X = -0.25  Y = -0.25
    73.     float2 offset = clipSpace.xy * _Parallax;
    74.     return texcoord - offset;
    75. }
    76. #endif
    77.  
    78. #if defined(TEST_2)
    79. inline float2 InteriorUV(float4 vertex, float3 normal, float2 texcoord)
    80. {
    81.     float4 clipSpace = mul(UNITY_MATRIX_MVP, vertex);
    82.     clipSpace.y = -clipSpace.y;
    83.     clipSpace = normalize(clipSpace);
    84.  
    85.     // This is the same as TEST_1, but we modify the texcoord so that the
    86.     // displacement occurs not using the screen center anymore, but the
    87.     // quads center. this seems to look nice at all angles and positions.
    88.     float2 offset = clipSpace.xy * _Parallax;
    89.     return (texcoord + texcoord * _Parallax * 2) - _Parallax - offset;
    90. }
    91. #endif
    92.  
    93. #if defined(TEST_3)
    94. // another approach that seems to work, figured out through trial&error.
    95. // I don't understand entirely why it works though :)
    96. // it seems to work really nice though.
    97. inline float2 InteriorUV(float4 vertex, float3 normal, float2 texcoord)
    98. {
    99.     float3 distance = normalize(mul((float3x3)UNITY_MATRIX_MV, reflect(ObjSpaceViewDir(vertex), normal)));
    100.  
    101.     float2 offset = distance.xy * _Parallax;
    102.     return texcoord - offset;
    103. }
    104. #endif
    105.  
    106.  
    107. void vert(inout appdata_full v, out Input o)
    108. {
    109.     UNITY_INITIALIZE_OUTPUT(Input, o);
    110.  
    111.     float2 texcoord = TRANSFORM_TEX(v.texcoord, _MainTex);
    112.     o.st_MainTex = InteriorUV(v.vertex, v.normal, texcoord);
    113. }
    114.  
    115. void surf (Input IN, inout SurfaceOutput o)
    116. {
    117.     fixed4 c = tex2D(_MainTex, IN.st_MainTex) * _Color;
    118.     o.Albedo = c.rgb;
    119.     o.Alpha = c.a;
    120. }
    121. ENDCG
    122.  
    123.  
    124. SubShader
    125. {
    126.     Tags { "RenderType"="Opaque" }
    127.     LOD 500
    128.  
    129.     CGPROGRAM
    130.     #pragma surface surf Lambert vertex:vert
    131.     #pragma target 3.0
    132.     ENDCG
    133. }
    134.  
    135.  
    136.  
    137. FallBack "Legacy Shaders/Diffuse"
    138. }
    139.  

    I believe my approach is rather unconventional, but I also know nothing about interior mapping except for the video you posted.

    My idea was that I could use the clip space coordinates to offset the mesh texture coordinates.
    Clip space is in range -1 to +1. If the vertex is at the left screen edge, x would be -1 in clip space. If the vertex is at the right screen edge, x would be +1 in clip space. The same principle applies for the y coordinate.

    If the interior quad is also mapped in this fashion (top-left = 0,0 and bottom-right=1,1 (but you need a border, see shader comments)) it should be possible to simply add the clip-space coordinates to the texture coordinates and we have the parallax effect already working.

    However, if we look to the left we actually want to reveal more of the left part of the texture. Just adding the clip-space coordinates would move the texture to the left, moving in more of the right texture part. So I simply changed the sign (subtract rather than add) to scroll it into the opposite direction.

    Since we just modify texture coordinates, we can also do everything in the vertex shader \o/

    You can find three different approaches to this problem in in "Assets/Interior.shader", basically 3 different tests showing my hackery.

    Here is the Unity 5.3 compatible project with all the assets shown in the video.
    http://www.console-dev.de/bin/Unity_Shader_Fake_Interior.zip
     
    Last edited: May 29, 2016
    Alverik and mgear like this.
  3. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,110
    It's a nice idea, it's not interior mapping as seen in bioshock infinite and friends, but it does seem much lighter on the gpu and not computationally expensive. I can see this being sort of ideal for mid and far lods.

    Check some of the tricks out on this site, here's one of them: https://simonschreibt.de/gat/windows-ac-row-ininite/
     
    Peter77 likes this.
  4. Piggyback

    Piggyback

    Joined:
    May 22, 2016
    Posts:
    7
    Hello again!
    Yours is an intresting solution, but it doesn't really take the position of the camera into account which makes it break when you strafe.

    I recently had time to start thinking about this again (got my degree in 3d art, officially unemployed!), and here is what I've come up with:



    If you take the normal direction of the verticies of the window pane and their position and the camera position all in world space shouldn't you be able to convert the camera position into a kind of vertex space that is relative to that vertex's position and rotation? Then you could simply take the x and y values of this new point and use them as offset values of the uv after setting the min and max values of the offset and a scaling factor.

    The problem is that my vector math is both rusty, and basic. I figured that the math needed would be close to when you calculate view space coordinates for verticies, so I tried to converting some code I found online:



    This is the method I use to create the vertex space matrix


    Code (CSharp):
    1. float4x4 viewFrom(float3 nor,float3 up, float3 vpos){
    2.  
    3.             //normal of the vertex, the way the matrix should "look"
    4.             float3 n = normalize(nor);
    5.             //up direction of the camera, I assume (0,1,0) when I send it
    6.             float3 v = normalize(up);
    7.  
    8.             //third angle, looking to the side, cross product of the two
    9.             float3 u = cross(nor,up);
    10.  
    11.             float4x4 m = float4x4(u.x, u.y, u.z, 0.0,
    12.                                   v.x, v.y, v.z, 0.0,
    13.                                   n.x, n.y, n.z, 0.0,
    14.                                   0.0, 0.0, 0.0, 1.0);
    15.  
    16.             float4x4 mpos = float4x4(1.0, 0.0, 0.0, vpos.x,
    17.                                        0.0, 1.0, 0.0, vpos.y,
    18.                                       0.0, 0.0, 1.0, -vpos.z,
    19.                                        0.0, 0.0, 0.0, 1.0);
    20.  
    21.                m = mul(m,mpos);
    22.            
    23.             return m;
    24.         }
    and then in my vertex shader I do this:

    Code (CSharp):
    1. //get the matrix
    2. float4x4 vertexMatrix = viewFrom(normalDirection.xyz,float3(0.0,1.0,0.0),vertexWorldPos.xyz);
    3.  
    4. //convert the position of the camera to this "vertex space"
    5.              float3 relPos =    mul(vertexMatrix,_WorldSpaceCameraPos).xyz;
    And finally I scale and clamp the value to a better scrolling offset that can be used to offset the uv:

    Code (CSharp):
    1. float2 offsetUV = ((relPos).xy*_OffsetScale);
    2.              offsetUV.x = clamp(offsetUV.x,-_MaxOffset,_MaxOffset);
    3.              offsetUV.y = clamp(offsetUV.y,-_MaxOffset,_MaxOffset);

    This works much better than my first try, but it still doesn't work when rotated, and behaves differently when moved further away. Am I missing something obvious, or have I completely misjudged the problem?
     
    Alverik likes this.
  5. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    6,450
    What you want for this kind of effect is the tangent space view direction.

    If you're using surface shaders I think you can just add viewDir to your input struct and you'll get the tangent space view direction in the surf function. Then you just need something like this:

    float2 interiorUV = IN.uv_MainTex + (IN.viewDir.xy / IN.viewDir.z) * _OffsetScale;
     
  6. Piggyback

    Piggyback

    Joined:
    May 22, 2016
    Posts:
    7
    Thank you! That worked out perfectly! I knew I was overcomplicating things! :)
    Just one question, what is the point of the devision by the z-value? It seems to work without it but acts a little differently..

    Oh and if somebody finds this and wonders how to do it in cg, here is how I got the tangent space view direction:


    Code (CSharp):
    1. TANGENT_SPACE_ROTATION;
    2. float3 viewDir  = mul(rotation,  ObjSpaceViewDir(v.vertex));
    You just need to make sure you've included the tangent in your vertex input.
     
    Alverik likes this.
  7. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    6,450
    If you use a normalized view direction vector, what you're likely starting with, it is going to be essentially moving the offset around in a circle / hemisphere. Dividing by the z component projects the vector onto a unit plane. The normalized vector is fine too use as it gets you most of the parallax look, and gets you a built in "max" offset, but the other way is more accurate. You may want to fade out the effect at extreme angles.
     
  8. Alverik

    Alverik

    Joined:
    Apr 15, 2016
    Posts:
    384
    Any chance we can see the final code? I love the effect, but I'm not a shader coder... I doubt I can get it to work by myself...