Search Unity

outline by pushing vertex away by a normal.

Discussion in 'Shaders' started by CauseMoss, Dec 4, 2019.

  1. CauseMoss

    CauseMoss

    Joined:
    Sep 16, 2017
    Posts:
    20

    I want to achieve a simple outline. you know: scale original object a bit and render it before the actual object. Standard stuff. But I want to do it by adding a normal (multiplied by a user defined factor ) to the mesh vertex position (pushing the vertex out in the normal direction), which should guarantee that the outline width will not depend on object size. unlike when you just scale vertex by a factor like 1.2 to make it a 20% bigger so the bigger objects have bigger outlines and in that case you have assume that the mesh origin is in the actual center. so my approach seems to be better, but it gives me the same frailty. the thick objects have thicker outline which i completely don't understand. where's the fault in my thinking?

    Code (CSharp):
    1.                 v2f o;
    2.                 float3 ver = v.vertex.xyz + v.normal.xyz * _OutlineWidth;
    3.                 o.vertex = UnityObjectToClipPos(ver);
     
  2. mouurusai

    mouurusai

    Joined:
    Dec 2, 2011
    Posts:
    350
    Try to normalize push direction
    float3 ver = v.vertex.xyz + normalize(v.normal.xyz) * _OutlineWidth;
     
  3. CauseMoss

    CauseMoss

    Joined:
    Sep 16, 2017
    Posts:
    20
    it's normalized by default, but i tried it just to be sure.
     
  4. mouurusai

    mouurusai

    Joined:
    Dec 2, 2011
    Posts:
    350
    I suppose you use scaled objects? Try push in world space then. Instead of use UnityObjectToClipPos macros multiply vertex position with object to world matrix(you get positions in world space), transform normals to word space, push positions, then multiply by vp matrix.
     
    Last edited: Dec 4, 2019
    CauseMoss likes this.
  5. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,352
    The vertex position you pass into the
    UnityObjectToClipPos()
    function is assumed to be in object space (hence the name of the function). Since it's in object space, it'll be affected by the object's scale. So if you move the object space vertex by 1 unit along the object space normal, if the object is scaled 200% it's now going to be 2 units when rendered.

    What you need to do instead is make sure you're moving the vertex position in world space. The
    UnityObjectToClipPos()
    function looks like this:
    Code (csharp):
    1. inline float4 UnityObjectToClipPos(in float3 pos)
    2. {
    3.     return mul(UNITY_MATRIX_VP, mul(unity_ObjectToWorld, float4(pos, 1.0)));
    4. }
    There's actually a little more in that function if you look at the source files, but that's for special VR rendering that 99% of people will never use, so I removed it. The main two things is the two
    mul()
    calls and the matrices they're using. One is transforming the object space position into world space, and the second is transforming the world space into clip space. If you want to know what clip space is, search online for homogeneous clip space and enjoy that rabbit hole, or search for some of my posts on this forum on the topic. TLDR, you can think of it as screen space for simplicity (but it's totally not).

    So for a world space outline, we can skip using the built in function and instead call each
    mul()
    separately and modify the world space before transforming to clip space.
    Code (csharp):
    1. float4 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0));
    2. float3 worldNormal = UnityObjectToWorldNormal(v.normal);
    3. worldPos.xyz += worldNormal * _OutlineWidth;
    4. o.vertex = mul(UNITY_MATRIX_VP, worldPos);
    Basically, what @mouurusai said.
     
    CauseMoss likes this.
  6. CauseMoss

    CauseMoss

    Joined:
    Sep 16, 2017
    Posts:
    20
    Yeah! the scale of the objects was the issue!
    Thank you both!
     
  7. DreamingWorld

    DreamingWorld

    Joined:
    Dec 29, 2013
    Posts:
    30
    Hey! I need I dig into a topic from long ago..
    But I try to do the same thing, I'm a newbie in shader coding.. but until now it worked good enough ha ha -

    I tried to implement the world normal solution to my shader.. and met with a strange result!
    upload_2022-11-15_9-22-1.png
    Like it's not totally bugged shader ? it have the outlines I expected, but ...
    Also I got the error :
    UnityObjectToWorldNormal: no matching 1 parameter function

    And also
    invalid subscript 'normal'

    Here is the outline area just for reference...
    Code (CSharp):
    1.  Pass
    2.             {
    3.                 Cull Front
    4.                 CGPROGRAM
    5.                 #pragma vertex vert
    6.                 #pragma fragment frag
    7.                 #include "UnityCG.cginc"
    8.                 fixed4 _StrokeColor;
    9.                 float _StrokeWidth;
    10.                 struct appdata
    11.                 {
    12.                     float4 vertex:POSITION;
    13.                 };
    14.                 struct v2f
    15.                 {
    16.                     float4 clipPos:SV_POSITION;
    17.                 };
    18.                 v2f vert (appdata v)
    19.                 {
    20.                     v2f o;
    21.                    // o.clipPos=UnityObjectToClipPos(v.vertex*_StrokeWidth);
    22.                    float4 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0));
    23.                    float3 worldNormal = UnityObjectToWorldNormal(v.normal);
    24.                    worldPos.xyz += worldNormal * _StrokeWidth;
    25.                    o.vertex = mul(UNITY_MATRIX_VP, worldPos);
    26.  
    27.                     ///return o;
    28.                 }
    29.                 fixed4 frag (v2f i) : SV_Target
    30.                 {
    31.                     return _StrokeColor;
    32.                 }
    33.                 ENDCG
    34.             }
    If you could help, it would be super useful!
    Thank you :D
     
  8. wwWwwwW1

    wwWwwwW1

    Joined:
    Oct 31, 2021
    Posts:
    769
    Hi, that's because there's no variable called "normal" in "v" (struct appdata).

    The information stored in a mesh's vertices will be passed to the "appdata" structure. (select a mesh and see its properties in the inspector panel)

    You need to add the normal to appdata (see how standard shader does this):
    Code (CSharp):
    1. struct appdata
    2. {
    3.     float4 vertex : POSITION;
    4.     half3 normal : NORMAL;
    5. };
     
  9. DreamingWorld

    DreamingWorld

    Joined:
    Dec 29, 2013
    Posts:
    30
    Hey again! thank you for your answer, however, now it says
    invalid subscript 'vertex'

    But it already declared in App data, no ? what's going on ?

    Code (CSharp):
    1.  Pass
    2.             {
    3.                 Cull Front
    4.                 CGPROGRAM
    5.                 #pragma vertex vert
    6.                 #pragma fragment frag
    7.                 #include "UnityCG.cginc"
    8.                 fixed4 _StrokeColor;
    9.                 float _StrokeWidth;
    10.                 struct appdata
    11.                 {
    12.                     float4 vertex:POSITION;
    13.                     half3 normal : NORMAL;
    14.                 };
    15.                 struct v2f
    16.                 {
    17.                     float4 clipPos:SV_POSITION;
    18.                 };
    19.                 v2f vert (appdata v)
    20.                 {
    21.                    v2f o;
    22.                    // o.clipPos=UnityObjectToClipPos(v.vertex*_StrokeWidth);
    23.                    float4 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0));
    24.                    float3 worldNormal = UnityObjectToWorldNormal(v.normal);
    25.                    worldPos.xyz += worldNormal * _StrokeWidth;
    26.                    o.vertex = mul(UNITY_MATRIX_VP, worldPos);
    27.                     return o;
    28.                 }
    29.                 fixed4 frag (v2f i) : SV_Target
    30.                 {
    31.                     return _StrokeColor;
    32.                 }
    33.                 ENDCG
    34.             }
    and if I switch to :
     v.vertex = mul(UNITY_MATRIX_VP, worldPos);

    I got

    Output variable vert contains a system-interpreted value (SV_POSITION) which must be written in every execution path of the shader. Unconditional initialization may help.

    But I don't know where I put a condition for the initialisation of app data's vertex infos ..
     
    Last edited: Nov 15, 2022
  10. wwWwwwW1

    wwWwwwW1

    Joined:
    Oct 31, 2021
    Posts:
    769
    Hi, so sorry for the late reply.

    This error should be related to line 26.

    "o" is a "v2f" structure, which does not contain any variable called "vertex", so compiler throws an error.

    What you need to do is changing from "o.vertex" to "o.clipPos".


    And I found that there're some strange lines.

    "worldPos" should be a float3 (instead of float4) containing coordinates of xyz-axes.

    You can refer to the format in Shader Graph's space transformation functions:
    TransformObjectToWorld.jpg

    I saw that the shader multiplies "float3/4 worldPos" with "float4x4 UNITY_MATRIX_VP" (World Space to Clip Space matrix).

    (refer to Shader Graph's functions)
    For 3-component directions & 4x4 matrices, you need to:
    Code (CSharp):
    1. newDirection = mul((float3x3)matrix, direction);
    For 3-component positions & 4x4 matrices, you need to:
    Code (CSharp):
    1. newPosition = mul(matrix, float4(position, 1.0));
    This error told you that there's no value in SV_POSITION (clipPos).
    SV_POSITION is a must needed variable for fragment (pixel) shaders.

    The x and y component (range from -1 to 1) represents the screen coordinate of this fragment (pixel).

    The z component is the depth value of the current pixel.

    The w component is related to projection.

    Without this variable, the fragment shader won't be able to render pixels.
     
    DreamingWorld likes this.
  11. DreamingWorld

    DreamingWorld

    Joined:
    Dec 29, 2013
    Posts:
    30
    Thank you! Now it works, and I think I start to better understand how shader coding works :D !
     
    wwWwwwW1 likes this.