Question Is doing a shader with an outer outline possible if the sprite touches the edge? It just cuts off

There's a ton of topics about doing outlines for sprites but there's always comments about people asking about this and there doesn't really seem to be an answer whether it's even possible.

Here's an example of an exterior outline failing on the sprite's edges

Now you could increase your image size from 16x16 or 32x32 to 18x18 or 34x34 but that feels dirty and it still limits you from doing something beyond another pixel beyond that as well. Plus if you use really nice tools like TexturePacker, they don't offer the ability to add padding inside the sprite itself, only padding outside of the sprite.

So is there an actual solution to this problem that has gone unanswered for many years?

Here's the code for an outer outline shader if it helps

Code (CSharp):

1. Shader "Custom/Outline (Outer)"
2. {
3.     Properties
4.     {
5.         _MainTex("Sprite Texture", 2D) = "white" {}
6.         _Color("Tint", Color) = (1,1,1,1)
7.     }
8.  
9.         SubShader
10.         {
11.             Tags
12.             {
13.                 "Queue" = "Transparent"
14.                 "IgnoreProjector" = "True"
15.                 "RenderType" = "Transparent"
16.                 "PreviewType" = "Plane"
17.                 "CanUseSpriteAtlas" = "True"
18.             }
19.  
20.             Cull Off
21.             ZWrite Off
22.             Blend One OneMinusSrcAlpha
23.  
24.             Pass
25.             {
26.                 CGPROGRAM
27.                 #pragma vertex vertexFunc
28.                 #pragma fragment fragmentFunc
29.                 #include "UnityCG.cginc"
30.  
31.             struct v2f
32.             {
33.                 float4 vertex   : SV_POSITION;
34.                 float2 uv : TEXCOORD0;
35.             };
36.  
37.  
38.             v2f vertexFunc(appdata_base v) {
39.                 v2f o;
40.                 o.vertex = UnityObjectToClipPos(v.vertex);
41.                 o.uv = v.texcoord;
42.  
43.                 return o;
44.             }
45.  
46.             sampler2D _MainTex;
47.             fixed4 _Color;
48.             float4 _MainTex_TexelSize;    // This is filled out by inspector, it gets size of
49.  
50.  
51.  
52.             fixed4 fragmentFunc(v2f IN) : COLOR // all caps cause its not just a name but a proper name
53.             {
54.                 half4 c = tex2D(_MainTex, IN.uv);    // Color at the current pixel
55.                 c.rgb *= c.a;                        // Make the RGB value equal to the alpha value
56.                 half4 outlineC = _Color;            // Color of outline that we decided to be (1,1,1,1)
57.  
58.                 fixed upAlpha = tex2D(_MainTex, IN.uv + fixed2(0, _MainTex_TexelSize.y)).a;
59.                 fixed downAlpha = tex2D(_MainTex, IN.uv - fixed2(0, _MainTex_TexelSize.y)).a;
60.                 fixed rightAlpha = tex2D(_MainTex, IN.uv + fixed2(_MainTex_TexelSize.x, 0)).a;
61.                 fixed leftAlpha = tex2D(_MainTex, IN.uv - fixed2(_MainTex_TexelSize.x, 0)).a;
62.  
63.                 fixed totalAlpha = min(ceil(upAlpha + downAlpha + rightAlpha + leftAlpha), 1);
64.                 outlineC.rgba *= totalAlpha;
65.                 return lerp(outlineC, c, ceil(totalAlpha * c.a));
66.             }
67.  
68.         ENDCG
69.         }
70.     }
71. }

Any help would be greatly appreciated, including from future googlers, thanks!

 

No it is impossible to draw outside the edges of a sprite because you need geometry to draw onto and if there is none you can not draw there.

However there might be ways to cheat around your issue. You can scale the vertices of your sprite outwards in the vertex shader to increase the size of the geometry and scale your sprite down in the fragment shader to retain the original size. Now you have some extra room to draw your outline. BUT scaling the vertices outwards is simple for a quad, but for tight sprite geometry this can be rather difficult because the normals will not point in the direction that you need so you need to calculate the right vector and you might be further constrained by two faces overlapping after being extruded.

 

Its just much easier to add that 1x1 padding, your problem is that you want something to come out of nothing. It might be dirty but its also the most simplest yet elegant solution to your problem.

 

Yeah it's a good idea to just expand, but with over 1000 files and all their sources, changing all that seems painful and not so sure I would justify it just for that feature.
Fleity's idea is neat. Got it to scale in the vertex function, not sure how to scale it back down in the fragment function though, as I just end up shifting the texture instead. Maybe there's a bunch more steps ahead like he mentioned that might be a bit too much, so maybe there's a different way to approach it..

How about inner outlines?
Fortunately for these, you don't have to go outside of your fragment space. Unfortunately inner outlines suffer the same issue where if they touch the sprite border, they too won't draw the inner outline.

If it's just a single sprite, you can just fix the edges like so:
fixed edgeX = (1 - floor(IN.uv.r + _MainTex_TexelSize.x)) * ceil(IN.uv.r - _MainTex_TexelSize.x);
fixed edgeY = (1 - floor(IN.uv.g + _MainTex_TexelSize.y)) * ceil(IN.uv.g - _MainTex_TexelSize.y);
and then multiply them at the end with the other directions, but it doesn't work with a spritesheet. Is there a way to determine the edge of not your texture, but your sprite in the fragment function? Or perhaps some other clever option to get around it?

Code (CSharp):

1. Shader "Custom/Outline (Inner)"
2. {
3.     Properties
4.     {
5.         _MainTex("Sprite Texture", 2D) = "white" {}
6.         _Color("Tint", Color) = (1,1,1,1)
7.     }
8.  
9.         SubShader
10.         {
11.             Tags
12.             {
13.                 "Queue" = "Transparent"
14.                 "IgnoreProjector" = "True"
15.                 "RenderType" = "Transparent"
16.                 "PreviewType" = "Plane"
17.                 "CanUseSpriteAtlas" = "True"
18.             }
19.  
20.             Cull Off
21.             ZWrite Off
22.             Blend One OneMinusSrcAlpha
23.  
24.             Pass
25.             {
26.                 CGPROGRAM
27.                 #pragma vertex vertexFunc
28.                 #pragma fragment fragmentFunc
29.                 #include "UnityCG.cginc"
30.  
31.             struct v2f
32.             {
33.                 float4 vertex   : SV_POSITION;
34.                 float2 uv : TEXCOORD0;
35.             };
36.  
37.  
38.             v2f vertexFunc(appdata_base v) {
39.                 v2f o;
40.                 o.vertex = UnityObjectToClipPos(v.vertex);
41.                 o.uv = v.texcoord;
42.  
43.                 return o;
44.             }
45.  
46.             sampler2D _MainTex;
47.             fixed4 _Color;
48.             float4 _MainTex_TexelSize;      
49.  
50.             fixed4 fragmentFunc(v2f IN) : COLOR
51.             {
52.                 half4 c = tex2D(_MainTex, IN.uv);
53.                 c.rgb *= c.a;
54.                 half4 outlineC = _Color;
55.                 outlineC.a *= ceil(c.a);
56.                 outlineC.rgb *= outlineC.a;
57.  
58.                 fixed upAlpha = tex2D(_MainTex, IN.uv + fixed2(0, _MainTex_TexelSize.y)).a;
59.                 fixed downAlpha = tex2D(_MainTex, IN.uv - fixed2(0, _MainTex_TexelSize.y)).a;
60.                 fixed rightAlpha = tex2D(_MainTex, IN.uv + fixed2(_MainTex_TexelSize.x, 0)).a;
61.                 fixed leftAlpha = tex2D(_MainTex, IN.uv - fixed2(_MainTex_TexelSize.x, 0)).a;
62.  
63.                 // For working with alpha
64.                 outlineC.rgb += (1- outlineC.a) * c.rgb;
65.                 outlineC.a = max(c.a, outlineC.a);
66.              
67.                 return lerp(outlineC, c, ceil(upAlpha * downAlpha * rightAlpha * leftAlpha));
68.  
69.             }
70.  
71.         ENDCG
72.         }
73.     }
74. }
75.  

Edit: The inner outline issue is from extruding the pixels of the sprite or having an adjacent sprite without padding. Inner outline issue is an easy fix, outer, not so much.

 

post what you have to far and I'll try to help you make it work

 

Just off the top of my head, set the texture to repeat, scale the mesh up 1 pixel, then query in the fragment neighborhood => if same color => outline

 

Yeah that's where fleity was leading me to, I think there's an extra step that fleity mentions though for outer outlines, where you might have to scale it back down.

So yeah admittedly later I discovered that the inner outline issue was simply due to extruding / not having a empty padding. Of course though the main goal was for outer outline which brings me back to fleity,

That's nice of you. Shaders are kind of black magic to me, but this is what I came up with. I used two different scales just to better understand what was going on. I get the scaledCoord in the fragment function but that only shifts the location on the texture. I guess it makes sense, it's the texture coordinates, but no amount of googling lead me to figure out how I can modify the actual size in the fragment function.

Code (CSharp):

1. Shader "Custom/Outline (Outer)"
2. {
3.     Properties
4.     {
5.         _MainTex("Sprite Texture", 2D) = "white" {}
6.         _SpriteScale ("SpriteScale", Range(-1, 10.0)) = 1.0
7.         _SpriteScale2 ("SpriteScale2", Range(-1, 10.0)) = 1.0
8.         _Color("Tint", Color) = (1,1,1,1)
9.     }
10.  
11.         SubShader
12.         {
13.             Tags
14.             {
15.                 "Queue" = "Transparent"
16.                 "IgnoreProjector" = "True"
17.                 "RenderType" = "Transparent"
18.                 "PreviewType" = "Plane"
19.                 "CanUseSpriteAtlas" = "True"
20.             }
21.  
22.             Cull Off
23.             ZWrite Off
24.             Blend One OneMinusSrcAlpha
25.  
26.             Pass
27.             {
28.                 CGPROGRAM
29.                 #pragma vertex vertexFunc
30.                 #pragma fragment fragmentFunc
31.                 #include "UnityCG.cginc"
32.  
33.             struct v2f
34.             {
35.                 float4 vertex   : SV_POSITION;
36.                 float2 uv : TEXCOORD0;
37.             };
38.  
39.             sampler2D _MainTex;
40.             fixed4 _Color;
41.             float4 _MainTex_TexelSize;    // This is filled out by inspector, it gets size of
42.             float _SpriteScale;
43.             float _SpriteScale2;
44.  
45.             v2f vertexFunc(appdata_base v) {
46.                 v.vertex.xyz *= _SpriteScale;
47.                 v2f o;
48.                 o.vertex = UnityObjectToClipPos(v.vertex);
49.                 o.uv = v.texcoord;
50.                 return o;
51.             }
52.  
53.             fixed4 fragmentFunc(v2f IN) : COLOR // all caps cause its not just a name but a proper name
54.             {
55.                 float2 scaledCoord = IN.uv * _SpriteScale2;
56.                 half4 c = tex2D(_MainTex, scaledCoord);    // Color at the current pixel
57.                 //half4 c = tex2D(_MainTex, IN.uv);    // Color at the current pixel
58.                 c.rgb *= c.a;                        // Make the RGB value equal to the alpha value
59.                 half4 outlineC = _Color;            // Color of outline that we decided to be (1,1,1,1)
60.  
61.                 fixed upAlpha = tex2D(_MainTex, IN.uv + fixed2(0, _MainTex_TexelSize.y)).a;
62.                 fixed downAlpha = tex2D(_MainTex, IN.uv - fixed2(0, _MainTex_TexelSize.y)).a;
63.                 fixed rightAlpha = tex2D(_MainTex, IN.uv + fixed2(_MainTex_TexelSize.x, 0)).a;
64.                 fixed leftAlpha = tex2D(_MainTex, IN.uv - fixed2(_MainTex_TexelSize.x, 0)).a;
65.  
66.                 fixed totalAlpha = min(ceil(upAlpha + downAlpha + rightAlpha + leftAlpha), 1);    // Only outer will be 0.  instead of 1, maybe c.a?
67.                 outlineC.rgba *= totalAlpha;
68.                 return lerp(outlineC, c, ceil(totalAlpha * c.a));
69.             }
70.  
71.         ENDCG
72.         }
73.     }
74. }

Thanks for the help

 

Your issue mainly sits in the first line of the fragment shader. What you have to understand is that UV coordinates start at the bottom left corner of your sprite with a value of x=0, y=0 and then go to 1, 1 in the top right corner.
Vertex coordinates on the other hand are relative to the objects pivot point (which for many sprites is the center).
You are already scaling the vertices correctly but to do the same with the UVs you have to offset them.

You can subtract 0.5 moving the UV range to -0.5 to +0.5 (UVs are not centered around 0, 0) and then scale them up and move them back in the correct range by adding 0.5 again. It might look at first if this does nothing but this is more or less the only part you were missing.

float2 scaledCoord = (IN.uv -0.5) * _SpriteScale + 0.5;

(* 2 - 1 and * 0.5 + 0.5 is also used very often to do this because this gives a clean -1 to 1 range)

This works for sprites with a centered pivot and their repeat mode set to clamp (you can also clamp the uv coords in the shader if that is more flexible for you). The result looks like this:



I didn't include a 1px transparent border in the sprite therefore as the last pixel is just repeated over and over we see it extended instead instead of drawing the outline there but that can be solved further down in the shader too.

Full function looks like this

Code (CSharp):

1.             fixed4 fragmentFunc(v2f IN) : COLOR // all caps cause its not just a name but a proper name
2.             {
3.                 float2 scaledCoord = (IN.uv -0.5) * _SpriteScale + 0.5;
4.                 half4 c = tex2D(_MainTex, scaledCoord);     // Color at the current pixel in the texture
5.                 c.rgb *= c.a;                               // Multiply the RGB value by the alpha value, darkens half transparent pixels
6.                 half4 outlineC = _Color;                    // Color of outline that we decided to be (1,1,1,1)
7.  
8.                 half outlineWidth = _SpriteScale - 1;
9.                
10.                 fixed upAlpha = tex2D(_MainTex, IN.uv + fixed2(0, _MainTex_TexelSize.y * outlineWidth)).a;
11.                 fixed downAlpha = tex2D(_MainTex, IN.uv - fixed2(0, _MainTex_TexelSize.y * outlineWidth)).a;
12.                 fixed rightAlpha = tex2D(_MainTex, IN.uv + fixed2(_MainTex_TexelSize.x * outlineWidth, 0)).a;
13.                 fixed leftAlpha = tex2D(_MainTex, IN.uv - fixed2(_MainTex_TexelSize.x * outlineWidth, 0)).a;
14.  
15.                 // mask for pixels outside original sprite range
16.                 fixed factor = outlineWidth * 1/_SpriteScale * 0.5;
17.                 fixed outsideUV = 1;
18.                 outsideUV *= step(IN.uv.x, 1 - factor);
19.                 outsideUV *= step(factor, IN.uv.x);
20.                 outsideUV *= step(IN.uv.y, 1 - factor);
21.                 outsideUV *= step(factor, IN.uv.y);
22.                
23.                 fixed totalAlpha = min(ceil(upAlpha + downAlpha + rightAlpha + leftAlpha), 1); // Only outer will be 0.  instead of 1, maybe c.a?
24.                 outlineC *= totalAlpha;
25.                 half4 finalColor = lerp(outlineC, c, ceil(totalAlpha * outsideUV * c.a));
26.  
27.                 return finalColor;
28.             }

All the step functions create a mask with a value of either zero or one where we are inside of the original UVs. Multiplied to the mask value in the last line this creates a solid outline. (I took the liberty of fixing a small mistake where an outline width of zero or a sprite scale of 1 would still create a small outline because texelsize needed to be multiplied by the outline width).

result looks like this:




And now a huge BUT
This might look super nice but it has it's drawbacks. As mentioned above this works well because the scaling point for the vertices aligns very easily with the centerpoint of the UVs if those are setup differently we need to account for that in the shader / material. Not nice but doable.
Non rectangular / circular sprites are more difficult because you can not just scale up the vertices because at some point you are going to get overlaps and z-fighting.
Lastly the biggest no go of this technique as it is is: it does not work with sprite atlases.
sprites on an atlas have their UVs altered in a way that the UVs are not longer 0 to 1 but something like float2(0.345, 0.12) to float2(0.54234, 0.15969). And as far as I know you don't easily have access to normalized UV coordinates in the shader alone. You can store them on a material property but that is one specific value per material (and you rather need per sprite).

Issue 1 and 2 are solvable or at least somewhat work-around-able but 3 is a dealbreaker for using this on a large scale if you ask me.

 

Oh very clever! Thanks a lot, that helps me understand it a lot better. I noticed for sprites it stays in place and for images like UI it shifts location with the scaling (assuming not at 0,0), wonder why, either way not a big deal since it's more a sprite shader.
That's a shame that we can't get the normalized UV coordinates from the just the shader since most of my art is in a sprite sheet, but I do have some that are not in one and could see this being potentially useful regardless. Plus I can imagine some uses of that mask in different ways, so definitely adding it to my repertoire. Very much appreciated fleity! Thanks a bunch!

 

https://forum.unity.com/threads/help-to-find-an-asset-solution.755273/#post-5246960

 

Uhh as usual bgolus's answer is pure gold.
Since you are on the build-in pipeline you can absolutely use multi pass shaders like in his link to solve the problem.
With a few render objects features it would probably even work in URP.

(Another approach could be to use a different setup for outlines alltogether if you were willing to batch process all your sprites to give them a very soft outside glow (hard oversimplification of turning their alpha channel into a SDF) and use a shaders with step functions to mask the outline out from the SDF. Soo many possibilites all with their own costs / drawbacks ^^)

 

oh man, i totally forgot you could you use normal extrusion second pass with sprites as well

 

There are heroes that people aspire to become, then there are people like bgolus that no mortal could even dream of becoming.

Amazing! Thank you for that link. For any other future googlers, you can add 'Cull Off' next to 'ZWrite Off' to allow sprite flipping. Destination Alpha one also seems to play nicer with UI Image ordering as well.

 

So I'm currently dealing with a similar problem.

I'm using Shader Graph for 2D, and I have a sprite where it touches the sprite border, however i've managed to make a shader where I have this previewed output:



However, when I go into the game, this is what happens:


Why is this happening?

 

No idea.

Post your shader.

 

Is it possible to accomplish a similar effect in shader graph? I'm struggling with the same issue as the OP using URP. I've tried multiplying the object position to scale the outline, but it's not centered properly.