Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice

Edge Detection via Shader not Image Effect

Discussion in 'Shaders' started by RuinsOfFeyrin, Nov 19, 2015.

  1. RuinsOfFeyrin

    RuinsOfFeyrin

    Joined:
    Feb 22, 2014
    Posts:
    785
    Hello Everyone,

    So I've been playing around with the unity edge detection image effect script but cant get it to provide the look i want. First I can only get the sobel depth and sobel depth thin to work, the other options show no edge detection at all, no matter what i change the values to.

    The Sobel Depth look ok, but pretty much misses any edge that doesn't have a huge distance difference (which is like 75% of the angles).

    I would like to highlight EVERY edge of a given object, and I'm looking to do this in the shader. Ive done some googling and some forum searching, but i always end up getting pointed in the direction of the edge detection script that i don't want to use.

    Does anyone have any ideas how to go about achieving this within the shader itself? Perhaps some links for reference? Im pretty sure i get the basic idea, im just new to shaders so im not sure how to go about it.

    I think i need to in my first pass create a secondary texture that i store the normal of the texel in. In a second pass i need to read this texture and somehow determine the texels around it and check for variations in the normal values. Variations outside a certain range mean its an edge, and i should change the pixel color to my edge color.

    Ill be honest though, i don't get shaders enough to understand passes, or how to get values from a texture outside the current on screen texel im processing in the fragment function.

    Well, hopefully someone can shoot me off in the right direction, thanks a head of time.
     
    ina likes this.
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,818
    There are a few main ways to go about drawing outlines / edges on objects.

    Fresnel - Use a clamped dot(view normal, view direction).
    Pros: Works in shader without a separate pass. Will highlight almost every edge on smooth shapes. Cheap. Works on transparent objects.
    Cons: Difficult to control line thickness as its dependent on the angle of the surface to the camera rather than the depth and the roundness of the surface. Impossible to have consistent line thickness. Doesn't work well on hard edged shapes or low poly meshes. Lines are only drawn on the "inside" edge.

    Push Hull / Mesh Doubling - Duplicate the mesh and expand along surface or view normals, full front faces. Unity's toon outline shader works this way.
    Pros: Cheap on low poly characters. Works on pretty much every device. Gives a lot of control for line thickness. Lines are outside the edge.
    Cons: Lines still won't be constant, and only works on smooth meshes. Hard edges will create holes in the outline. Thicker outlines can make smaller details and concave edges loose their outline as its dependent on the outline mesh not intersecting other geometry.

    Edge Detection (Depth based) - Test depth between multiple screen pixels.
    Pros: Draws edges on everything that's opaque. Precise pixel size control over line width.
    Cons: More expensive than other options, but constant cost regardless of geometric complexity. Will "edge" surfaces that are simply angled away from the camera because the depth discontinuity between pixels is high enough on glancing angles without heavy depth biasing and then will miss seemingly obvious edges because of the bias. Each pixel of thickness increases the cost exponentially. Doesn't

    Edge Detection (Depth & Normal based) - Test depth and normal between screen pixels.
    Pros: Everything above, plus catches edges depth only solutions don't.
    Cons: Even more expensive.

    Geometry Shader Edge Detection - Test adjacent polygons to find corners, construct outline geometry.
    Pros: Catches every single edge, guaranteed. Complete control over outline thickness. As close to hand drawn edging as you can get.
    Cons: Super, super expensive, especially on high poly objects. Also, doesn't work at all in Unity since they don't provide adjacency information to the geometry shader.


    Geometry shader based outlines are kind of the holy grail of outlines, but as I said they're stupidly expensive (a 100,000 polygon model can bring a modern computer to it's knees), and they don't work in Unity.

    So, lets go back to Edge Detection. The depth & normals (Roberts Cross as it's called in Unity's Image Effect) method is really quite good, but requires you're using the deferred rendering path, or your camera has to render depths and normals for all opaque scene geometry. Obviously that's not working for you for some reason.

    Unity's Sobel implementation misses a ton of edges because of how it handles the issues of depth discontinuity between pixels on glancing angle geometry. The depth disparity needs to be greater and greater the further away something is so edges disappear on small details as they get further away. This works for the style a lot of people are going for when doing outlines. You can write your own by not increasing the disparity required in the distance, but you might start running into issues of depth precision and that glancing angle issue I mentioned before. However, unless you need really thick outlines, this really is the best you can get.

    If you don't want to use an image effect, you can do it as an extra pass in the shader. This can let you limit outlines to a specific object instead of applying it to an entire scene. This is actually something I've been working on lately. You can only do the depth based edge detection when doing this, unless you're using deferred, but it offers a bit more control than the other options.
     
  3. RuinsOfFeyrin

    RuinsOfFeyrin

    Joined:
    Feb 22, 2014
    Posts:
    785
    Hey, Thank you for the reply.

    I mainly want to detect the edges of geometry that is cube shaped, which is why i was assuming some sort of normal test would be the best approach.

    What you are talking about in your last paragraph is exactly what I would like to achieve, i just have no clue where to even begin.
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,818
    If you've just got a cube and you want to highlight all of the edges, there's a bunch of ways to do that. The easiest way is to just have another texture with the lines in it. None of this fancy shader stuff.

    Most of the techniques above are more about detecting outline and overlap edges rather than the internal edges. Since internal edges like this are "constant" in that the don't depend on the camera facing people generally just draw them into their textures. The above are to catch the edges that are view dependent.

    You could do something like an image effect using the normals, but that's spending a lot of resources on something that doesn't require it.

    Some alternative approaches would be using distance fields or modified barycentric coordinates stored as vertex colors or UVs. Ultimately these options are all pretty similar to just drawing the lines in a texture as they require authoring the content ahead of time rather than something that can be done 100% in shader. Well, you can do wireframe stuff purely in shader using geometry shaders, there's a free example on the Asset store, but it's not quite what you want.
     
  5. RuinsOfFeyrin

    RuinsOfFeyrin

    Joined:
    Feb 22, 2014
    Posts:
    785
    @bgolus

    I guess i should of clarified about the shapes of the mesh. They are "Cube-ish". Think of tetris for example. All the shapes were made from smaller cubes. This is essentially how the mesh is generated. If i were to put the edge in the texture i would be outlining the smaller cubes, not the face edges of the mesh. Does that make sense?
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,818
    Then your only option, really, is to use an image effect of some kind. If the surface is made up of separate cubes, there's nothing a regular shader, or even a geometry shader, can do as these do not know about other faces.

    If you only want to render edges on a specific object you'll need to render out a mask and that object's normals to a full screen texture and then do an image effect pass on that. There's another thread that has a nice image effect for getting all the hard edges that just popped up too.

    http://forum.unity3d.com/threads/image-effect-edge-detect-normals-colours-rel.310280/
     
  7. RuinsOfFeyrin

    RuinsOfFeyrin

    Joined:
    Feb 22, 2014
    Posts:
    785
    @bgolus

    YES YES YES! You are the man(or women)! That is exactly the effect i wanted to achieve! Thank you.
     
  8. Doidel

    Doidel

    Joined:
    Jul 10, 2013
    Posts:
    12
    I landed here on my search for edge detection for specific objects, and then decided to make depth&normals edge detection based on a DepthNormals RenderTexture I render first. In this video I'm explaining what I did:


    Sorry for the necro, but I think it's worth it, it would have helped me
     
    wxxhrt, Xerrok and PedroHN like this.
  9. Honorsoft

    Honorsoft

    Joined:
    Oct 31, 2016
    Posts:
    81
    I came across this post looking for a way to do a 'sonar' effect (like in Batman: Dark Knight), a kind of neon edge detection. But I have done work with this type of effect before, and I thought I could offer advice to anyone who's working on this now. There are other ways to do this, and might be better suited to you depending on what you're doing, but you could just put the 'lines' in the texture, seen in games like Xbox 360's Borderlands 2. You can get the same effect either using a texture or with a cell-shading shader. Unity is already able to do this edge detection without you having to do anything extra, just take a look at the Unity Editor's scene-view Shading Mode tab's 'Shaded Wireframe' setting. So you could easily use line renderer to add that in the LateUpdate(). But doing it through a texture is nice too, you can have multiple textures for switching it off and on or for different variations of edge detection. Unity has a cell-shading example project that demonstrates cell-shading in Unity. Also, texture-based cell-shading can look better than 'rendered'. Look at this 'rendered' version: http://ws.cis.sojo-u.ac.jp/~izumi/U...n/Images/manual/script-EdgeDetectEffect-0.jpg
    PS, For my 'sonar effect', I just found a good way to do it: http://www.codeavarice.com/dev-blog/tutorial-burning-edges-dissolve-shader-in-unity
     
    Last edited: Dec 4, 2019
    wxxhrt and Doidel like this.
  10. Zante

    Zante

    Joined:
    Mar 29, 2008
    Posts:
    429
    Anyone have the links to the original scripts posted in the video, the link doesn't work anymore? : (
     
    ina likes this.
  11. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,029
    but this version requires image effects though?
     
unityunity