Search Unity

AO and Vertex Displacement?

Discussion in 'Shaders' started by Dreamteck, Aug 22, 2018.

  1. Dreamteck

    Dreamteck

    Joined:
    Feb 12, 2015
    Posts:
    336
    I have a very simple vertex displacement surface shader:
    #pragma surface surf Lambert fullforwardshadows vertex:vert addshadow nolightmap noforwardadd

    The issue with it is that Unity's AO does not pick up the vertex displacement resulting in AO being drawn around the undisplaced vertices.

    upload_2018-8-22_11-27-7.png

    I used the Debug mode of the Post Processing stack to check what's going on. It turns out that the shader properly writes to the Depth buffer:
    upload_2018-8-22_11-27-52.png
    But the normal texture stays the same:
    upload_2018-8-22_11-28-21.png

    How can I make my shader update the normal pass as well?
     
  2. Dreamteck

    Dreamteck

    Joined:
    Feb 12, 2015
    Posts:
    336
    After some research and thinking I solved my issue. See, the problem is that Unity renders the scene with the DepthNormals replacement shader before the vertex displacement takes effect for some reason. They have internal workaround for some of their shaders (vegetation mainly) but do not support custom vertex offset functions like the wing bending I wrote. So the solution for me was to download the built-in shader packages for the current Unity version, take out the Internal-DepthNormalsTexture from the package and import it in the project. I then created a custom RenderType for my plane shader - "RenderType" = "Player". Inside the DepthNormalsTexture shader I created a new subshader with the same RenderType, then I copied the offset logic I wrote for the plane shader inside the vertex function of the DepthNormalsTexture shader. Finally in Edit->Project Settings->Graphics I replaced the DepthNormalsTexture shader with the new one I created. Now both the depth and the normals textures render fine and the AO issue is gone.

    Ugly fix, plus I had to pass all properties for the wind bending form my script to the shaders as global (otherwise DepthNormalsTexture doesn't have access to these parameters), meaning that I can't have a second paper plane with unique parameters, but that's okay cause we only have one player and the shader is used only for it
     
  3. Medoc

    Medoc

    Joined:
    Feb 26, 2014
    Posts:
    1
    Jeez ! I had the exact same issue and no other solutions seemed to work, yet yours did ! Thank you so much for taking the time to write what you found.

    Adding to the discussion, I'm assuming you moved on since it was 2 years ago, but to avoid writing the same function 2 or 3 times in several shaders (the blending of your paper plane for example) you can write it in a .cginc and then include it in every shader that needs it. That way it's easier to modify the function since it's written in one place only.