hard edges showing up when using fresnel

void surf (Input IN, inout SurfaceOutput o)
{

//create fresnel from dot product of world normal and view direction
float fresnel = dot(IN.worldNormal,IN.viewDir);

//clamp
fresnel = saturate( fresnel);
//use power to scale//


...
o.Alpha = c.r * fresnel *fadeAmount

}

so i'm using what seems to be the standard in calculations of an object's fresnel, however when I put it on my objects I notice that the more intense the fresnel, the more the edges of the mesh show up.

Is this simply a product of how fresnel works or is there a way to clean this up? I've always done fresnel in visual shader code so trying to approach it in HLSL instead.

 

Have you double checked that the object normals are smoothed in the source mesh?
You can either do this in the authoring software (presumably blender). Or in the fbx import options.
Try settings normals to "calculate" with a value of about 30

 

Try to normalize vectors.

 

You mean like


float fresnel = dot(normalize(IN.worldNormal ) ,normalize(IN.viewDir));

or
float fresnel = dot(IN.worldNormal ,IN.viewDir);
fresnel = normalize(fresnel);?

the former I didn't see any change in result and the latter broke the fresnel effect back to 0.

 

That was the first thing I did. I even went and changed it from import to calculate and it didn't help. This issue is on any object I use. If I throw it on a unity cylinder it has the same issue.

 

A few things:

You should absolutely be doing this as a vertex fragment shader instead of using a Surface Shader. Surface Shaders do a lot more "work" than is needed for any unlit shader. That "LightingUnlit" Surface Shader function that gets reused so often is an abomination, and the alternative of setting the Albedo to black isn't as effective as you think. Surface Shaders were never intended to be used to make unlit materials, and it'll generating a shader that's more expensive than a straight vertex fragment shader. More expensive to compile, more expensive in memory usage, and more expensive to render.

All that said, they're not the problem here. There's no reason a hand written vertex fragment shader will look any different than a Surface Shader for this particular case, assuming the issue is what I think it is.


Second, the normalize() function is intended to be used on vectors. A vector is a value with multiple components, like a float4 or float3 has four and three components respectively. A float is not a vector, and you should not try to normalize it. Also, I believe Surface Shaders already normalize the IN.viewDir, but I don't remember if they normalize the IN.worldNormal. That may help a little, but that's not really the issue either.


As for what the issue is, that's a little harder to nail down. However here's my guess:

You're seeing an interference pattern of sorts from the front and back faces of the fresnel interacting. Notice the "front" face where it isn't overlapping isn't showing any banding? I suspect if you change Cull Off to Cull Front or Cull Back the banding disappears. I'm also guessing you're not using an HDR render target; the camera doesn't have allow HDR enabled or it's disabled on the project level. The problem is while the front and back faces by themselves are fine, they're hitting the limits of a 32 bit render target and you're getting 1 color value "steps". When you layer two gradients over each other you're magnifying those limits and they become larger steps.

Enabling HDR should help. Alternatively using some kind of noise texture to break up the gradient a little will also help hide the issue.

 

All that bgolus said, and:

It's really not. Fresnel is in short the amount of reflection of a surface based on the viewing angle. As a general rule, the reflection gets stronger at grazing angles. So fresnel is:

• Reversed from what you have, more intense at grazing angles.
• Specifically related to the intensity of reflections.
• A lot more specific than just dot(V, N).

 

Thanks for your in depth explanation. I will indeed transfer this over to a vertex shader now that you mention the poor performance.

I tried doing cull back and or front but still get same issue. What's odd is cull off doesn't make it double sided. It stays cull back for some reason.Perhaps this is part of t he issue.

as for HDR, i'm working on a project for a specific device api and it doesn't support HDR so unfortunately.

I"ll play around with adding noise once I rebuild this shader in a vertex shader

P.S Doh! Didn't realize I was normalizing a float like that. I totally glazed over the fact of using a dot with two vectors would go from vector 3 to a float.

 

The image above is absolutely using Cull Off, or you're using a mesh with doubled up geometry. If you're using doubled up geometry, then the cull mode should probably be Back. But you said this error happens with the default cylinder too? That geometry is not doubled, so unless the shader is set to Cull Off it should not have the same problem. I wonder if you don't have have the culling set in multiple places in the shader already.

Does enabling HDR fix it though? Regardless of if you can use that solution or not, if it fixes the problem that gives you information about if my theory is correct or not. If it does not fix it, then my theory may not be correct.

 

I suppose it's interpolation error on long triangles with acute corners and it's impossible to fix without geometry changes.