Normal mapped Beveled edges look weird.

Hello, I am having trouble getting beveled edges in a normal map.

In Blender, if I set my low poly cube to flat shaded, and bake from the high poly, I get this result in Blender


It looks the same in Unity.


If I set my low poly cube to smooth shading before baking, you see my normal map is much more colorful, because it is compensating for my cube being set to smooth shaded. However, in Blender my cube looks perfect!


the edge looks not as good in Unity, but not too bad.


I would ask how to fix that seam in Unity, but that isn't the bad part. The bad part is when I see the whole cube. It looks shaded really weird.


I want it to be shaded like a normal flat shaded cube, but with beveled edges. It seems that Blender and Unity are doing something different, so they aren't 1 to 1 on how to display normal maps.

Here are both cubes back to back. The one on the left has good bevels, but the rest of the cube is shaded weird. The cube on the right is shaded good, but the bevels have those weird artifacts. What is the best way to fix this? I am using cubes for now, because I want to figure this out before I do this with more complex models.

 

Compare the two normal maps to find the answer to your question. You will probably need to edit the second normal map in 2D package to correct the shading and get the bevel you desire.

 

Thank you for the response theANIMATOR2b,

I don't know how I would best edit normal maps in a 2d package, as each color corresponds to a different direction. (There has to be a way to paint strokes, and the computer knows if it is an indentation or protrusion.)

Anyways, I thought my problem must be that when I think about it, I'm trying to do the impossible. Make a 6 sided cube with all 90 degree angles look like one with over 4k faces, and much smoother angles. Setting a simple cube to smooth shaded (without normal maps) always looks weird because the angles are so strong.

I figure, in a real case scenario I would never have such a huge flat surface that is all 1 polygon without any supporting loops. So I tried putting a few edge loops around the cube to ease the shading a bit. The results are pretty good. Its not perfect, as you can see a tiny bit of shading errors in the corners, but its not very noticeable (could even pass as a dirty spot on the texture)




the closer I bring in the edge loop the smaller the shading error becomes.


Here is a more complex object. I think it works good enough as there aren't as many huge open polygons by themselves trying to be shaded. The shading gets a little weird at the outer edges of the cylindrical parts, but its not too bad. I'm sure more polygons to fill the silhouette would fix that.

 

Google tangent base and look if there is a plugin for Blender that renders Unity tangent space.
Alternatively you could try to bake an object space normal map and use handplane to convert it to a tangent space normal map which works with Unity. I can't guarantee it will be perfect, but it should be a lot better.

 

The answer to your first question is visible in the normal map in your image. Your hard edges are using bilateral filtering to decide normals where to aim normals, and they are running into trouble where the faces meet. You have not given any space for edges to be padded (VERY important when filtering). This is causing your edges to sharply change angle between faces where the UVs meet. You need to separate your UVs. As a general rule, Any hard edge MUST also be a split in UVs. Don't worry about the added vertexes, because Unity is using two vertexes anyway to store your hard edge!

Separate out your faces in UV space. Be sure to bake with a good cage (cage needs to not split at hard edges)

Here is an example image I found with a quick google search:

Posted by user Avanthera on the polycount forums.

 

Findus, That's a good idea to look at the different types of normal maps, and see if I can convert them. Honestly, I've only ever used Tangent based, because that is what every tutorial I have came across has used. I don't even know what the other kind are used for.

Void24, I tried seperating out the UVs just like in the bottom left example in your piture, and I still get the same result. Maybe something with my baking settings needs to be tweaked. However, if that does indeed fix the bevel, having seperate islands for each side of an object will make texturing a nightmare. I would have seams all over the place!

 

Guys please can you make step by step tutorials
I beg you guys!
I have same issue