[SOLVED] Wavey normal maps from Blender/xNormal

Hi!

I'm still trying to generate perfect normal maps in Blender and in xNormal and in both getting the same issues: waves. I made some screenshots in Unity editor:




Those waves appear on other models too, and not only at the corners. I create my high and low models in Blender.
I even wanted to paint the waves away in Photoshop but I can't see them in any of the channels. Tried to normalize the maps with Photoshop plugins from xNormal and nVidia with no changes.

Does anyone have any suggestions on how I could get rid of these waves?

Thanx a lot in advance!

 

That's banding caused by the bit depth of the normal map and DXT compression. There's not much you can do about it, but they won't be as noticeable once you texture the objects.

 

Thanx, aNTeNNa trEE. After your reply I gave up generating new normal maps over and over again and textured my model with a test grid texture from Blender, which made the artifacts less noticeable. But I'm sure with a brighter texture you could clearly see these waves. So I tried to hide these with a detail normal map over the existing one, with tiny scratches, cracks, bumps, which made pretty good results with the right shininess of the material and light settings. With the highest glossiness of the material I can clearly see the waves coming through the detail normal map. So I think that a detail normal map is an acceptable solution for my problem but not for every model I plan to create. Some of them just have to be flat and clean.
What I have noticed is that these artifacts occur near to the edges where I made creases for subsurf weighting for my highpoly model, to make them less rounded. Any idea how these creases could influence the generation of normal maps?

EDIT: You said this is a fault of the DXT format, but these artifacts occur with every texture format.

 

Read the section called Normal mapping and characters and let us know if that helps you.

 

Jessy, I imported my model with a smoothing angle of 180 and this is what I get:



It seems like Unity calculates normals differently.

 

You can't import a mesh with automatically-calculated normals if you use your own creasing. I was talking about the latter portion of that section. I've never used Split tangents across UV seams, so I didn't know if it was appropriate for this situation, but it sounds like it would be.

 

Jessy, Split Tangents is always on, because I indeed have problems with normals on UV seams. But the problem I described in my post above has nothing to do with seams. Those waves appear not only on seams. And on my screenshots, where the waves are, there are no seams at all.
It looks really like an issue with the bit depth of the texture, as aNTeNNa trEE said. I don't know.
Thanx anyway for your help, guys!

 

Hi guys! It's me again, still trying to fight the artifacts in my normal maps

Here is what I did since my last post (with results):
- Changed the topology of the lowpoly model (In Unity: artifacts; In Blender, xNormal, Modo, zBrush: no artifacts)
- Baked the map in Blender, xNormal, Modo, zBrush, Melody (In Unity: artifacts; In xNormal, Modo, zBrush, Melody: no artifacts)
- Exported the lowpoly model and normal map in all available formats with different settings (In Unity: artifacts)
- Imported the model with and without Calculate normals and Split tangents checked (Artifacts)
- Imported the map with and without compression (Artifacts)
- Inverted every single channel of the map (just to be sure) (Artifacts)
- Painted out the artifacts in Photoshop (1. looks just wrong; 2. too many artifacts to paint out)
- Renormalized the map in nVidia and xNormal Photoshop plugins (nVidia: artifacts get more pronounced; xNormal: no changes)
- Added detailed bumps and scratches over the map + diffuse and specular maps (Artifacts are still very well noticeable)

Today I opened an old project created in Unity 2.5 and checked all normal maps I made for it. I'm 95% sure that I noticed no artifacts when I imported them 3 months ago. Now, guess what, I see the same "waves". That's why I have to ask, whether there have been made some changes in the texture/mesh importer or in the shader?

Now more about the artifacts: They appear at the gradients of the normal map, where the surface of the model builds an angle (not even a sharp angle) and are NOT uv-seams related (there are no seams). Even when I make a bevel at this corner, it's still there without changes or "compressed" into this bevel.

I don't really want to blame Unity for these artifacts, but, as I wrote it above, I baked and tested the maps in several modeling packages and even in Ogre and Esenthel engine. And they looked perfectly. I'm too scared of the UDK to test it as well

I desperately need your help, because all I've done in the last 2 weeks was baking the same normal maps, importing, testing, learning zBrush/Modo, baking... and so on, without any progress

Thank you in advance for any suggestions or ideas!

 

Looks like texture compression artifacts,maybe you saved the normals in JPG?Be mindful what format you are exporting the texture in,use bmp for testing.Try to experiment with texture import formats in Unity too,DXT formats can also lead to banding if not compensated for.

 

Thanx for your suggestion, but no, compression is not the problem. I'm using always BMP or Raw Targa (Blender), importing in Unity with 32bit, no mipmaps.
The worst is, I can't even SEE these waves in Photoshop, neither on a single channel nor on all together. There are just these gradients on each RGB channel, which combined render in Unity ugly pixelated waves, which indeed look like a compression issue.. But again, I'm using only uncompressed texture formats.

 

Maybe you have the UVs very stretched on those curved parts? If you used any kind of automatic unwraping, maybe you need to expand those polygons some more on the UV, so they display the gradients better.

 

What about quality settings?

Can you post a screenshot of texture import settings?

 

@GusM: Thanx, I'll give it one more try and play around with UVs.

@KaelisAsur: Yes, sure, here it is:

 

Well, no luck with UVs... Tried to expand/relax them, like GusM said, but it made absolutely no changes to the artifacts.
I'm out of ideas now, don't know what to try as next.

 

Your aniso is set to 1 and filter mode to bilinear. Have you tried setting aniso to 9 and filter mode to trilinear?

Also, have you played with graphics quality settings?

 

Hmm, I set aniso to 9 and filter mode to trilinear: no changes.. The graphics quality settings are always set to Fantastic (high quality, full res and so on). Tried to change every single setting, with no difference on the artifacts

 

I finally found a solution for my problem.

For those who have the same problems with their normal maps in Unity, just change this line in the pixel shader

Code (csharp):

1. // get normal from the normal map
2. float3 normal = tex2D(_BumpMap, i.uv2).xyz * 2.0 - 1.0;
3.  

to

Code (csharp):

1. // get normal from the normal map
2. float3 normal = normalize(tex2D(_BumpMap, i.uv2).xyz * 2.0 - 1.0);
3.  

I don't know, why the devs didn't normalize the normals here.. Performance gain?.. Whatever. The positive thing is, I had to learn shaderlab to find this solution. Now the artifacts are gone; I'm happy; the development can go on.

 

The cause of these banding artifacts is the 8bit discretization of normal maps. For large shininess values (I believe the shininess exponent is 128 if you move the slider in Unity all the way to the right, but that's not really documented), any normals bisecting viewer and light directions (=causing the viewer to see the highlight in the Phong model) react *very* sensitively to normal lengths != 1.

I'll explain the artifacts in a moment. First, it is important to notice, there *is* no "neutral normal", when using 8bit normal maps. tex2D() will interpret pixel value 0 as 0.0, and pixel value 255 as 1.0. As a result, pixel value 128 is interpreted as 128/255=0.50196. tex2D() on a "neutral" normal of 128/128/255 will therefore result in 0.50196/0.50196/1.0. Expanding from range-compression ([0:1]->[-1:1]) gives the actual normal vector 0.00392/0.00392/1.0 (a deviation of 0.3178 degrees to true 0/0/1), and as a sidenote, the length of that normal is 1.000015379, not 1.0. You can confirm this behaviour in the attached testcase, both on the plane, and in the material preview. Compare the images rendered...
a) ...with simple specular shader (note, the preview is ugly, it's a vertex lit shader)
b) ...with a normal map of constant color 128/128/255
c) ...with a normal map of constant color 127/127/255
The highlight will shift slightly between all three cases, which it shouldn't if the normal was truly neutral. A truly neutral normal would be 127.5/127.5/255, but this cannot be represented with 8bit textures.
(edit: after disabling default texture compression DXT1 for the normal maps I noticed there is no longer a difference between 127/127/255 and a shader with unassigned normal map, so it seems the shader emulates a missing normal map by using 127/127/255. When switching to a non-bumped specular shader(=simple specular), however, one can see that the highlight does change positions.

If we now store normalized normals in our map, due to the 8bit discretization there comes a point where the z-component will flip from bit value 255 to bit value 254. For example, consider these normals at such a rounding border:
8bit normal 138/128/255 represents the normal 0.08235/0.00392/1.00000 (actual length 1.00339)
8bit normal 139/128/255 represents the normal 0.09020/0.00392/1.00000 (actual length 1.00407)
8bit normal 139/128/254 represents the normal 0.09020/0.00392/0.99216 (actual length 0.99626)
So the best representation (the one closest to length=1.0) for x=139 is 254, not 255.
Now, weighting these normals with the shininess exponent without the additional normalization shown in the previous post will result in:
1.00339^128=1.54274
1.00407^128=1.68124
0.99626^128=0.61870
The visual effect of this 255->254 transition will therefore be a drop in brightness to about 1/3, which is a *lot*. (Note in the testcase the highlight for 139/128/254 is *very* dark, and 139/128/255 is washed out)

Re-normalization of normals after range-decompression, as proposed in the previous post, will result in lengths of 1.0, and the Unity developers should fix the default Unity shaders accordingly.
However, there will still be a discrepancy in the *direction* of these normals:
138/128/255 will be re-normalized to 0.08207/0.00391/0.99662. The angle to a true 0/0/1 normal is 4.713 deg.
139/128/255 will be re-normalized to 0.08983/0.00391/0.99595. The angle to a true 0/0/1 normal is 5.159 deg.
139/128/254 will be re-normalized to 0.09054/0.00394/0.99589. The angle to a true 0/0/1 normal is 5.199 deg.
Again, weighting these normals with the shininess exponent will still result in discrepancies:
cos(4.713)^128=0.64820
cos(5.159)^128=0.59480
cos(5.199)^128=0.58993
These discrepancies are much less noticeable than for the case above, but they can still create visible artifacts (=banding) for large shininess exponents (they are not noticeable in the testcase). Unfortunately, the only way to get around this is to use 16bit normal maps.

An extension to the normalization solution would be to maximize the available bit depth of the normals, by not storing them normalized at length 1, but instead scaling them to their max x/y/z component to 1. So a 45 deg normal would not be represented as 0.707/0/0.707=218/128/218, but instead as 1/0/1=255/128/255. This approach further reduces the visible artifacts (but still can't eliminate them completely), and since we now re-normalize all stored normals after texture sampling anyway, it doesn't matter anymore that the normals stored in the map are of length >1.

(edit: had DXT1 compression enabled in the testcase, which is a BIG mistake for normal maps. fixed that by switching to RGB24bit)

 

As an extension to the previous testcase, here is a water surface normal map, where the banding with no re-normalization can be seen clearly.

 

sometimes you just need to have a denser mesh to bake the normal maps onto. You can tell by what the object looks like with just smoothing on. If you have a loop tool, try adding a few and see what happens. adding an extra loop near an edge will clean up the smooth shading and recieve a better normal map.

I use 3dsmax and make my low res model, and UV map that, then I insert a bunch of loops before baking that gets rid of the ugly smoothing, bake the normal map and then remove the new loops I added.

I'm not familiar with your tools so I don't know if you can add loops without breaking the UVmapping of your objects like I can in 3dsmax.

 

Your suggestions might help in certain cases, but are unrelated to the distinct artifacts discussed in this thread (as can be seen when using the above water texture on a plane).

 

yeah, that post was posted before I finished writing mine. I was talking about a different problem that I noticed in your screenshots.

 

Taking a closer look at the two aspects of the problem I mentioned, I think the re-normalization fixes *all* relevant artifacts. The artifacts occur because a length !=1 modifies the maximum intensity of a highlight.
The other aspect (the deviation of the angle) is not really noticeable, since it only influences the direction of the normal (which is what the normal maps are designed for, anyway), not the overall intensity of the highlight.

 

Wolfram, your description of this problem is quite technical but it doesn't really explain why our 8 bit normal maps look just fine in other programs and yet look like crap in Unity

It looks to me like our specular maps or normal maps are being reduced to 4 bits per channel. But yet when I look at texture memory usage it seems to imply that the full quality maps are being stored in memory.

So I dunno..

And what's even more puzzling is why more people aren't complaining about this issue. I don't have exactly the best eye sight on the planet and so it's not like I'm just being fussy about this. Hell, even my pet bat could spot the banding!.

 

Well one of us must be reading Wolfram's explanation incorrectly, because it explains precisely that, to me.

Unity's shaders handle the normals incorrectly and that causes the artifacts. It looks just fine in other programs because they handle their normals correctly. Is that not what Wolfram's saying, in very simple terms?

If you can't follow Wolfram's solution (which I think he intended the Unity developers to implement, not the users!) then SONB's solution at least looks as though it would minimize the problem. Are you saying that you've implemented Wolfram and/or SONB's solutions and that neither works?

 

Woah!, you're right! SONB's method worked!.

After quickly reading through Wolfram's posts (skipping the technical details) it seemed his conclusion was that Unity needed to use 16 bit normal maps, and so this made me dismiss SONB's method and not try it out.

Thanks for kicking me in the right direction!

And thanks to SONB for finding a solution!

 

UPDATE: this problem appears to be fixed in Unity 3 beta5

EIDT: more info:
The problem is fixed, the normal length is now always correcttly normalized. However, when importing the testcase packages, Unity3 will ask that it needs to "fix" the texture settings of the normal maps. If you do that, there's a bug in the beta that flips the red and blue channel of the normal map (ABGR instead of ARGB conversion). So you need to press "Ignore", and then manually check "Normal Map" in the texture import settings of each normal map, and press "Apply" everytime.
And some general info: An unassigned normal map is still assumed to be 127/127/255, while the highlight generated by that is different from a regular Specular shader. However the normal maps 128/128/254 and 128/128/255 are now *identical* there is no longer a shift in normal direction. The same is true for the testcase with 139/128/{253|254|255} - which is kinda strange, since the resulting normal directions should differ.
I am not sure what exactly they are doing now in the shader code, since they just call a function named "UnpackNormal", but I cannot find that function anywhere - neither in the code, nor in the documentation...

EDIT2: from the 3.0.0 a1 changelog:
"Normal map compression. All shaders need to use UnpackNormal(n) instead of n*2-1."