Symmetric models with mirrored normal maps - Shader Fix?

I have so many models with symmetry that re-use the same texture coords for both sides of the model (mirroring). Great for textures, but now I'm trying to normal-map them.

The problem with this (just in case anyone isn't familiar with the issue) is that you end up with one side of the model having 'inverted' lighting behaviour. That is, everything is fine on one side but on the 'other' side of the model, the bumps facing the camera go dark. It also results in big ugly seams where the two sides meet.

This is a call for solutions, as many people seem to have this problem.


Initial idea (flip the normals in the shader pipeline)

I mean, How difficult can that actually be ?

Since symmetric model are usually mirrored along one axis at 0, it makes sense that all the 'good' normals are on one (lets say the +x) side of the model, and the 'inverted, nuisance' normals are on the other (in this case, the -x side)

So, I'm assuming theres a simple way to effect the following in the shader pipeline... If the X is < 0 then invert the normal. Presumeably by normalising X to a unit value (X = -1 or +1) and using that to invert (or not invert) the broken normal peturberation by a simple operation.

I'm not sure that it is possible to know the coordinates of the model in a fragment shader though. Would this need a geometry shader too? Can a geometry shader pass a +1/-1 value to the fragment shader ?

So...

Any examples of getting local Y coordinates from inside the pixel shader ?
Any examples of a geometry shader passing additional parameters to a fragment shader ?
Any examples of a working mirroring shader - I don't mind if it is Unity, HLSL, GLSL or even a compiled FXO, just anything to start from.


Idea 2 (Splitting the task)

If there is no way for a fragment shader to know which side of the model it is processing - how about rendering the model in two passes ? First pass is non-inverting and a simple geometry shader passes in only one side of the model - second pass is inverting and the geometry shader passes in the other side of the model.

Is that possible ? Any idea on how to have a geometry shader act as a filter ? I'm sure it can be done because I have an FXO in my collection which allows me to 'cut away' a mesh using a arbitrary plane.



Fallback (cheat)

Without a shader solution the simplest compromise I can see would be to switch from an XYZ normal map to a simple one channel bump map. I've seen software to create a 3-channel normal map from a 1 channel bump map ... is there any software out there which can do this conversion in reverse ?

Not sure if this is a valid solution. Detail and accuracy will be lost and I think the shader would be slower so this is just my backup plan if all else fails. Still, it would give me something to work with while I try to come up with a true shader-based solution.



I know other people have solved this. From looking at their assets I notice that a LOT of games use mirrored normals.

So, can we work on some solutions here ? If we can come up with something that works I will happily put in the time to convert the existing shaders to various 'mirroring' versions for those who need them.

Any hints, tips, rumours or snippets welcome.


-Gary

 

hello,
maybe, (if your model is really symetrical) you can think of using Half the geometry. and compute the otherside in the shader.

(assuming that the half model is in X+ part)
for each vertex you can compute:
- the normal and reflected normal (in object space so -x)
- the reflected Position (in object space so -x)

compute WorldSpace position for each vertex and reflected vertex
do the magic computation for normal map (tan,Bi...)

then for each pixel you can compute:
- the original pixel color with normal and RGB normalMap
- the mirrored one with reflected normal + (-R)GB normalMap

they did this on a car racing game i worked on. (working on Half the Hull then they computed the other side inGame)

hope that helped.

 

Crytek and Epic support mirrored normal maps at the engine level. I'm interested to see how this works in unity though.

 

Ah, yes. I think I am using the wrong terminology. The meshes normals are correct... it is the normal map which gets applied badly in texture space due to the flipped tangents.

Does anyone know how I'd repair that in the mesh ? I have 3DS Max, Blender and DAZ3Ds Hexagon.

Is the code you posted something which should appear in the shader (my code), or something unity does for me (behind the scenes) - Sorry, had problems following along.

So... if I loop through the mesh ... for each X Y Z W, I would need to set W to a magnitude of 1 and adopt the sign of the chosen axis (such as X)

Ah, theres the problem then. I believe the mesh is using three component vectors.

Wow, that is much better response than others I've read elsewhere which tend to range from - you cant unless your engine supports -to- it's bad practice to mirror the textures in the first place.

Can anyone provide info on applying these kinds of changes to the mesh ? Meanwhile I'll stare at Blender for a while - since if anything can do this it's probably blender.


-Gary

 

anyone has a solution for this? i have the same problems with flipped tangents

 

Yes, turned out to be quite simple in the end. I went back to my modelling software and exported the model again, but this time in .fbx format - making sure to tell the exporter to save the normal and tangent/bitangent data into the mesh. This worked really well - but I had to change normals and tangents from 'calculated' to 'import' in the mesh's importer section within unity.

Only one model had a slight 'lighting seam' but the bumps were correct on both sides of the model. There are tutorials out there for repairing lighting seams, and a little fading in one channel of the normal map and the seam dissappeared.

The models all look fantastic now..


I cant understand why this is such a big mystery in other forums, with people saying things like "This problem is difficult even for the big software houses" ... pah! Use a format that stores the tangent frame with the mesh and you're golden!

I've just done it in XNA too. Process was considerably different but the end result was the same : )
So don't be put off by all those threads out there saying you cant/shouldnt mirror normal maps : )

Anyway, Hope that helps.


-Gary

 

I mirror by scaling the transform by a negative value. In that case recalculating the tangents on the mesh with the tangent.w negated does the trick. I use the script below. Note the second argument 'mirrored'. Just pass that as 'true' for mirrored meshes, false otherwise.

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4.  
5.  
6.  
7. /*
8. Derived from
9. Lengyel, Eric. “Computing Tangent Space Basis Vectors for an Arbitrary Mesh”. Terathon Software 3D Graphics Library, 2001.
10. http://www.terathon.com/code/tangent.html
11. */
12.  
13. class TangentSolver
14. {
15.     public static void Solve( Mesh theMesh , bool mirrored )
16.     {
17.         int vertexCount = theMesh.vertexCount;
18.         Vector3[] vertices = theMesh.vertices;
19.         Vector3[] normals = theMesh.normals;
20.         Vector2[] texcoords = theMesh.uv;
21.         int[] triangles = theMesh.triangles;
22.         int triangleCount = triangles.Length/3;
23.         Vector4[] tangents = new Vector4[vertexCount];
24.         Vector3[] tan1 = new Vector3[vertexCount];
25.         Vector3[] tan2 = new Vector3[vertexCount];
26.         int tri = 0;
27.        
28.         for ( int i = 0; i < (triangleCount); i++)
29.         {
30.             int i1 = triangles[tri];
31.             int i2 = triangles[tri+1];
32.             int i3 = triangles[tri+2];
33.            
34.             Vector3 v1 = vertices[i1];
35.             Vector3 v2 = vertices[i2];
36.             Vector3 v3 = vertices[i3];
37.            
38.             Vector2 w1 = texcoords[i1];
39.             Vector2 w2 = texcoords[i2];
40.             Vector2 w3 = texcoords[i3];
41.            
42.             float x1 = v2.x - v1.x;
43.             float x2 = v3.x - v1.x;
44.             float y1 = v2.y - v1.y;
45.             float y2 = v3.y - v1.y;
46.             float z1 = v2.z - v1.z;
47.             float z2 = v3.z - v1.z;
48.            
49.             float s1 = w2.x - w1.x;
50.             float s2 = w3.x - w1.x;
51.             float t1 = w2.y - w1.y;
52.             float t2 = w3.y - w1.y;
53.            
54.             float r = 1.0f / (s1 * t2 - s2 * t1);
55.             Vector3 sdir = new Vector3((t2 * x1 - t1 * x2) * r, (t2 * y1 - t1 * y2) * r, (t2 * z1 - t1 * z2) * r);
56.             Vector3 tdir = new Vector3((s1 * x2 - s2 * x1) * r, (s1 * y2 - s2 * y1) * r, (s1 * z2 - s2 * z1) * r);
57.            
58.             tan1[i1] += sdir;
59.             tan1[i2] += sdir;
60.             tan1[i3] += sdir;
61.            
62.             tan2[i1] += tdir;
63.             tan2[i2] += tdir;
64.             tan2[i3] += tdir;
65.            
66.             tri += 3;
67.         }
68.        
69.         for (int i = 0; i < (vertexCount); i++)
70.         {
71.             Vector3 n = normals[i];
72.             Vector3 t = tan1[i];
73.            
74.             // Gram-Schmidt orthogonalize
75.             Vector3.OrthoNormalize(ref n, ref t );
76.            
77.             tangents[i].x  = t.x;
78.             tangents[i].y  = t.y;
79.             tangents[i].z  = t.z;
80.        
81.             // Calculate handedness
82.             tangents[i].w = ( Vector3.Dot(Vector3.Cross(n, t), tan2[i]) < 0.0f ) ? -1.0f : 1.0f;
83.             if ( mirrored ) tangents[i].w *= -1;
84.         }      
85.        
86.         theMesh.tangents = tangents;
87.     }
88. }
89.  

Use something like this to recalculate a mesh's (and its mirror's) tangents:

Code (csharp):

1.  
2. TangentSolver.Solve( mesh , false );
3. TangentSolver.Solve( mirrorMesh , true );
4.  

 

I use the same procedure as GaryC to import my mesh its always worked for me. I bake my normal maps with Xnormal using the obj format. When I do I only bake half the mesh then fold over the uv's when I get back into max. I also use the fbx format to go from max to Unity.

Many company's have had problems with Binormals and Tangents, My old company included because up Until at least Max 2008, Max's exported Binormals and Tangents were not quite orthogonal. To make matters worse the format Max used to display in its real time view port was different to what it exported and rendered.

This lead to all sorts of crazy workarounds and the strange rules you hear that individual studios use to get around the issue. I knew one guy who use to manually align the border uvs to be perpendicular and then flatten the details on the seams.

Follow Garys advice and avoid old versions of max if you are going to bake your maps. (by the way I don't actually know if Autodesk have fixed this issue yet).

 

"- but I had to change normals and tangents from 'calculated' to 'import' in the mesh's importer section within unity."

How do you do this?

---edit ---

Never mind. Found it. Thanks.

 

Did anyone already tried to invert the green channel in the normal map?

 

So that's mean we have to re-edit our model again in another 3D software? Btw is this case only for Unity or every engine has this kind of problem?

 

Ah I see, thanks for the explanation

 

I know this is an old thread, but for anyone else having this problem:

Check that your Normal textures are set to "normal map" in the inspector, and make sure "Generate from greyscale" is unchecked. This did the trick for us without having to rework our models, or re-export with special settings.

 

Heya,

Ok, I know its an old thread... but I just tried to texture a model with many UV's inverted sharing the same texture (great for Modular Modeling!!!) But... once again, I got stuck with normals not displaying correctly. I've done the trick of exporting as an fbx and all the binormal and tangent info, and then making sure Unity Imports Tangents as well. This hasn't helped.
Any new tricks on how to deal with this? This would help a lot!

Thanks!

Nick.

 

Mirroring Normals in the Games industry is quite a regular practice, and it frustrates me that trying to do so in Unity gives me issues. As I mentioned, I'm trying to do a few models using a Modular Technique (You save a huge bunch of texture space by overlapping and inverting UV's to create all sorts of panels that interconnect with each other). But all this is worthless if the engine doesn't support inverted UVs (or mirrored UVs... whatever it is called).
I've been searching the forums with no satisfactory answer. If there is a Shader that solves the issue, I would really love to know about it (even if it is payable).

I hope someone can help me on this... can't continue working at the current state

Thanks

Nick.

 

I too have this problem.

I have exported model with tangent and binomals included using maya fbx format 2009,2010,2011

Then inside unity I have checked imported normals + tangents.

and then i tried the other trick of fading out red channel or green channel of normal map to see if this fixes problem. But only time that I don't see any reasonable seam is when i fade out both channels. (which deletes any normal info ..)

if i don;t import tangent then result is different as i don't get the seam,but lighting is incorrect on the mirrored side. When i import tangent, lighting seems to be correct on the mirrored side but can't get rid of the seams.

 

ok, inverting Y channel of the map has solved this issue. But when I import tangent, it didn;t work, I had to resolve to : import normal + calculate tangent. Somehow this seems to work for me.

 

The import tangent stuff is very flaky. I've yet to have decent results using either compute or import.

But then it's hard to know precisely whether it's the import or the export that's wrong :/

 

So, for anyone that needs an easy answer to this but thinks all the solutions posted before me are too complicated, this is what I did: On my 3d model inside unity on the FBX importer in the Normals tangents part I selected: calculate. Then I increased the smoothing angle to 180, don't forget to hit apply. This removed the seams and took care of the inverted bumps on one side. ( I still had one problem though, the normals were inverted evenly all over the model, but that was the fault of the software where I made the texture.)

In my case the bumps appeared to be inverted on the normal map so I then took it into photoshop, selected the green channel and then inverted the colors (the green channel is actually grayscale on a normal map). That's all, I have no seams, the normal map shows the bumps in the right way on both sides, hope it helps.

 

Well, what I would do is;

Bake the normal maps using my xNormal plugin for Unity, using FBX as the format.
Import the same FBX into Unity for use in your game.
Set Normals to Calculate.
Set Smoothing Angle to 180.
Set Normals to Import.
Set Tangents to Calculate.

I'm pretty certain that will work.

 

hello all.

I am just working on a modular sci env and I found a simple solution to this.
I had created a tangent based normal map using nDo2 and high poly models (baked with xnormal). I found when I rotate a piece with this on all is cool, but when i mirror it in unity (by setting the scale to -1) it renders wrong. That problem lead me to this forum post.

I tried flipping the Y and the x channel in the normal map, but that did not work for me.
I tried importing normals and tangents and that produced no normal map shading for me. My solution is the defult imported normals and calculated tangents.

However, if you are working with modular pieces with a symmetrical shape the simple solution is to just mirror it in the UV channel and not the mesh (see attached example).
Its going to be more tricky with a character or something where you need to mirror the mesh, I think what you are going to need to do is develop a work around where you do something like after mirroring the mesh you mirror the uvs separately. Try taking half a mesh, saving the uvs, duplicating and mirroring the mesh applying the saved uvs and then mirroring them in the UV channel, that might work.

My solution in general is a bit lame and wasteful, but im just going to avoid all cases of mesh mirroring, for this modular stuff it should be easy, and for organic stuff ill just use more texture space.

 

I'm using FBX 2012.2 from Maya 2010, and exporting the tangents bi-normals still doesn't seem to fix the mirrored seams .. is this FBX version too old ?

Just throwing it out there, but it makes more sense that if we use Unity "calculate" tangents on imported normals, it should sort mirrored tangent issues on the engine's end ...

 

here is the simple way. drop this script into you game object.

using UnityEngine;
using System.Collections;

public class NormalInverter : MonoBehaviour
{
private MeshFilter filter;
private Mesh mesh;
private Vector4[] tangents;
void Start ()
{
if (transform.lossyScale.x < 0 || transform.lossyScale.y < 0 || transform.lossyScale.z < 0)
{
filter = gameObject.GetComponent<MeshFilter>();
mesh = filter.mesh;
tangents = mesh.tangents;
for (int i = 0; i < tangents.Length; i++)
{
tangents.w = tangents.w * -1;
}
mesh.tangents = tangents;
}
}
}

 

I have that problem with normals myself but it only happens with tangent space normals. and I think it takes a bit of an artistic fix on the normals or another bit more complicated process. Here is a reference to normal maps, maybe it helps you understand the problem: http://wiki.polycount.com/wiki/Normal_Map_Technical_Details

I'm also trying to fix the normal map issue myself: http://forum.unity3d.com/threads/tangent-space-normal-map-seams-problem.384063/

 

hi,I know this is a very old topic, but we have a new issue on it when we use static batch on mirror geometries.
I am very sure that we use float3 binormal = cross( v.normal, v.tangent.xyz ) * v.tangent.w to calculate binormal, and when static batch is disable ,everything is ok; but when we use static batch to combine mirror geometries, tangent result seems to be incorrent.
Our unity version is 2018.2.1f1, and the issue also occurs in 2017 version.