Search Unity

Calculating Unity's tangent basis for xNormal?

Discussion in 'Shaders' started by SONB, Jan 12, 2010.

  1. SONB

    SONB

    Joined:
    Jul 24, 2009
    Posts:
    89
    Hi!

    I'm going to write a plugin for xNormal, which should calculate exactly the same tangent basis as in Unity, so the pixel shader and the normal map generated in xNormal would match pixel-by-pixel perfectly. For this I need to know, which approach do the Unity devs use to calculate tangent basis.
    I don't expect that you (Unity devs) provide some excerpts from the source code here but I would be thankful for some support :)
    I found some articles about tangent base calculation but which of them applies for Unity?

    Thanx a lot in advance!
     
  2. maxfax2009

    maxfax2009

    Joined:
    Feb 4, 2009
    Posts:
    410
  3. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Can you store tangents in the model before Unity imports it? Presumably Xnormal uses stored tangents as well (unless you override them). This would guarantee that both Unity and Xnormal were using the same tangent space basis.
     
  4. Imperator

    Imperator

    Joined:
    Oct 5, 2010
    Posts:
    1
    That's not really the issue, the normal map need to be calculated with the same tangent basis that unity's render uses for the normal map to display correctly, making sure the mesh's are identical is not the issue here.
     
  5. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Unless I'm missing something, the information in the mesh is exactly the issue. Here's how Unity computes the tangent space basis in its vertex shaders:
    Code (csharp):
    1. #define TANGENT_SPACE_ROTATION \
    2.     float3 binormal = cross( v.normal, v.tangent.xyz ) * v.tangent.w; \
    3.     float3x3 rotation = float3x3( v.tangent.xyz, binormal, v.normal )
    4. #endif
    As you can see, it uses the normals and tangents stored in the mesh in a pretty straightforward way. If you supply meshes with tangents and normals, then the tangent space basis should be the same for both Unity and Xnormal.

    The only problem would be if you're letting Unity generate tangents and normals for you. In that case, knowing the method it uses is crucial to getting good results from Xnormal.
     
  6. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    Bump...

    It would be GREAT if UT would write a plugin for xnormal so that there is a proper way of generating normals for use in Unity.

    This whole thing about having to go all kinds of hoops and loops (assigning lots of smoothing groups, splitting the UVs at harsh bends and so forth) is a pretty big showstopper.

    Any thoughts on this matter from UT?
     
  7. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Does anyone know what the actual issue is? I explained above how Unity computes the tangent space basis from mesh data. Does xNormal use a different method?
     
  8. RElam

    RElam

    Joined:
    Nov 16, 2009
    Posts:
    375
    I think they're asking how to calculate the same thing Unity does when ModelImporterTangentSpaceMode is set to calculate. This is a different problem than calculating the rotation matrix in the shader.
     
  9. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Why get Unity to calculate tangents at all, then? If you already have tangents that work in your 3D program, just use those.
     
  10. RElam

    RElam

    Joined:
    Nov 16, 2009
    Posts:
    375
    That assumes the 3D app supports their export, as does the model format being used (which I think would eliminate OBJ, for instance). In short, I agree with you that that's the best approach, might not be an option though.
     
  11. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    @Daniel; My grief with the way Unity currently handles normalmaps i that you have to do all sorts of tricks to get the normal map to display correctly.

    A simple example would be how you normal map a box. Currently the only way I can display a normal mapped box without weird shading is to have several different smoothing groups and to have the UVs split wherever I have a new smoothing group. Even with this setup I actually get small seams along all edges although not very visible when the diffuse has been applied. Still its a pain in my artist eye ;)

    In some other engines such as Marmorset or CryEngine it is not necessary to split up the UVs and the mesh in seperate smoothing groups. Simply apply one smoothing group and keep it in one single UV isle. Marmorset can display the normals you generate in max straight out the box whereas CryEngine supports normal maps created using their own baker.

    What Marmorset and CryEngine does is much preferred. You do not need to add unneeded geometry or to split your UVs and smoothing groups. This is good both for performance and for memory use. Also for the artist for whom its a pain to paint textures if the UVs are split up in too many isles.

    From my understanding its a matter of how engines interprets the generated normal maps and in theory there should be nothing keeping Unity from actually displaying normal maps in this more "correct/modern/optimized" way. From my understanding the actual problem is that there currently are no tools available to generate normals that work correctly with Unity. This is where Xnormal comes into play as you can write plugins for this program depending on how a given engine interprets normal maps. So in essence if we had a plugin for Xnormal aimed at Unity we could get much better and more efficient normal maps.

    Please correct me if I am wrong. I am just a simple environment artist who doesnt really understand much of the technicalities but who indeed is slightly annoyed with the normal maps thing. It would be very interesting to get UTs input on this too because to be frank I do not understand why we have to go through these hoops to get decent normal maps working.

    Cheers :)
     
    Last edited: Jun 27, 2011
  12. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    What mesh format are you using, and what are your import settings for normals and tangents? Where are you getting your normal maps, and how are you importing those?
     
  13. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    FBX from 3DSMax, even when exported with tangents, screws up on Unity import and you have to tell it to calculate them.

    Matching the mesh's normal/binormal/tangent for baking to those used by the game engine when displaying a normal map is very important to get the most out of your normal map.

    It almost entirely removes the need for support geometry and other hacks like UV splits and smoothing splits. Making the model more efficient and it's creation far less fuss.
     
  14. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    @Daniel; Well basically what Farfarer said. If we had a way to generate normal maps with the same tangent basis as Unity uses then we could display normal maps the way it should be dsiplayed (without excessive geom, UV and smoothing splits). An easy way for UT to help us create these correct normal maps would be to write a plugin for xnormal as the original post by SONB was about.

    The normals and tangents of the mesh is not the issue. Mesh is imported as fbx (single smoothing group, single UV isle) and normal map is created in max or current version of xnormal (neither displays correctly in Unity).
     
  15. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Heh, are you Janus Kirkegaard? Did you go to Teesside for Game Art?
     
  16. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    @Farfarer/James; You got me ;) Will send you a pm
     
  17. the_motionblur

    the_motionblur

    Joined:
    Mar 4, 2008
    Posts:
    1,774
    I don't really think I get what the problem is with the normalmaps, then.

    I know FBX often does not translate Edgebreaks/Smoothgroups over too well. Farfarer said 3DS Max also has problems with it, I know Cinema 4D simply doesn't export edgebreaks in FBX. So a Normalmap calculated by xNormal (or any other normalmapping program) takes the lowres mesh with or without edgebreaks and translates the high detail information over into the normal map. Good. So far I'm very used to the process.

    A normalmap also is to some extend capable of translating harder and softer edges on a model without edgebreaks applied.
    My impression was that normalmaps pretty much carry the same information and different programs simply differ in R/G/B channels inverted. The UV seams I get by viewing a model in different realtime engines. So far only worldspace normals could get rid of them.

    Now what I don't understand is ... why is this a matter of How Unity treats normalmaps? Edgebreaks are more of a problem with the model than the normals in the first place.
    And how is this an issue that can be fixed in the normalmap itself?
     
  18. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    The way normal maps work is the same across all programs.

    However, all programs and games have subtly different ways of generating the mesh normals for displaying the model in the viewport (and often different, high quality ones for the actual rendering when baking the model).

    These differences mean that a fair bit of extra work has to go in to ensure that what looks good in the viewport of your 3D app also looks good in your game engine.

    Here's a good example of 3DS Max's own normal discontinuity - the normals it generates for the viewport are completely different from those it generated for the model when the normal map is baked (top row). And an example image of the same baked normal map, but using normals in the viewport that are generated to be identical to those used in the render/bake (bottom row).

    It's a substantial difference and that's just within one 3D app. You can start to see how even a small dis continuity cross-engine/cross-app can cause horrible visual issues.

     
  19. the_motionblur

    the_motionblur

    Joined:
    Mar 4, 2008
    Posts:
    1,774
    Okay ... but in that way it's more of an issue that the normalmap matches the normals/edgebreaks from the model, right?

    Or in other words: if the edgebreaks during the baking process match the edgebreaks in the realtime solution the result should be correct, right?

    The example here is from the Cinema4D viewport. Normals baked from Topogun.
    The normals have been baked into the model with no edgebreaks included in the lowres model whatsoever.

    The top image shows the normalmap aplied without any edgebreaks, either.
    The bottom image shows the model with edgebreaks (stadard Phong-Sngle only) active.

    The result in Unity is similar: when applying the normals onto the model without calculated normals and edgebreaks it looks pretty much okay. Only as soon as the model has edgebreaks active it starts to look weird.

    Granted - the edgebreaks aren't created specifically for the object. it's just a phong angle.
    It's only to illustrate what I mean and what I don't understand about it.



    (yeah - it's the example from ryan kingslien's book ... you gotta start somewhere with hard surface in Zbrush, eh? ;) )
     
    Last edited: Jun 29, 2011
  20. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    The normal map should *always* match the normals and edgebreaks. The vertex normals, positions or smoothing should not be touched after the normal map is baked.

    You can clearly see in that "pretty good" image (the top one) that it's got smoothing issues all over it. So we know that Topogun and C4D do things differently.

    If the normals and tangents that you baked it with were identical to the ones you display it with in the viewport/Unity - there would be no issues at all. Just like the example I posted.

    Check out the Polycount thread that brought all this up; http://www.polycount.com/forum/showthread.php?t=68173

    (I'm also entirely unsure why you'd do a hard surface gun in ZBrush either, but that's another matter :p)
     
  21. the_motionblur

    the_motionblur

    Joined:
    Mar 4, 2008
    Posts:
    1,774
    Woah. Very interesting read, thanks.
    I knew that the edgebreaks and the baking result should match up at any cost. It was just a matter of trying to explain what I meant.

    Still - that comes as a little shock ...
    So far I always was under the impression that something still wasn't right with my models if the normals map seemed too far "off".

    (Also: For learning purposes. ;) )
     
  22. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    Hi again guys

    Did some more testing but unfortunately I didnt get any closer to a solution.

    I tried installing the 3ds max 2012 trial to export the tangents of my test mesh instead of letting Unity calculate them. However the surface still doesnt look right.



    Please ignore the error message in the lower left corner ;) This is another object in the scene.

    Luckily it appears like it indeed is possible to work around these issues. I was looking at some of the (awesome) art for Crasher when I noticed that the normal maps indeed appear to be generated for a mesh sporting a single smoothing and UV group;
    http://www.zbrushcentral.com/attachment.php?attachmentid=263111
    Guess these guys may have succeded because perhaps they have written their own plugin for xnormal?
     
  23. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Turn on Qualified Normals in the viewport for Max. It uses whatever tangents the viewport uses for the FBX export.

    Gotta be done via an .ini file, check the Autodesk blog posts on it.
     
  24. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    Cheers James but I have already done that. All you got to do is to add;

    [ViewportNormalMapping]
    ViewportNormalMappingType=Qualified

    ... to the 3dsmax.ini file right?

    Have you got normal maps in Unity working right?
     
    Last edited: Jul 31, 2011
  25. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Yep, that's all it needs.

    Not got normal maps rendering perfectly yet, but I'm baking them in modo, which uses another tagent basis again which gets Nixed as soon as I export via Max.
     
  26. Westmark

    Westmark

    Joined:
    Nov 1, 2009
    Posts:
    187
    Any update on this?

    I bake normals in max and it looks perfect with max Qualified Normals, but when I export with "smoothing groups" and "tangents and binormals" checked, it doesn't make much difference compared to unity calculated tangents.
     
  27. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Unity devs reading this; what's the code you use to generate the tangents of a model?
     
  28. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    I'm working on this issue, too.

    Thankfully I just figured out exactly how max messes up its rendering: it calculates the bitangent wrong! Along with some quirks, but yeah. Maya could very well do it wrong in some way, too. In my Maya tests the result is certainly different than the Maya viewport. I'm guessing most of the problem comes down to the fact that Unity doesn't import bitangents, despite them being right there in the file. :( That would likely solve at least the max problem since you could just use the wrong bitangents as-is.

    I'm now thinking that the best way to solve this issue is to exactly copy an existing baker's tangent space basis with a bitangent calculation that matches how Unity "binormal" is calculated and use OnPostprocessModel() to just completely recalculate tangents to match.

    So in terms of xNormal, just use an existing basis that is well-defined and has the bitangent as a cross with flip in tangent w, then copy that code and put it in OnPostprocessModel(), and you should be good to go.
     
    Last edited: Sep 24, 2011
  29. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    Pity that importing bitangents never made it in... Has anyone ever done any digging around to see if they can customize the FBXImporter? IMO model importing really should be open source or at least very transparent, that would be a big help.
     
  30. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    I tried to copy Maya 2010's tangent basis from a file I found in the SDK. It doesn't match at all yet, so maybe I made a mistake porting it from C++. I don't have any experience with it so I wouldn't be surprised. Maybe someone else can take a look or build off of it.

    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. /***************************
    5.  *
    6.  * Pulled from Maya 2010 SDK
    7.  * hwDecalBumpShader_NV20.cpp
    8.  *
    9.  * NOTE: Maya computes bitangents first and then makes tangents orthogonal
    10.  *
    11.  * ************************/
    12.  
    13. class Maya2010TangentSpaceSolver
    14. {
    15.     // Compute array of binormals. Return the pointer to the array if succesful,
    16.     // NULL otherwise.
    17.     public static void Solve ( Mesh theMesh, bool mirrored )
    18.     {
    19.         int vertexCount = theMesh.vertexCount;
    20.         Vector3[] vertices = theMesh.vertices;
    21.         Vector3[] normals = theMesh.normals;
    22.         // assuming 1st UV set
    23.         Vector2[] texcoords = theMesh.uv;
    24.         // .triangles is the list of vertex indices, 3 per
    25.         int[] indexArray = theMesh.triangles;
    26.         int triangleCount = indexArray.Length / 3;
    27.         Vector4[] tangents = new Vector4[vertexCount];
    28.         Vector3[] bitangents = new Vector3[vertexCount];
    29.         int tri = 0;
    30.        
    31.         for (int i = 0; i < triangleCount; i++)
    32.         {
    33.             int i0 = indexArray[tri];
    34.             int i1 = indexArray[tri + 1];
    35.             int i2 = indexArray[tri + 2];
    36.            
    37.             // Get the xyz coords of the corners of the triangle.
    38.             //
    39.             Vector3 v0 = vertices[i0];
    40.             Vector3 v1 = vertices[i1];
    41.             Vector3 v2 = vertices[i2];
    42.            
    43.             // Get the st coords of the corners of the triangle.
    44.             //
    45.             Vector2 uv0 = texcoords[i0];
    46.             Vector2 uv1 = texcoords[i1];
    47.             Vector2 uv2 = texcoords[i2];
    48.            
    49.             // Get the normals at the corners of the triangle.
    50.             //
    51.             Vector3 n0 = normals[i0];
    52.             Vector3 n1 = normals[i1];
    53.             Vector3 n2 = normals[i2];
    54.            
    55.             // Get pointers to the binormal vectors.
    56.             //
    57.             Vector3 b0, b1, b2;
    58.            
    59.             // *********************************************
    60.             // ******** compute the binormal vector ********
    61.             // *********************************************
    62.             Vector3[] plane = new Vector3[3];
    63.             Vector3 c0, c1, c2;
    64.             Vector3 s0, s1;
    65.            
    66.             // Calculate plane equations for the planes defined
    67.             // by the (x, s, t), (y, s, t), and (z, s, t) coords.
    68.             int idx;
    69.             for (idx = 0; idx < 3; ++idx)
    70.             {
    71.                 // Set up the three corners
    72.                 c0 = new Vector3(v0[idx], uv0[0], uv0[1]);
    73.                 c1 = new Vector3(v1[idx], uv1[0], uv1[1]);
    74.                 c2 = new Vector3(v2[idx], uv2[0], uv2[1]);
    75.                
    76.                 // Calculate two sides
    77.                 s0 = c0 - c2;
    78.                 s1 = c1 - c2;
    79.                
    80.                 // Calculate the normal of the plane
    81.                 plane[idx] = Vector3.Cross(s0, s1);
    82.             }
    83.            
    84.             // Now solve for the texture gradients dsdx, dsty, dsdt, ...
    85.             Vector3 ds = Vector3.zero;
    86.             Vector3 dt = Vector3.zero;
    87.             Vector3 dn = Vector3.zero;
    88.            
    89.             for (idx = 0; idx < 3; ++idx)
    90.             {
    91.                 ds[idx] = -plane[idx][1] / plane[idx][0];
    92.                 dt[idx] = -plane[idx][2] / plane[idx][0];
    93.             }
    94.            
    95.             dt.Normalize();
    96.             dn = Vector3.Cross(ds, dt);
    97.            
    98.             // Make sure that our computed normal vector points in the
    99.             // same direction as the input normal vector.
    100.             Vector3 normal = n0;
    101.             if (Vector3.Dot(normal, dn) < 0)
    102.             {
    103.                 // They pointed in different directions, negate
    104.                 ds = -ds;
    105.                 dt = -dt;
    106.                 dn = -dn;
    107.             }
    108.            
    109.             // [This might look funny but I'm trying to preserve the same logic]
    110.             // Compute the biNormal vector and store them in the biNormal array
    111.             Vector3 vn0 = n0;
    112.             Vector3 vb0 = Vector3.Cross(vn0, dt);
    113.             vb0.Normalize();
    114.             b0 = vb0;
    115.            
    116.             Vector3 vn1 = n1;
    117.             Vector3 vb1 = Vector3.Cross(vn1, dt);
    118.             vb1.Normalize();
    119.             b1 = vb1;
    120.            
    121.             Vector3 vn2 = n2;
    122.             Vector3 vb2 = Vector3.Cross(vn2, dt);
    123.             vb2.Normalize();
    124.             b2 = vb2;
    125.            
    126.             // Set the result to our array
    127.             bitangents[i0] = b0;
    128.             bitangents[i1] = b1;
    129.             bitangents[i2] = b2;
    130.            
    131.             Vector3 newTan = new Vector3(tangents[i0].x, tangents[i0].y, tangents[i0].z);
    132.             //newTan -= normals[i0] * Vector3.Dot(newTan, normals[i0]);
    133.             newTan = Vector3.Cross(bitangents[i0], normals[i0]);
    134.             newTan.Normalize();
    135.            
    136.             if (Vector3.Dot(normals[i0], Vector3.Cross(newTan, bitangents[i0])) >= 0)
    137.                 tangents[i0].w = 1;
    138.             else
    139.                 tangents[i0].w = -1;
    140.            
    141.             tangents[i0] = new Vector4(newTan.x, newTan.y, newTan.z, tangents[i2].w);
    142.  
    143.  
    144.             newTan = new Vector3(tangents[i1].x, tangents[i1].y, tangents[i1].z);
    145.             //newTan -= normals[i1] * Vector3.Dot(newTan, normals[i1]);
    146.             newTan = Vector3.Cross(bitangents[i1], normals[i1]);
    147.             newTan.Normalize();
    148.            
    149.             if (Vector3.Dot(normals[i1], Vector3.Cross(newTan, bitangents[i1])) >= 0)
    150.                 tangents[i1].w = 1;
    151.             else
    152.                 tangents[i1].w = -1;
    153.            
    154.             tangents[i1] = new Vector4(newTan.x, newTan.y, newTan.z, tangents[i2].w);
    155.  
    156.  
    157.             newTan = new Vector3(tangents[i2].x, tangents[i2].y, tangents[i2].z);
    158.             //newTan -= normals[i2] * Vector3.Dot(newTan, normals[i2]);
    159.             newTan = Vector3.Cross(bitangents[i2], normals[i2]);
    160.             newTan.Normalize();
    161.            
    162.             if ( Vector3.Dot(normals[i2], Vector3.Cross(newTan, bitangents[i2])) >= 0 )
    163.                 tangents[i2].w = 1;
    164.             else
    165.                 tangents[i2].w = -1;
    166.            
    167.             tangents[i2] = new Vector4(newTan.x, newTan.y, newTan.z, tangents[i2].w);
    168.            
    169.             tri += 3;
    170.         }
    171.  
    172.         theMesh.tangents = tangents;
    173.     }
    174.  
    175. }
    176.  
     
    Last edited: Sep 26, 2011
  31. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Trying to write something that mimics 3DS Max's normal and tangent generation - that way you can use the 3DSMax Scanline equivalent setting for xNormal to bake with.

    Code's a bit of a pain as it's all in C and it uses a bunch of 3DS max's inbuilt functions and classes, which I'm having to dig out and convert to java... but it's getting somewhere.

    Will keep you posted :)
     
  32. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    I bet you're going to run into some of the same problems I have. The tangent and bitangent is NOT orthogonal to the vertex normal. And since Unity can't import bitangents and instead sets them to be orthogonal, you're going to have a tangent space mismatch.

    The main cause is that max calculates the tangent space orthogonal to the FACE normal, not the individual vertex normals. You have seen those two blog posts about max tangent space, right? The code is all right there in one place.

    In general, ANY solution that we try and come up with will be hosed if we can't import bitangents, that is unless the bitangents are calculated exactly the same way that Unity calculates them: orthogonal to tangent * tangent.w, the sign of which is calculated within Unity.

    OH. I almost forgot. The number of vertices, normals, and tangents changes depending on Unity's TangentSpaceImportMode. I haven't fully tested the effect it has, but it's definitely something to look out for.

    Here's an example from a good test model I got from an expert in working out tangent space problems. The FBX file has 133 verts (from 399 floats), 786 indices (i.e. 262 tris), 133 normals (399 floats), 786 tangents (from 2358 floats), 786 "binormals" (from 2358 floats), 293 UVs (from 586 floats), 786 UV indices. In Unity, based on the TSIM mode, and Normals set to Import, I get the following:
    • Import - 786 indices, 210 verts/normals/tangents/UVs
    • Calculate - 786 indices, 207 verts/normals/tangents/UVs
    • None - 786 indices, 207 verts/normals/UVs and 0 tangents
    This could certainly be an issue in cases where the vertex's position, normal, and UV are the same, but the tangent and bitangent are different, if the Unity mesh simplification logic decides to combine things not based on a differing tangent (and of course it won't be based on a differing bitangent since that is ignored).
     
    Last edited: Sep 29, 2011
  33. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Yeah, using the stuff on Autodesk's blog. Got it recreating the mesh's normals and tangents using what's now pretty close to Max's own basis (need to double-check a few more functions)... it's close, but it's not correct yet.

    The binormal thing is no problem. Max calculates it by cross product of normal and tangent, then checks if the map normal is flipped and negates the bitangent if it is. Which is exactly what Unity does - using the w component of the tangent to determine whether the binormal should be flipped or not.

    The real issue I'm having is that calculating the normals for the imported mesh as Max calcs them gives all the UV seams a hard edge and I'll be damned if I know how to get past that. Max holds all of that information separately for each vertex, but Unity holds the information uniquely for each individual vertex, so the smoothing groups can't be reconstituted separately from the UV splits.

    And either Max's FBX Export tangents or Unity's Import tangents appears to be entirely sodding useless...

    So, assuming Unity's normal importing isn't as shafted as it's tangent importing, I'll have to rely on the normals I'm given and hope they're right...
     
    Last edited: Sep 29, 2011
  34. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
  35. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    Yeah that link is very good news. :)

    And you are right, I read it wrong, the binormal is also being calculated per face instead of per vertex, so I guess it's fine... The confusing part is the summing for smoothed faces, though. I don't get why the basis is being added instead of just set if it's a per-face calculation.

    ...and holy crap is the mikktspace code thorough! Ex:
     
    Last edited: Sep 30, 2011
  36. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    You calculate tangent per-face (each face shares UV triangle, so it gives the same result whether you calculated it per face or per vertex of that face) and apply it to the vertices of that face. If that vertex is shared by another face, it'll add the tangent to it. Once it's all calculated, you normalize the value and get the smoothed tangent for that vertex (and any non-smoothed vertex only has one or two tangents applied to it, so it remains hard on one edge).

    Yeah, the mikktspace thing is pretty impressive. If Unity already uses it then there's an extant XNormal baker there already, which is cool.

    If not, I've been reading elsewhere of Aras pointing folk at the Lengyel method... so perhaps Unity uses that?
     
  37. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Sod it, I give up.

    If Unity does use mikktspace, either they or xNormal are doing it wrong. It's close - about as close as I've seen, but it's still buggered.

    I can't afford the time to find out which contrived method of exporting/importing/baking/goat sacrificing workaround I need to use to get proper normal maps.
     
  38. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
  39. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    I guess I don't understand why you calculate tangents per face instead of per vertex. Does max still only support smoothing groups for editing normals? If you can edit normals manually, then you certainly can't rely on faces either being averaged or flat.

    I guess I assumed Unity doesn't use mikktspace yet. I figured it would be a 3.5 thing since his gamedev.net post was 4 months ago. The generation step requires a complete re-indexing plus both vertex and pixel shader calculations, so I would guess switching over would require quite a few changes and testing, especially when you support about 10 platforms. :p

    Regarding the Lengyel approach, I have tested that and it seems to be messed up in mostly the same way as an import from Maya. Lengyel vs. Maya with a Maya normal map is about a 90% match just speaking qualitatively. Default Calculate is quite different. I'm friends/ex-boss of one of the 3Point guys and got a good test model from him, which looks great in Maya minus a couple quirks. I also asked about tangent space generation with their shader and he said they're very tight-lipped and last time someone asked about it on Polycount it led to some drama (my words), heh.

    But looking at the 3Point shader code, they send their vertex data from max through uv coords, so they've already been calculated at least to some degree. If the mesh was set up the same in Unity and if you could import bitangents, then you could get a similar result as their shader.
     
    Last edited: Sep 30, 2011
  40. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Yeah, I was curious about that. 3P Modifier stores the values in vertex color channels, so tangent could be copy/pasted to channel 1 and normal to normal and then reconstruct binormal in the shader. Then do that strange rsqrt(dot(vector,vector))*vector thing they do in the shader.

    But it's hardly a magic bullet :/
     
    Last edited: Sep 30, 2011
  41. Cameron_SM

    Cameron_SM

    Joined:
    Jun 1, 2009
    Posts:
    915
  42. Kuba

    Kuba

    Moderator

    Joined:
    Jan 13, 2009
    Posts:
    416
    Unity imports bitangents from the FBX file, but only uses them to determine the sign for the bitangent stored in tangent.w.

    Yes, Unity uses a slightly modified Lengyel's approach. A quick look doesn't really reveal what this modification is though... ;)

    What fofo mentioned on the feedback page:
    "AFAIK, xNormal 3.17.6 Beta 1 allows to import the Unity's tangent space using FBX files.
    http://eat3d.com/forum/beta-and-release-candidates/xnormal-3176-beta-1"
    is the result of my talks with Santiago from the xNormal team. This ensures that the tangent basis exported in the FBX file is both used in xNormal and in Unity (when both normals and tangents are set to "Import").

    The above solves one path (importing tangent space info from file), but there is one left -- calculations being the same between applications, which is important when using file types other than FBX.

    There is no reason (apart from development time needed) not to switch to mikktspace, especially since both xNormal and Blender already switched to that approach. I'm planning to do it for the version coming after 3.5. No time for 3.5, sorry guys.
     
  43. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    Kuba, thanks for the info! It is much appreciated to have an official word on all this.

    Sounds like we're just a few months away from mikktspace, and in the meantime xNormal is the best bet.
     
  44. EiknarF

    EiknarF

    Joined:
    Apr 28, 2011
    Posts:
    142
    I would love to know whats happening with mikktspace and unity, any updates?
     
  45. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Kuba : Any chance you could find out exactly what modifications to Eric Lengyel's method you're using?

    I've written a plugin for xNormal that replicates the code found on Eric's site ( http://www.terathon.com/code/tangent.html ) but the baked maps are still not matching up with whatever Unity is doing internally with it's tangent space.
     
  46. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Ok, almost nailed this. Works with tri-only meshes at the moment, just need to rejig it work with quad tri meshes and give it some more testing.

    It's not 100% perfect and you've gotta normalize the light direction in the shader... but it's pretty damn close.

     
    Last edited: Mar 1, 2012
  47. God-at-play

    God-at-play

    Joined:
    Nov 3, 2006
    Posts:
    330
    NICE WORK

    I can't wait to have decent normal maps in Unity ^_^
     
  48. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
  49. copenhagenjazz

    copenhagenjazz

    Joined:
    Oct 2, 2005
    Posts:
    45
    Hi James

    Thanks for keeping on truckin on this issue ;)

    However I didnt manage to get it workin properly. This was my result when I tried generating a normalmap with your tangent generator;



    As you can see there are some artefacts in the normalmap and it doesnt look too sharp in Unity either.

    Basically here is what I did;
    - Created and unwrapped the object in max (2010). One smoothing group for each UV isle
    - Exported high and lowpoly as fbx without the tangents and binormals
    - Generated the normals in xnormal using your TB calculator
    - Imported the same lowpoly fbx into unity importing the normals and calculating the tangents

    Does this sound like the right way or did I do something silly somewhere?

    Cheers
     
  50. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Damn, sounds like you did it right.

    Any chance you could email me the FBX files and I can try and figure out what's going wrong? jamesohare@gmail.com