Search Unity

3DS Max 2010 to Unity object smoothing problem.

Discussion in 'Formats & External Tools' started by Bryan A, May 18, 2010.

  1. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Hi, I have model a character and exporting it with both FBX and Obj file format but still there is something wrong with the smoothing in you Unity. I view and render it in 3DS Max with no smoothing problem. This is how it looks it Unity

    http://img265.imageshack.us/img265/6328/whyk.jpg

    http://yfrog.com/7dwhykj

    Is there something that I must do in Unity to get the smoothing right?
     
  2. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    Try to triangulate it in max (if you didn't do it). As we triangulate polys - the bug might be in here somewhere.
     
  3. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Triangulate it in max? You mean I triangulate all my quads to triangle? I thought Unity can do that job?
    Or I maybe be misunderstood you Alexey.
     
  4. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    Hey, just try it out 8). Backup your model triangulate and re-import.
    I'm just looking at possible problems.
     
  5. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    I've tried it and it work. But the same problem happen again after I start to apply the normal map. If I only apply the diffuse map, its still ok. I must have screwed up the retopologizing the mesh loop and normal mapping process.

    What a waste of time. So now I know, if it works in Max, doesn't mean it gonna work in any 3D game engine. :(
    Wish I had known about this earlier.
     
  6. br0kenp0ly

    br0kenp0ly

    Joined:
    Jun 3, 2008
    Posts:
    481
    That seems like a smoothing groups issue. Try to fix your smoothing groups so that it looks good in 3dsmax, then take it into Unity and make sure the "Calculate Normals" option is turned off.

    Should work, but please keep us posted if it doesn`t.

    Edit: You could also try to select all the vertices in your model and weld with a very small threshold. It could be that some edges are overlapping, so either do this or try to turn them manually in the editable poly modifier panel.
     
  7. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    I have tried what asked me to do Brokenpoly, but it is not the problem.
    I've tried putting a smooth modifier on top and collapse my model back to edit poly and when I export it in FBX or trough 3ds max auto conversion in Unity.

    The low poly seem to be all smooth out fine, until I start applying back the normal map and the same error appear. Could it be the Error from the normal map?
    If so, I tried to paint the normal where the problem is but still no luck.

    Btw I've sent to Alexy the model and hope he can find the source of the problem.
     
  8. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    http://yfrog.com/5jimportkmj
    so, in the very-very latest version it looks like this (imported from fbx as is - without triangulation).
    So the simple answer is: if you are ok with slightly convoluted workflow - wait for 3.0 ;-)
    the long answer - i can check what really was (could be) the issue. And i check if i can create for you special exporter version (maya plugin, or unity dll) to ease your life a little.
     
  9. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    Also, brokenpoly really does have a point. Cause i get artifacts similar to yours if i check "Calculate Normals". So you should have it turned off. (please, recheck).
    And even if you want it on (it can mess with your smoothing) you can select smoothing angle manually in unity. But better - no "Calculate Normals" at all ;-)
    I'll look into possible problems with normal map later
     
  10. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Hey thanks Alexey,If you can spot the problem, it would be good so that I know next time what to look out for when modeling 3D model for games.
    I didn't use any of that calculate normal. I've tried it out and still not the solution.
    I tried FBX and OBJ export and it seem that FBX gives good smothing, with OBJ the smoothing is much worst. But nevertheless, both still gives same problem when I applied my normal map.
     
  11. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    ok, i'll look, but just to clarify, your bad output was from fbx WITHOUT "Calculate normals"?
    also
    so for now problem only in normal map? or it is simply not that bad.
     
  12. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    The smoothing from FBX is ok i guess. Its just that when I apply my normal map, it turns out like in the picture. So could be from my normal map? But can a normal map really do that kind of problem? If so then why when I view it in 3ds max 2010 or xnormal 3d viewer, it doen't gives such error?
    Forget about .OBJ. Maybe OBJ and unity has got some error in giving good smoothing.


    A note. This is actually my 1st serious attempt in getting my finished character inside Unity.
     
  13. br0kenp0ly

    br0kenp0ly

    Joined:
    Jun 3, 2008
    Posts:
    481
    Hey Bryan!

    Could you by any chance send me that model + normalmap, so I can have a look at it? Would be interesting to see what`s going on with it, so feel free to send me a mail on:

    roald [at] unity3d.com
     
  14. br0kenp0ly

    br0kenp0ly

    Joined:
    Jun 3, 2008
    Posts:
    481
    Ok. I got the model now and after some magic it seems the model works just fine. I`m not using 2.6 at the moment though, so I won`t be able to test it in that version until tomorrow. Will keep you posted :)

     
  15. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Sorry for late reply Brokenpoly. Yes!! that is exactly how I saw it in 3DS Max and Xnormal 3D viewer.
    What version of 3DS Max you're using? How do you get it to work just fine?
    Could it be an incompatibility issues with 3DS Max2010 and Unity 2.6.1?

    Can't wait to see you result with Unity 2.6.1.
    Thanks mate!!
     
  16. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    Nope, it is exactly unity. It's just current developer's build. He's talking with rendering guy right now, to better understand how ok-ish it that ;-)
     
  17. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Ok, thanks for the support guys. I'm glad there is nothing wrong with my workflow or 3DS Max.

    Can't wait for the fix. :D
     
  18. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Have you tried "Split Tangents" option in mesh import settings? That is specifically to split tangent space across UV discontinuities.

    (I think that option should be the default... let's see if I remember to do that for next Unity release :))
     
  19. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    You mean the FBX Importer Split tangent setting in the Unity Inspector panel? Yup, tried and play all of the setting in the inspector but still didn't fix the problem. I'm so curious as why 2.5 version has no problem about it.
    Have you guys in Unity tried in 2.5 too?
    What do Brokenpoly mean by this"Ok. I got the model now and after some magic it seems the model works just fine."
    Did he somehow do some script tweaking?
     
  20. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    ok, what brokenpoly did - just disabled tangents calculation. So it works but not really correctly (light artifacts will come), cause shader needs them. Split tangents in itself won't help you much, simply because you have ugly stuff in continuous areas of uvs ;-). Ok, i found the best combination for you (at least for now - we better tune tangent space calculation for 3.x ;-)):
    in (FBXImporter) pane of inspector
    generation = all (tangents and normals)
    calculate normals : on
    smoothing: max (180)
    split tangents: on
    while these settings are far from perfect (read: you can lost some hard edges - depending on how they were created and handled by unity) - but anyway - you'll get nice model ;-)

    NOTE: i did this in current build - if something is absent in 2.6 - ping me
     
  21. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    OMG!! :D It works!!! It is fixed!! Thanks for the help guys! I appreciate your help very much.
    I got a few more question.

    1.What texture format is the best for textures? So far the ARGB 32 and RGB 24 works best for me. What is the down and up side of using this format?

    2. I'm using the Bumped Specular shader. I notice that there are no Texture slot for specular, only can be controlled by shininess and specular colour. Could there be other shader that let be put my own specular colour map? Or is this just it?
     
  22. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    1. basically the input image will be converted (preprocessed) based on TextureInspector->Format. Alpha is basically up to you (will you need it or not - is is usual to move to alpha additional stuff to be read later by shader). So upside/downside for source image is just alpha (if you don't need and provide it - you waste you memory, e.g. dxt1 works only with no-alpha images)
    2. Here we go with alpha once again. For bumped specular (read inspector carefully) - diffuse RGB is diffuse and alpha is for glossiness. Although if you want not glossiness but specular color you should write shader yourself (or find the one readily available ;-))
     
  23. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Emm.. let me try to figure this out. So you were saying if I use the ARGB 32(comes with alpha). I would get the glossiness level controlled automatically by the image alpha?
     
  24. spacefrog

    spacefrog

    Joined:
    Jun 14, 2009
    Posts:
    734
    Just to jump in here to post my observations about "editable poly" and "triangulation issues" with Max after exporting to latest versions of FBX (don'T know if this is true for older versions too).
    The FBX format seems to tread a poly object exported from max to FBX as a Poly only: no triangulation info is getting stored, this info seems to be STRIPPED from the FBX. The result is that if you make changes to your editable poly triangulation (via "Edit Triangulation", "Retriangulate" or "Turn") those changes will NOT SURVIVE a FBX export.
    You can test this simply by exporting an editable poly where you specifically "turned" some internal (aka invisible) poly edges and export to FBX. After you import the FBX again into Max, your triangulation changes are lost, seems FBX itself decides how to retriangulate the polys internally on import.
    To keep your edited triangulation on editable polys when exporting to FBX, simply place a "Turn to Mesh" modifier on top of the stack ( "Select Mesh" will not do it...). After that, the triangulation is kept in the FBX, and Unity (and Max'S FBX importer) are able to recieve the correct triangulation....
    I posted this because maybe this clitch can cause the bad shading the OP had problems with.

    To the OP:
    could you try placing a "Turn to Mesh" modifier onto you object prior FBX export and check if this removes your normal map shading probs in Unity ?
     
  25. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    I have tried putting the turn to poly modifier like you ask and it doesn't fix it. I actually have no problem with the smoothing from importing with FBX. It's just when i apply my normal map, that problem happen. So I need to generate normal 180 and split tangent on to fix it.
     
  26. spacefrog

    spacefrog

    Joined:
    Jun 14, 2009
    Posts:
    734
    i meant the "Turn to Mesh" modifier, just in case...
    It might be still be possible that that flipped edges might cause problems, because the normals depend on the triangulation too...
     
  27. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    I have tried turn to mesh too. Still the same error. I don't think I am running into the problem anymore as Alexey tips do work, but there is still 1 thing that bugs me. When I apply the bumped specular shader, the specular looks kind of low resolution. I have tried all the texture format and still the specular looks wierd. I even embeded my own grey scale of alpha to the diffuse in photoshop as it should be the source of the specular level and still the same weirdness. My texture resolution is 2048 and it shouldn't be a problem.

    This the the result I am talking about

    http://img193.imageshack.us/img193/1961/whyman.jpg
     
  28. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
  29. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    in the texture inspector for your diffuse/normal - what are the setting in the bottom (max size, compression).
    By default it will be compressed. You'll get better results with uncompressed (as you use max as reference - it uses uncompressed ;-)), but it will eat up more memory
     
  30. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    You mean uncompressed is like RGB24, ARGB32?
    I have tested all of the texture format but still the same result.
    I'm using the non Pro version of unity, so could there be an extra texture feature that I don't get that Pro has to make it more nicer?
     
  31. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Any response to this question? Anybody? If it is then I guess I should get the Pro soon.
     
  32. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    No. In Texture inspector. Unity converts your source texture.
     
  33. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    I guess its just Unity limitation for the given shader provided. Guess if I need a damn high quality specular or anything else. I might need to script the shader which I doubt I would.
    It's ok, I think I got no more problem and question. Maybe Unity 3 would so much superior.
    I love this game engine. Saving money to get Unity 3 pro. I'm so excited about it.
    Thanks for all your support guys. :D
    Now, on to make more nice game asset and game too!
     
  34. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    Just in case, i'm talking about this.
    Try to select true color in there for both diffuse normal and see if artifacts go away
     

    Attached Files:

  35. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    Yup, I've tried the true color(ARGB32) and still nothing happen.
     
  36. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,624
    hm, that's strange. Although maybe the shaders were changed (i'm using 3.0 as you might imagine ;-))
    so maybe just wait a little till 3.0 came out to properly test this out 8).
     
  37. Bryan A

    Bryan A

    Joined:
    Feb 9, 2010
    Posts:
    43
    There is no compressed or uncompressed setting like in the picture you shown in 2.6.1. I guess it is only on version 3.
    Mean time, I'm having fun playing around with Unity. :D