Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Upgrading to 2019.3a - FBX Import - Identifier uniqueness violation

Discussion in '2019.3 Beta' started by kalineh, Aug 2, 2019.

  1. kalineh

    kalineh

    Joined:
    Dec 23, 2015
    Posts:
    241
    Testing out some upgrades to 2019.3a, we are getting some fbx mesh import warnings, along with some missing mesh filter references.

    Identifier uniqueness violation: 'Name:SurfaceMesh, Type:Mesh'. Multiple Objects with the same name/type are generated by this Importer. There is no guarantee that subsequent imports of this asset will properly re-link to these targets.
    It seems like the FBX importer cannot handle child nodes with non-unique names. In the model hierarchy for example:

    Root/ChildA/SurfaceMesh
    Root/ChildB/SurfaceMesh

    ^ will give warning for non-unique SurfaceMesh name. I can't seem to find anything in the ModelImporter docs explaining this clearly, or if it's a problem at all (we've had this structure for a long time, but only recently has it caused missing mesh filter references).

    It seems possible to rename all the generated hierarchy nodes on model import (OnPostprocessMeshHierarchy) but this doesn't affect the actual UnityEngine.Mesh instances in the asset.

    Is there some recommended way to deal with this? Can we override the name on import somehow? Force generating unique names? Should it be fixed in some FBX export step instead?

    Seems like it would be rather common but I'm not finding much posting about it.
     
  2. Lynxed

    Lynxed

    Joined:
    Dec 9, 2012
    Posts:
    121
    I'm actually interested in this too. I would prefer to have hierarchy-type matching rather then by name one. Or an ability to choose one. Some of my import stuff relies on object names being the same.
     
  3. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    Hi there!
    TL;DR: When multiple subassets are registered with the same name, we cannot certify they will keep the same local file ids if the source changes because in that particular case ids are assigned sequentially. Which means that if the order changes in the source file, the resulting ids will be mixed up.

    The recommended way would be to prevent/fix meshes names in your fbx export so there is no duplicate. The FBX sdk report them only by name, which prevents us to generate fixed ids when we have multiple ones with the same name.

    The AssetPostprocessor cannot help you in that case because Meshes are already registered in the Importer when you can access them.

    Long story:
    Before 2019.1, local file ids were generated in sequence and stored in the meta-file, with the subasset name as a key, in order to match them with the previous id during the next import. This forces the meta-file to be changed each time a new subassets is created during the import and would trigger a re-import until there is no change to the meta-file.
    When two subasset are using the same name, the next reimport will find two entry for this name in the saved id list and will use them in order. That means if a new subasset with the same name is found before an existing one, it will take that existing id and the original subasset will be assigned to a new one. <- this is really the only moment this warning is for and we feel it's better to let you know that upfront instead of leaving people in the dark when all their references get mixed up in a scene because someone added a new "CubeMesh" at the top of the fbx file hierarchy.

    In 2019.1 we changed that mechanism to make it more deterministic. The local file ids are based on a hash of the registered name and no map is needed anymore.
    Unfortunately, we had to keep a sequential registration if multiple subassets are registered using the same name in the ModelImporter because there is no way to differentiate them. The fbx sdk allows using multiple objects with the same name, especially for meshes, and because we cannot prevent that during the import, we decided it would be better to let users know about it with a warning instead of ignoring the problem. The warning got added in 2019.3 and will be backported to a future 2019.2 release.

    It does not mean we don't support it anymore, or that you cannot do that. It is OK to continue doing so and we will never prevent it for the ModelImporter, it's just that we tell you that references to these meshes can break without notice when the fbx file is changed. If you are aware of this limitation and using that in your development process, it is still fine to do so.
     
    starikcetin likes this.
  4. kalineh

    kalineh

    Joined:
    Dec 23, 2015
    Posts:
    241
    Okay, understood! I'd request some info added to the ModelImporter docs, it seems pretty important!

    Does the FBX SDK not give you the hierarchy info? It seems like generating the hash from a full hierarchy name would be more stable than just the child node name - though it still doesn't prevent some cases. Our testing with maya it seemed that for a given node each child name had to be unique anyway.
     
    fherbst likes this.
  5. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    Yes, we have the hierarchy info for each node in the fbx, but that part is building the transform hierarchy, and not the meshes.
    Multiple nodes may refer to the same mesh instance in the fbx, and at the time we resolve the meshes it is not possible to use that hierarchy. Also the implementation of the FBX importer is very old in Unity, and while we are trying to make changes to make it better, we have to keep it compatible with old stuff, which makes it very difficult to change everything. We will try to see what can be done in the future to fix such bad behaviour...

    While this is prevented in maya, you can do such things in 3dsmax and the FBX SDK allows it anyway, so we still need to support anyone using that functionality in the SDK.
     
  6. kalineh

    kalineh

    Joined:
    Dec 23, 2015
    Posts:
    241
    Thanks for the detailed explanation!
     
  7. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,373
    Is this related the bug report below? I cannot import my CAD based model (Inventor->3dsMax->Unity) since 2019.1 because lots of mesh references are missing on import.
    https://issuetracker.unity3d.com/is...hes-become-broken-when-updating-to-2019-dot-1

    Judging by the post above, this inocrrect FBX import behavior is a new feature and not a bug, yet in the bug report above it is not marked as "By Design".

    The weird thing is that if I import the FBX into Unity, it works fine, but if I upgrade my project form 2018.x to 2019.x, thousands of meshes are missing.

    I am ok with things breaking if I change the mesh source again and then re-importing, can we at least get a project upgrade to a newer Unity version to work?
     
    Last edited: Aug 6, 2019
  8. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    This bug is a particular issue we had with the local file ID migration from 2018.x to 2019.x.
    The next version of 2019.2 (probably 2019.2.2f1) should be the first release with a fix for it, we got the bug report in our hands too late to make a fix and ship it before the latest 2019.1 release.

    So yes, to be clear, the project upgrade should always work, if it does not, it's a bug, but changing the mesh source may break and that's why we added a warning for.
     
  9. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,373
    Ok, thanks for the explanation. Looking forward to the fix.

    By the way, I really appreciate you take the time to explain what is going on with this issue. I was left in the dark for months, not knowing if this was ever going to be fixed at all. Now I know there is light at the end of the tunnel. So again, thanks :)
     
  10. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    If we've already upgraded to 2019.2.0 will this 2019.2.2 fix help us? We've got a ton of missing references as well, just been going through and fixing them by hand. And if it does fix it, if the FBX is again updated will the references stay in tact? Thanks
     
  11. markvi

    markvi

    Joined:
    Oct 31, 2016
    Posts:
    118
    Can you submit a bug report with a minimal project that demonstrates your problem and post the case ID here? That way we can test against your FBX file and make sure it works.
     
    kalineh likes this.
  12. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,373
    Here is my repo: 1141428. It is marked as a duplicate but you might want to double check that it works.

    I had to submit the same project twice to some other QA staff because initially they thought it was caused by a deprecated Texture2D reference which I assured them wasn't the cause, but they wouldn't believe me. :rolleyes:
     
    kalineh likes this.
  13. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    I submitted 1189683 as well. I included some more information, like which unity version my fbx import works on and when it stops working after, with a helper script to check post-reimport which references in the fbx are missing..
     
    LeonhardP and kalineh like this.
  14. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,373
    It is fixed in 2019.2
     
  15. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,136
    QA was able to reproduce the issue: https://issuetracker.unity3d.com/product/unity/issues/guid/1189683/

    For the reproducible that @funkyCoty provided: when upgrading to 2019.2.2f1, at first references are not lost but after Reimport All references become missing.
     
  16. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    I had the same results, yeah. Thanks for working on this! We're time locked, so praying for a fix soon.
     
    kalineh likes this.
  17. SanaHassan

    SanaHassan

    Joined:
    Nov 16, 2020
    Posts:
    4
    can someone please tell me the exact solution to this warning I am new to unity and I just added a model from 3D warehouse and getting this issue.
     
  18. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    This warning is in place to let you know that multiple nodes in your model have the same name and parent and that because of that, there is no guarantee from Unity that the generated Meshes will have a consistent ID.
    The long term result could be that at some point you may update that model and all Meshes in your scene will disappear.

    To avoid the issue (and get rid of the warning), you should make sure that the model you are using does not have any name duplicates in its hierarchy. If you got this mesh from another company you should reach out to them to ask for a fixed version with unique names at each node level. You can also probably open that Model in any other DCC (like Maya or Blender) and rename the faulty nodes.
    Here is a picture that illustrates an invalid and a valid hierarchy from an imported Model:
    upload_2020-11-18_15-5-28.png
     
  19. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,411
    but replacing them is (nearly) impossible for large CAD models..

    (i mean i guess you could create some 3ds max script to add unique suffixes for all duplicates,
    but it just creates extra step, and then if your cad model needs to be updated, you should have same unique names on next import rounds also..)
     
  20. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    I see what you mean. We will discuss it with the team.
    I think the warning is important (we had a bunch of bugs opened related to meshes and components references missing after a model update, which is why we added the warning), but maybe we can add an importer option that would remove the warning for a given model ("I know what I'm doing and I'm fine with conflicting names, don't warn me for this one")
     
  21. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    Why doesn't unity just use the full transform hierarchy in the name, if the names need to be flattened?

    • ParentObject
      • ChildA
        • SubChildA
        • SubChildB
      • ChildB
    Could just be flattened to:
    • ParentObject
      • ChildA
      • ChildA_SubChildA
      • ChildA_SubChildB
      • ChildB
    Right? At least for updating meshes, it could be handled internally this way, so that this isnt doesnt pop up? It's kind of insane to require every single node to have a unique name.
     
  22. bastien_humeau

    bastien_humeau

    Unity Technologies

    Joined:
    Jun 14, 2017
    Posts:
    191
    The hierarchy you are providing is working without any issue.
    Duplicate names at the same hierarchy level can hide further reference issues and will display a warning.
    • ParentObject
      • ChildA
        • SubChild
        • SubChild
      • ChildB
    This hierarchy may have an issue because we have two nodes under ChildA with the same name.
    However, there is no issue with this one:
    • ParentObject
      • ChildA
        • SubChild
      • ChildB
        • SubChild
    Because even if they have the same name they belong to a different parent, so everything is fine.

    To give some more insight, we have to generate a unique FileID for each node (so you can reference them, and their Meshes, Material, components, etc... from your scene or another asset). That FileID is based on the full path of the node, the node name itself, and what the component type is.
    In that case, we combine the path from the root node with the node name (ParentObject/ChildA/SubChild) and add the node type (a GameObject in that case). That will gives us our FileID.

    If you have two children named the same with the same hierarchy path, you end up with the same FileID being generated, which is not possible. To make it work, we arbitrarily add an incremental value to the second FileID so it does not collide with the existing one (and that's when the warning occurs).
    But that solution assumes we always read both nodes with the same name in the exact same order... But then, what happens if the first SubChild node is removed? The second node is now alone and will not have the FileID incremental bump, which means its FileID changes and all references to it get lost... (and even worse, references to the deleted node now point to the second one...)
     
  23. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    I see. Thanks for clarifying!