Search Unity

Editing prefab causes references to it to become null until the object referencing it is reloaded

Discussion in 'Prefabs' started by Prodigga, Oct 12, 2018.

  1. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    @SteenLund @LeonhardP @runevision Hi all - I'm not sure if our problems are directly related to the projects that have been shared here in the thread, but Unity 2019.1 and 2019.2 (currently we're using 2019.2.9f1 on Mac) have been absolutely terrible maintaining references in scenes, prefabs, scriptable objects, etc. in our project.

    We play and stop in the Editor, references randomly get lost. We edit prefabs, references randomly get lost. ScriptableObjects we're using for configuration lose their references to prefabs. Prefabs, nested prefabs, and prefab variants periodically lose all sorts of references to themselves, their parents, their overrides, etc. Materials fall off MeshRenderers randomly during editing or after returning from play mode. Sometimes after stopping play mode, game objects that were loaded during play mode into the 'DontDestroyOnLoad' scene end up in the scene that is open for editing and then saved along with any other changes that might be made.

    We cannot file bug useful reports because we cannot replicate these issues reliably so I'm not sure how best to communicate the problem. We've talked to quite a few other developers and a few plugin authors this year and they all seem to agree that Unity serialisation has been quite brittle this year, with prefabs and lost references being a particular problem.

    We have written editor tools to try to flush Scriptable Objects so they can be reloaded when they lose references. Almost always the data on disk is fine, so it seems to be something happening to loaded assets with lost references. Prefabs are trickier, but eventually unloading, loading, quitting, reloading the editor, reimporting the prefab or scene assets, or outright deleting the Library folder and reimporting everything gets things going again.

    This is obviously a big problem and a massive timesink for us. A quick search of the issues, forums, answers, etc. shows that there have been quite a few 'missing reference' type issues raised over the last year. Is Unity aware of any ongoing serialisation and reference errors with scenes, prefabs, scriptable objects, etc? Are there any fixes in progress at the moment that we can we expect to improve things any time soon?

    Thanks for any response and suggestions you can offer.
     
  2. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    633
    I had similar nightmare problems with 2019.2 - every 4-6 weeks, it would corrupt all ScriptableObject -> ScriptableObject references (which fundamentally destroys your entire project if you're using SO's for anything important).

    At Unity 2019 I asked a few Unity devs about it, and the best idea they came up with was to create a script that intensely triggers loads/reloads of the assemblies, serialiszes SO's (copy, delete, repeat), etc - and see if you can get a point where the script is able to consistently reproduce the bugs within a set amount of time (e.g. if the corruption only happens one time in 1000, a script running continuously might be able to reliably get a reproduction after a minute or two of grinding).

    For me ... it seemed that removing a few raw floats from an SO made all the problems go away (so the Unity bugs I was hitting are possibly to do with sizeof( data structure )), and haven't seen a corruption since - but the "grind" script idea for making something that Unity support would be willing to / able to look into was a good one.
     
  3. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    We never had a response on forum threads or bug reports about these serialisation issues. We're still hitting them in 2019.2.17.
     
  4. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    633
    Well, if we can't give them a repro, I sympathise with them - what are they going to do?

    If it's some codebase change introduced by a different team (e.g. when they were doing the work for multi-scene editing, which broke a LOT of things, and which was prime suspect for me), then it's too late - it should have been caught by the original devs doing better testing of their commits. Going back and re-reading every line of code in the hope of magically spotting something is probably orders of magnitude too expensive.

    The people I spoke to agreed it was both a worrying serious problem - but also one that was very hard to act on or get resource for until/unless we could start getting reliable repros.
     
  5. runevision

    runevision

    Unity Technologies

    Joined:
    Nov 28, 2007
    Posts:
    1,723
    There is not one but multiple different issues with references going missing existing in various places in the interplay between Prefabs, scripting, and the Asset Database. Some have been fixed while others are still open. This is something that's actively being looked at, but as touched upon in the thread already, it's very tricky issues to fix.

    In the mean time we can only recommend the workarounds that you've already found to be useful to have the good data on disk reflected in the imported assets: Reimport Prefabs, or as a last resort, rebuild the Library. We strive to improve upon this situation as soon as possible.
     
    BobFlame, Theroxas and a436t4ataf like this.
  6. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    @runevision thanks for letting us know it is being worked on - much appreciated.
     
  7. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    186
  8. BobFlame

    BobFlame

    Joined:
    Nov 12, 2018
    Posts:
    6
    @runevision One similar issue is still exist in 2019.3.2f1. You make a ScriptableObject reference a Prefab. After you press CTRL+S, the prefab reference stored in the SO assect become "null"(no matter in a list or not), although it appears in editor as normal. Would this be fixed soon?
     
    a436t4ataf likes this.
  9. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    633
    Have you got a simple reproduction? That sounds like one of the bugs I hit, but I couldn't get a reliable repro on it :(. EDIT: I had something like this in 2019.2.2, IIRC.
     
  10. BobFlame

    BobFlame

    Joined:
    Nov 12, 2018
    Posts:
    6
    @a436t4ataf just like @SteenLund said, the best practice now is using prefab reference to the "GameObject", not to the component attached to it. This would avoid lots of reference missing bugs.
     
  11. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    290
    That's an awful workaround though. It's terrible for safely referencing things in the project (mitigated only partly by a custom property drawer), massively hurts code-readability and means you'll have to have a second field to cache a GetComponent() for the mono you're actually referencing for every prefab reference. Doing this would require we go through thousands of lines of code, and then if a fix was rolled out, going back through reverting those changes.

    Since encountering this issue, it's put a big dent in my faith about trusting what's happening behind the scenes; now whenever I encounter a referencing exception, I have to account for the possibility it's just Unity failing to load an asset, rather than my code, which makes debugging much harder and frustrating.

    I have a huge amount of sympathy for Unity given how hard it is to put together a repro around this; after all I can't really expect a fix to be rolled out without one, but it adds to my concerns from last year that the new prefab system was not ready to be rolled out and that such fundamental releases aren't a good fit for the TECH release cycle.
     
    a436t4ataf, BobFlame and angrypenguin like this.
  12. SteenLund

    SteenLund

    Unity Technologies

    Joined:
    Jan 20, 2011
    Posts:
    481
    Since the last post from me on this thread more issues with references to monobehaviours have been found and fixed, not all fixes have shipped to the public yet.
     
    a436t4ataf and BobFlame like this.
  13. BobFlame

    BobFlame

    Joined:
    Nov 12, 2018
    Posts:
    6
    I agree, may be it's not appropriate to be called "best practice". however, it's a possible workaround at this time. so, hope it will be fixed soon.
     
    tweedie likes this.
  14. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    186
    Even when using gameobject instead of a monobehaviour reference i still have this issue
     
  15. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    The serialisation issues are still an ongoing giant pain for us in 2019.2.17 (we are unable to upgrade further just yet due to other bugs in recent Unity versions). Earlier post above: #5092490

    eg. if anyone in our team edits a prefab or nested prefab, checking stuff out from source control breaks references seemingly randomly. It's been like this for over a year and it's pretty tiring wondering what will null ref next and what steps are needed to find and fix it. Sometimes even while editing scenes materials will just go missing and turn magenta. Most of our issues continue to be with prefabs and scriptable objects, though.

    Sometimes manually reimporting the prefab works. Other times reimporting the prefab and some nested prefabs works. Other times the reimporting the scene works. Sometimes restarting Unity works. Other times restarting Unity twice works. Sometimes all of the above.

    @SteenLund This has obviously been a huge productivity killer and we are looking forward to some of the fixes that you're talking about making their way into public releases we can hopefully update to sometime soon. Thanks for your efforts.
     
    Last edited: Mar 25, 2020
    xjjon, angrypenguin and tweedie like this.
  16. VEA_Games

    VEA_Games

    Joined:
    Jun 2, 2016
    Posts:
    53
    This is true sabotage.

    Please, Unity team - make something with this problem.

    Every time when I'm planing to roll-out a new release to Google or Apple - I should recheck ALL references on prefab variations - because of almost every-time something of them are set to null (file ID mismatch).

    UPDATE: the same shthing with simple prefabs (not variations) too

    ---------------------------

    My case

    I have the same issue with prefab variations during last 3 months. I have no upgrades of Unity - it happens randomly. FileID changing and references are set to null.

    There are two refs are now shows as None (sometimes as missing)
    https://gyazo.com/5f2d69d8a715484b5198a9f19aafc4f7

    After I fix an issue, I see that FileID is changed (why?):
    https://gyazo.com/f2f91981f2e248c83b6a1e929c063784

    The prefab variation
    https://gyazo.com/63dd80bd12ef37e7cb36b7df7faeee91
    (prefab variation meta)
    https://gyazo.com/179f74aaf052340666b3b31357d0c658
     
    Last edited: Apr 2, 2020
    xjjon likes this.
  17. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,486
    This isn't a solution, but can you write a script to check these for you to at least take some of the manual effort out of it? Could potentially extend that to also fix them (when they aren't broken, store the refs yourself via some other means, like relative file paths, and use that to replace them when they go missing).
     
  18. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    186
    So I had to write some 'hacks' to get around this, because it is quite annoying and destructive to our game.

    For where possible, we load from relative path in order to avoid this issue (i.e. don't need to have references). So this works where you want to load a set of levels, etc.

    But it's not possible for all cases.

    For example I create a 'level prefab' and then inside i add some 'entity or decoration prefabs'

    When I edit the entity/decorations, it doesn't set the references in the level prefab to null. It's worse! It resets the transform! So I cannot even have prefabs placed in the levels without worrying about their position being reset.
     
  19. VEA_Games

    VEA_Games

    Joined:
    Jun 2, 2016
    Posts:
    53
    Probably, but in my case I think more effective will be to replace prefab variations to simple prefabs.
    I like the feature of prefab variations (one of the couple features I want to see in ue4) but now it seems to dangerous.
     
  20. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,486
    @SteenLund @runevision We're running into this again ourselves just now.

    Project is on Unity 2019.3.0f6. Merged a couple of relatively large branches in our version control yesterday. Things are groovy, incuding when I do a completely fresh pull on a different PC. However, for two of our developers that's not the case - they get a couple of dozen null references in different prefabs when opening the project.
    • Opening the prefab files and looking them up by hand in a text editor the reference is definitely there. The relevant fileID shows numbers which match up with the target object.
    • Looking at the prefab in the Editor the reference is definitely gone. It shows as "None", not "Missing".
    • If they look at the prefab in a scene before they run into the issue, the reference is "None" in the scene but the fileID remains in the file.
    • If they open the prefab in prefab editor mode the reference shows as "None", and (presumably because the prefab was auto-saved) the fileID now shows as "fileID: 0" when looked up in a text editor.
    The specific reference we looked at here is a reference from one object inside the prefab to another object inside the same prefab.

    Potentially useful contextual information. Both of the developers who ran into this issue had, immediately prior, ran into an issue with the PackageManager. We have a dependency which is hosted by a 3rd party on GitHub which was inaccessible because they didn't have a standalone version of Git installed. This caused a bunch of issues which were seemingly solved, but then their prefabs lost a bunch of references. I can't see why a fixed PackageManager issue would be related to components inside prefabs losing references to each other in future runs of the Editor, but that's the only thing I can see that makes those developers different to my two computers where everything is fine. Furthermore, all three of us had just done completely fresh pulls from version control.
     
  21. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,623
    We run into the same bug sometimes. I change a prefab, commit it, some of our developers gets issues when they pull it where the assigned values in the prefab is "none", even through there is a valid GUID there in the .prefab file.

    We haven't had issues with the package manager just before hand.

    Reimporting the prefab does not fix the issue. The only good fix we've found is to delete the prefab, and then discard the deletion in git.

    Creating a repro is hard since I have no clue what triggers it, and it requires two different computers (or at least two different checkouts of the project).
     
    angrypenguin and xjjon like this.
  22. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    186
    Anyone have an open bug report for this?
     
  23. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    290
    Just to add a +1 that this is the only thing I've found so far which resolves it
     
  24. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    I did but they were closed as 'could not reproduce' so they're not exactly helping our cause unless Unity is at least collecting stats on their bugs.

    Serialisation issues have been killing our productivity and sanity nearly every day over the 18 months in 2018.x, 2019.1 and 2019.2. We have not used 2019.3 due to other Unity bugs holding us back so I don't know if our problems will improve or not. Various forum posts suggest it's still a problem. Hopefully the fixes that @SteenLund and @runevision have mentioned make their way to public builds that we can use soon.
     
  25. iMobCoding

    iMobCoding

    Joined:
    Feb 13, 2017
    Posts:
    43
    I have similar problems and I noticed it happens during the renaming of the assets. For example:

    1. I have an Icon.png and Icon2.png in my assets
    2. I have a prefab which references Icon2.png
    3. I don't need Icon.png anymore, so I delete it and then rename Icon2.png to Icon.png
    4. Prefab loses reference to the png mentioned in #2

    I suspect that when you rename Icon2.png to the name of the deleted asset (Icon.png), instead of keeping its own guid, it actually gets guid of the deleted asset. So, at the end, prefab still references to the old guid, which doesn't exist anymore. Maybe that's the core of the problem - during the renaming, if the new name was present before, renamed asset doesn't keep its own guid but probably gets the old one from some cache?

    Best
     
  26. SteenLund

    SteenLund

    Unity Technologies

    Joined:
    Jan 20, 2011
    Posts:
    481
    @iMobCoding

    That sounds like a bug in the asset pipeline, i'll aks the team to investigate.

    @greg-harding
    The fixes are coming to 19.3 but a little slower than usual in these times
     
    a436t4ataf and VEA_Games like this.
  27. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    290
    Are they coming to versions prior to 19.3, even if that won't happen immediately?
     
  28. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    411
    Thanks for the update @SteenLund :)
     
  29. iMobCoding

    iMobCoding

    Joined:
    Feb 13, 2017
    Posts:
    43
    One more thing I just noticed. During extensive prefab editing, I saw that prefab changes its own guid from time to time, without any obvious need. That leads to my scene losing prefab references. And that happens only if prefab auto save is on. If auto save is off and I manually save, it doesn't happen
     
  30. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    290
    I, and several others I know, have had a variety of strange things happen sporadically with auto save on, with no real way of reproducing it. My advice would be to always leave it off.

    On a separate note, I've never personally seen the point of the auto save; I think it's capable of a lot more harm than good, especially for hobbyists that might not have their project on version control - where they could make a breaking change with no way to revert. Sure, that's "their fault" for not using a VCS, but I expect that accounts for a large percentage of the userbase. If you leave it off and have unsaved changes, you get a save dialogue anyway, which means you can treat your time in prefab mode as a completely safe sandbox.
     
  31. SteenLund

    SteenLund

    Unity Technologies

    Joined:
    Jan 20, 2011
    Posts:
    481
    @tweedie
    We are not going to backport further back the 19.3

    @iMobCoding
    That sounds really strange. Do you have any idea how we might reproduce this, we have never encountered this issue
     
  32. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,623
    Is the S in LTS a lie, or are you actually worried that making the prefab system not be broken will break something else?

    "Use LTS to make your games if stability is important to you. Bugs that arbitrarily break your data will of course not be fixed. That'd be crazy, who wants that?"
     
    a436t4ataf and tweedie like this.
  33. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    290
    I am a little baffled by that. 18.4 LTS came out in May 2019. It is meant to be supported for 2 years after that.

    Prefabs aren't just some extra feature we opt into expecting some level of risk, like the SRPs, they're fundamental to workflow in Unity and non-optional. These breaking changes came with the new prefabs introduced in 18.3! How are those working in versions 18.4-19.2 meant to have any faith in their project without these fixes?
     
  34. iMobCoding

    iMobCoding

    Joined:
    Feb 13, 2017
    Posts:
    43
    Unfortunately not. In my current project it happens pretty often with one prefab, but not to others. And even with problematic one, I couldn't find a pattern...
     
  35. SteenLund

    SteenLund

    Unity Technologies

    Joined:
    Jan 20, 2011
    Posts:
    481
    @Baste
    I meant we are not backporting further than 19.3

    So the fixes will be in 19LTS
     
  36. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    633
    So these fixes don't fix any of the bugs in 18.4? Those are different bugs?
     
  37. SteenLund

    SteenLund

    Unity Technologies

    Joined:
    Jan 20, 2011
    Posts:
    481
    @a436t4ataf

    Some of the bugs are new bugs, some of the fixes rely on AssetDatabase V2 that was introduced in 19.3.
    I'll take a chat with the about what can go to 18.4, if anything.
     
    a436t4ataf likes this.
  38. DigitalSalmon

    DigitalSalmon

    Joined:
    Jul 29, 2013
    Posts:
    51
    +1 Getting endless issues with prefab dereferencing.
     
  39. Vitaly_Unity

    Vitaly_Unity

    Unity Technologies

    Joined:
    Sep 4, 2013
    Posts:
    4
    @iMobCoding
    Hi, I am QA from AssetPipeline team, I tried to reproduce that issue with no luck. Could you please share more details about your setup and anything that might be specific for your case? Like:
    - What exact version of Unity do you use and what version of AssetPipeline (v1 or v2)?
    - Do you use CacheServer/Accelerator?
    - Do you use version control system for the project where you saw this issue?

    I was not able to reproduce this issue while working in Editor on Windows 10 (used different versions, AssetPipeline v1 and v2, tried with Cache Server and without). But I noticed that when you do it outside of the Editor - it really happens, but that is expected behavior (as Editor treats outside renamed asset as a new asset):
    1 Drag and drop two image files (1.png and 2.png) into the Editor
    2 Open Assets folder in Explorer and check .meta files for those image files, remember their GUIDs
    3 Go to the Editor and delete 1.png file
    4 Go to Explorer and rename 2.png to 1.png
    5 Go to Editor to start reimporting files
    6 Go to Explorer, open .meta file for 1.png and check its GUID - it is the same as was GUID for 1.png on step 2

    so my assumption is that version control or something else could cause change on the disc (or make Editor "think" that it is change on the disc)...

    so any additional info will be appreciated
     
  40. iMobCoding

    iMobCoding

    Joined:
    Feb 13, 2017
    Posts:
    43
    Ok, I will try to reproduce exact steps and get back to you. In the meantime:

    - The Unity version in which this happened is 2019.2.21
    - Asset Pipeline v1
    - Cache Server is Local
    - Using GIT for version control, not from command line but GUI ones from Visual Studio 2019 Team Explorer and GitKraken
    - And I never rename files outside Unity, I know how things work with meta files. If I need to do some mass-renaming, I copy files outside of Unity, delete them in Unity (which also deletes meta files), do the renaming externally and then re-import and re-assign everything in Unity
     
  41. IllTemperedTunas

    IllTemperedTunas

    Joined:
    Aug 31, 2012
    Posts:
    127
    Just wanted to say this is a pretty big issue in my project right now, specifically with scriptable game objects that I use for spawn lists.

    Any time I make a change to my NPC variants that all inherit from one core prefab, many of them will randomly disappear from the scriptable object spawn lists.
     
unityunity