Editing prefab causes references to it to become null until the object referencing it is reloaded

@SteenLund @LeonhardP @runevision Hi all - I'm not sure if our problems are directly related to the projects that have been shared here in the thread, but Unity 2019.1 and 2019.2 (currently we're using 2019.2.9f1 on Mac) have been absolutely terrible maintaining references in scenes, prefabs, scriptable objects, etc. in our project.

We play and stop in the Editor, references randomly get lost. We edit prefabs, references randomly get lost. ScriptableObjects we're using for configuration lose their references to prefabs. Prefabs, nested prefabs, and prefab variants periodically lose all sorts of references to themselves, their parents, their overrides, etc. Materials fall off MeshRenderers randomly during editing or after returning from play mode. Sometimes after stopping play mode, game objects that were loaded during play mode into the 'DontDestroyOnLoad' scene end up in the scene that is open for editing and then saved along with any other changes that might be made.

We cannot file bug useful reports because we cannot replicate these issues reliably so I'm not sure how best to communicate the problem. We've talked to quite a few other developers and a few plugin authors this year and they all seem to agree that Unity serialisation has been quite brittle this year, with prefabs and lost references being a particular problem.

We have written editor tools to try to flush Scriptable Objects so they can be reloaded when they lose references. Almost always the data on disk is fine, so it seems to be something happening to loaded assets with lost references. Prefabs are trickier, but eventually unloading, loading, quitting, reloading the editor, reimporting the prefab or scene assets, or outright deleting the Library folder and reimporting everything gets things going again.

This is obviously a big problem and a massive timesink for us. A quick search of the issues, forums, answers, etc. shows that there have been quite a few 'missing reference' type issues raised over the last year. Is Unity aware of any ongoing serialisation and reference errors with scenes, prefabs, scriptable objects, etc? Are there any fixes in progress at the moment that we can we expect to improve things any time soon?

Thanks for any response and suggestions you can offer.

 

I had similar nightmare problems with 2019.2 - every 4-6 weeks, it would corrupt all ScriptableObject -> ScriptableObject references (which fundamentally destroys your entire project if you're using SO's for anything important).

At Unity 2019 I asked a few Unity devs about it, and the best idea they came up with was to create a script that intensely triggers loads/reloads of the assemblies, serialiszes SO's (copy, delete, repeat), etc - and see if you can get a point where the script is able to consistently reproduce the bugs within a set amount of time (e.g. if the corruption only happens one time in 1000, a script running continuously might be able to reliably get a reproduction after a minute or two of grinding).

For me ... it seemed that removing a few raw floats from an SO made all the problems go away (so the Unity bugs I was hitting are possibly to do with sizeof( data structure )), and haven't seen a corruption since - but the "grind" script idea for making something that Unity support would be willing to / able to look into was a good one.

 

We never had a response on forum threads or bug reports about these serialisation issues. We're still hitting them in 2019.2.17.

 

Well, if we can't give them a repro, I sympathise with them - what are they going to do?

If it's some codebase change introduced by a different team (e.g. when they were doing the work for multi-scene editing, which broke a LOT of things, and which was prime suspect for me), then it's too late - it should have been caught by the original devs doing better testing of their commits. Going back and re-reading every line of code in the hope of magically spotting something is probably orders of magnitude too expensive.

The people I spoke to agreed it was both a worrying serious problem - but also one that was very hard to act on or get resource for until/unless we could start getting reliable repros.

 

There is not one but multiple different issues with references going missing existing in various places in the interplay between Prefabs, scripting, and the Asset Database. Some have been fixed while others are still open. This is something that's actively being looked at, but as touched upon in the thread already, it's very tricky issues to fix.

In the mean time we can only recommend the workarounds that you've already found to be useful to have the good data on disk reflected in the imported assets: Reimport Prefabs, or as a last resort, rebuild the Library. We strive to improve upon this situation as soon as possible.

 

@runevision thanks for letting us know it is being worked on - much appreciated.

 

I also have this issue still.

Is there a open bug to track this? I found https://issuetracker.unity3d.com/issues/prefabs-references-are-lost-when-modifying-prefab but it is closed now.

Is work around to use 'gameobject' instead of monobehaviour?

Reimport didn't actually work for me

 

@runevision One similar issue is still exist in 2019.3.2f1. You make a ScriptableObject reference a Prefab. After you press CTRL+S, the prefab reference stored in the SO assect become "null"(no matter in a list or not), although it appears in editor as normal. Would this be fixed soon?

 

Have you got a simple reproduction? That sounds like one of the bugs I hit, but I couldn't get a reliable repro on it . EDIT: I had something like this in 2019.2.2, IIRC.

 

@a436t4ataf just like @SteenLund said, the best practice now is using prefab reference to the "GameObject", not to the component attached to it. This would avoid lots of reference missing bugs.

 

That's an awful workaround though. It's terrible for safely referencing things in the project (mitigated only partly by a custom property drawer), massively hurts code-readability and means you'll have to have a second field to cache a GetComponent() for the mono you're actually referencing for every prefab reference. Doing this would require we go through thousands of lines of code, and then if a fix was rolled out, going back through reverting those changes.

Since encountering this issue, it's put a big dent in my faith about trusting what's happening behind the scenes; now whenever I encounter a referencing exception, I have to account for the possibility it's just Unity failing to load an asset, rather than my code, which makes debugging much harder and frustrating.

I have a huge amount of sympathy for Unity given how hard it is to put together a repro around this; after all I can't really expect a fix to be rolled out without one, but it adds to my concerns from last year that the new prefab system was not ready to be rolled out and that such fundamental releases aren't a good fit for the TECH release cycle.

 

I agree, may be it's not appropriate to be called "best practice". however, it's a possible workaround at this time. so, hope it will be fixed soon.

 

Even when using gameobject instead of a monobehaviour reference i still have this issue

 

The serialisation issues are still an ongoing giant pain for us in 2019.2.17 (we are unable to upgrade further just yet due to other bugs in recent Unity versions). Earlier post above: #5092490

eg. if anyone in our team edits a prefab or nested prefab, checking stuff out from source control breaks references seemingly randomly. It's been like this for over a year and it's pretty tiring wondering what will null ref next and what steps are needed to find and fix it. Sometimes even while editing scenes materials will just go missing and turn magenta. Most of our issues continue to be with prefabs and scriptable objects, though.

Sometimes manually reimporting the prefab works. Other times reimporting the prefab and some nested prefabs works. Other times the reimporting the scene works. Sometimes restarting Unity works. Other times restarting Unity twice works. Sometimes all of the above.

@SteenLund This has obviously been a huge productivity killer and we are looking forward to some of the fixes that you're talking about making their way into public releases we can hopefully update to sometime soon. Thanks for your efforts.

 

This is true sabotage.

Please, Unity team - make something with this problem.

Every time when I'm planing to roll-out a new release to Google or Apple - I should recheck ALL references on prefab variations - because of almost every-time something of them are set to null (file ID mismatch).

UPDATE: the same shthing with simple prefabs (not variations) too

---------------------------

My case

I have the same issue with prefab variations during last 3 months. I have no upgrades of Unity - it happens randomly. FileID changing and references are set to null.

There are two refs are now shows as None (sometimes as missing)
https://gyazo.com/5f2d69d8a715484b5198a9f19aafc4f7

After I fix an issue, I see that FileID is changed (why?):
https://gyazo.com/f2f91981f2e248c83b6a1e929c063784

The prefab variation
https://gyazo.com/63dd80bd12ef37e7cb36b7df7faeee91
(prefab variation meta)
https://gyazo.com/179f74aaf052340666b3b31357d0c658

 

This isn't a solution, but can you write a script to check these for you to at least take some of the manual effort out of it? Could potentially extend that to also fix them (when they aren't broken, store the refs yourself via some other means, like relative file paths, and use that to replace them when they go missing).

 

So I had to write some 'hacks' to get around this, because it is quite annoying and destructive to our game.

For where possible, we load from relative path in order to avoid this issue (i.e. don't need to have references). So this works where you want to load a set of levels, etc.

But it's not possible for all cases.

For example I create a 'level prefab' and then inside i add some 'entity or decoration prefabs'

When I edit the entity/decorations, it doesn't set the references in the level prefab to null. It's worse! It resets the transform! So I cannot even have prefabs placed in the levels without worrying about their position being reset.

 

Probably, but in my case I think more effective will be to replace prefab variations to simple prefabs.
I like the feature of prefab variations (one of the couple features I want to see in ue4) but now it seems to dangerous.

 

@SteenLund @runevision We're running into this again ourselves just now.

Project is on Unity 2019.3.0f6. Merged a couple of relatively large branches in our version control yesterday. Things are groovy, incuding when I do a completely fresh pull on a different PC. However, for two of our developers that's not the case - they get a couple of dozen null references in different prefabs when opening the project.

• Opening the prefab files and looking them up by hand in a text editor the reference is definitely there. The relevant fileID shows numbers which match up with the target object.
  
• Looking at the prefab in the Editor the reference is definitely gone. It shows as "None", not "Missing".
• If they look at the prefab in a scene before they run into the issue, the reference is "None" in the scene but the fileID remains in the file.
  
• If they open the prefab in prefab editor mode the reference shows as "None", and (presumably because the prefab was auto-saved) the fileID now shows as "fileID: 0" when looked up in a text editor.

The specific reference we looked at here is a reference from one object inside the prefab to another object inside the same prefab.

Potentially useful contextual information. Both of the developers who ran into this issue had, immediately prior, ran into an issue with the PackageManager. We have a dependency which is hosted by a 3rd party on GitHub which was inaccessible because they didn't have a standalone version of Git installed. This caused a bunch of issues which were seemingly solved, but then their prefabs lost a bunch of references. I can't see why a fixed PackageManager issue would be related to components inside prefabs losing references to each other in future runs of the Editor, but that's the only thing I can see that makes those developers different to my two computers where everything is fine. Furthermore, all three of us had just done completely fresh pulls from version control.

 

We run into the same bug sometimes. I change a prefab, commit it, some of our developers gets issues when they pull it where the assigned values in the prefab is "none", even through there is a valid GUID there in the .prefab file.

We haven't had issues with the package manager just before hand.

Reimporting the prefab does not fix the issue. The only good fix we've found is to delete the prefab, and then discard the deletion in git.

Creating a repro is hard since I have no clue what triggers it, and it requires two different computers (or at least two different checkouts of the project).

 

Anyone have an open bug report for this?

 

Just to add a +1 that this is the only thing I've found so far which resolves it

 

I did but they were closed as 'could not reproduce' so they're not exactly helping our cause unless Unity is at least collecting stats on their bugs.

Serialisation issues have been killing our productivity and sanity nearly every day over the 18 months in 2018.x, 2019.1 and 2019.2. We have not used 2019.3 due to other Unity bugs holding us back so I don't know if our problems will improve or not. Various forum posts suggest it's still a problem. Hopefully the fixes that @SteenLund and @runevision have mentioned make their way to public builds that we can use soon.

 

I have similar problems and I noticed it happens during the renaming of the assets. For example:

1. I have an Icon.png and Icon2.png in my assets
2. I have a prefab which references Icon2.png
3. I don't need Icon.png anymore, so I delete it and then rename Icon2.png to Icon.png
4. Prefab loses reference to the png mentioned in #2

I suspect that when you rename Icon2.png to the name of the deleted asset (Icon.png), instead of keeping its own guid, it actually gets guid of the deleted asset. So, at the end, prefab still references to the old guid, which doesn't exist anymore. Maybe that's the core of the problem - during the renaming, if the new name was present before, renamed asset doesn't keep its own guid but probably gets the old one from some cache?

Best

 

Are they coming to versions prior to 19.3, even if that won't happen immediately?

 

Thanks for the update @SteenLund

 

One more thing I just noticed. During extensive prefab editing, I saw that prefab changes its own guid from time to time, without any obvious need. That leads to my scene losing prefab references. And that happens only if prefab auto save is on. If auto save is off and I manually save, it doesn't happen

 

I, and several others I know, have had a variety of strange things happen sporadically with auto save on, with no real way of reproducing it. My advice would be to always leave it off.

On a separate note, I've never personally seen the point of the auto save; I think it's capable of a lot more harm than good, especially for hobbyists that might not have their project on version control - where they could make a breaking change with no way to revert. Sure, that's "their fault" for not using a VCS, but I expect that accounts for a large percentage of the userbase. If you leave it off and have unsaved changes, you get a save dialogue anyway, which means you can treat your time in prefab mode as a completely safe sandbox.

 

Is the S in LTS a lie, or are you actually worried that making the prefab system not be broken will break something else?

"Use LTS to make your games if stability is important to you. Bugs that arbitrarily break your data will of course not be fixed. That'd be crazy, who wants that?"

 

I am a little baffled by that. 18.4 LTS came out in May 2019. It is meant to be supported for 2 years after that.

Prefabs aren't just some extra feature we opt into expecting some level of risk, like the SRPs, they're fundamental to workflow in Unity and non-optional. These breaking changes came with the new prefabs introduced in 18.3! How are those working in versions 18.4-19.2 meant to have any faith in their project without these fixes?

 

Unfortunately not. In my current project it happens pretty often with one prefab, but not to others. And even with problematic one, I couldn't find a pattern...

 

So these fixes don't fix any of the bugs in 18.4? Those are different bugs?

 

+1 Getting endless issues with prefab dereferencing.

 

Ok, I will try to reproduce exact steps and get back to you. In the meantime:

- The Unity version in which this happened is 2019.2.21
- Asset Pipeline v1
- Cache Server is Local
- Using GIT for version control, not from command line but GUI ones from Visual Studio 2019 Team Explorer and GitKraken
- And I never rename files outside Unity, I know how things work with meta files. If I need to do some mass-renaming, I copy files outside of Unity, delete them in Unity (which also deletes meta files), do the renaming externally and then re-import and re-assign everything in Unity

 

Just wanted to say this is a pretty big issue in my project right now, specifically with scriptable game objects that I use for spawn lists.

Any time I make a change to my NPC variants that all inherit from one core prefab, many of them will randomly disappear from the scriptable object spawn lists.

 

Yeah this is pretty much the issue I have.

We have re-wrote most of our code to load asset from json and stuff but it is a pain in the ass for designer to use those tools instead of the editor.

 

It's good to know we likely don't have an edge case issue and hopefully this can be solved in the future.

As for you Unity devs, it's hard enough keep one project up and running, you guys are having to smoothly solve these issues with as little data loss as possible across a myriad of pipelines which must be some pretty intense considerations and the effort to fix and supply these tools is appreciated, even during crazy times such as these.

 

Ok, tried really hard to reproduce exact steps, but Unity just keeps randomly doing stuff without any reason, thus not enabling exact reproduction steps...

- Unity 2019.2.21, Asset Pipeline v1, Local Cache and git
- Fresh new project from 2d template, though I doubt that's important
- Added Image1.png and Image2.png as only assets
- Added Canvas / Image and assigned Image2 to it
- Dragged that Image to Project to make a prefab from it, and left it in Hierarchy
- Closed Unity
- Made 1st commit so everything is clean

Image1.meta shows guid: 41a871c79cf1b3c40af48b7c51135bff
Image2.meta shows guid: 10fe1f68b32d59643926761e73f3a6a5

- Now, open Unity and delete Image1 and rename Image2 -> Image1
- Close Unity and inspect changes in git (doesn't matter which git client you use, it reports the same)

Here comes Unity weirdness. Sometimes git reports 4 changes and sometimes 3, I really couldn't reproduce exact behavior, although I tried like 100 times. Here are the outcomes:

1. When git reports 4 changes:
- Image1 is modified and has data from Image2 image, which is correct
- Image1.meta has guid: 10fe1f68b32d59643926761e73f3a6a5 (transfered from Image2)
- Image2 deleted
- Image2.meta deleted

In this case I can safely discard git changes and revert the previous state in Unity (before delete/rename), which is what I expected to have

2. When git reports 3 changes
- Image1 is modified and has data from Image2 image, which is correct
- Image2 deleted
- Image2.meta deleted

Here we obviously miss 2nd change where Image1.meta is updated with guid from Image2. In this case, if we discard git changes, project gets messed up, reporting various GUID warnings, like

GUID for asset 'Assets/Gfx/Image2.png' conflicts with:
'Assets/Gfx/Image1.png' (current owner)


Again, I couldn't reproduce exact steps to understand when git reports 4 and when 3 changes. In most times it reports 4 changes if there are no Prefabs in hierarchy which reference any of those images. If there are prfabs, in most cases it shows 3 changes. But again, even that is not always true.

To remind you, after each delete/rename I closed Unity to be sure all changes are saved, and then inspected changes in meta files. I hope devs can reproduce such issues, because it's evident that this is the reason why various bugs appear with prefabs, references, scriptable objects, etc...

 

Are you using Odin for serialization? Or just vanilla serialization?

 

Vanilla

 

Code (CSharp):

1. @@ -115,11 +115,11 @@ PrefabInstance:
2.      - target: {fileID: 8476175317561794800, guid: 7962000a5b6804b488c7f6227f82b428,
3.          type: 3}
4.        propertyPath: m_Radius
5. [B]-      value: 0.4
6.  
7. +      value: 0.5[/B]
8.        objectReference: {fileID: 0}
9.      m_RemovedComponents: []
10.    m_SourcePrefab: {fileID: 100100000, guid: 7962000a5b6804b488c7f6227f82b428, type: 3}
11. [B]---- !u!4 &1562757734743476767 stripped
12.  
13. +--- !u!4 &834242086832068517 stripped[/B]
14. Transform:
15.    m_CorrespondingSourceObject: {fileID: 2171813792884674011, guid: 7962000a5b6804b488c7f6227f82b428,
16.      type: 3}
17. @@ -130,7 +130,7 @@ PrefabInstance:
18.    m_ObjectHideFlags: 0
19.    serializedVersion: 2
20.    m_Modification:
21. [B]-    m_TransformParent: {fileID: 1562757734743476767}
22.  
23. +    m_TransformParent: {fileID: 834242086832068517}[/B]
24.  

As you can see... all I did was change the radius of a collider and it basically broke my prefabs as the GUID changed but it stays the same on all the other references...

now.. if I manually edit the file and change the GUID back to the old one, it will fix the broken stuff.. but really wtf!