Big Issue: Serialization Enforces no API Calls

Hello,

I'm testing 5.4b11 with my plugin, Advanced Multi-Scene. The plug-in needs to know what kind of state the Editor is in to determine what it should be saving as its state. For instance, it's trying to save off what Scene a particular object belongs to, but it obviously shouldn't do this during the build process (where the scene is considered a temporary scene), and don't want to perform editor-only serialization steps during play mode.

The new beta enforces that no APIs can be called during the Serialization callbacks which is breaking this previously working functionality. Specifically, I'm using BuildPipeline.isBuildingPlayer, Application.isPlaying, and AssetDatabase.AssetPathToGUID which was all previously working in all versions of Unity.

Previously, you could call these functions and it would sometimes throw an exception if you were caught on the loading thread. However, many things like inspecting in the Editor, and creating SerializableObjects cause the Serialization callbacks to run on the main thread where calling this functionality is perfectly acceptable.

Please return the old functionality, as there's no way to work around these issues. There still don't appear to be callbacks when builds start or end and caching states for playing vs. not inside a behaviour is really ugly.

 

Hi

Thanks for taking the time to test out your plugin with the 5.4 beta, it really helps us understand the use cases better.

The change that happened in 5.4 is that most APIs are not allowed to be calling during serialization and from another thread by default. There are more details in the "Script Serialization Errors" section of this page in the manual: http://docs.unity3d.com/540/Documentation/Manual/script-Serialization.html

APIs such a Debug.Log, Mathf and others are allowed to be called during serialization and from other threads.

Also note that the errors are currently harmless in that they do not throw a managed exception and do not break any projects.

I think the use case for the APIs you listed make sense and I will add a task for making these safe to call during serialization and from other threads.

 

QA has processed your bug report for this issue (#776880) and we will fix this for 5.4.

 

BuildPipeline.isBuildingPlayer and reading of Application.isPlaying can be made serialization and thread safe, and this will be fixed in 5.4.

AssetDatabase.AssetPathToGUID has never been safe to call and will not be made serialization and thread safe. You need to change your code to take this into account.

 

Is there a method to check if we are on the main thread or not? Often times I'm relying on the fact that the serialization callback is happening on the main thread, not the serialization thread. It seems to occur most often when viewing the object in the inspector, or whenever I create a SerializedObject of the instance in question.

 

Note that most of the Unity API is no safe to call during serialization, even if on the main thread. We might change serialization that is called on the main thread today to be called from another thread in the future.

All the MonoBehaviour callbacks (Start, Update, etc.) are always called from the main thread.

If you are still interested in detecting the main thread, then you do as proposed here:
http://stackoverflow.com/questions/26452609/find-out-if-im-on-the-unity-thread

 

@lukaszunity : We have a major problem similarly with this API with TextMesh Pro, (the mostly widely used text rendering plugin for Unity) please see case # 782065. I was having a hard time reproducing it in a smaller project, but you should be able to see our issue there in the OnValidate() method.

A number of errors are firing here, including:

Load is not allowed to be called during serialization, call it from Awake or Start instead. Called from script 'TextMeshProUGUI' on game object ''.
See "Script Serialization" page in the Unity Manual for further details.
UnityEngine.Resources:Load(String, Type)
TMPro.TMP_FontAsset:ReadFontDefinition() (at Assets/TextMesh Pro/Scripts/TMP_FontAsset.cs:344)
TMPro.TMP_FontAsset:OnValidate() (at Assets/TextMesh Pro/Scripts/TMP_FontAsset.cs:163)

 

It seems a bit odd to me that MonoBehaviour::OnValidate would be called during serialization. I will look into this case and get back to you.

 

Also note that currently in 5.4, any error produced during the build process of a player will cause the build to fail.

This behaviour will become optional in a future beta.

 

Hi, I tried the latest TextMesh Pro and I was unable to reproduce this issue. Could you send a repro project as a bug and post the case number here, then I can have another look.

 

@Jodon: After some discussion internally about making Application.isPlaying and BuildPipeline.isBuildingPlayer thread and serialization safe, we currently leaning towards not doing it. The issue is that will it might be valid in a few use cases, but it will not be in the majority of cases where the properties are accesses from constructors and other threads.

Instead, we are wondering if providing some new functionality in the Unity API would solve your issue. Would it work for you if we had an attribute that specifies that some fields are only serialized for the player? Like so

class MyBehaviour : MonoBehaviour
{
[SerializeForPlayerOnly]
Texture2D bakedTexture;
}

We are also open to other suggestions on how we can make it possible for you to still provide the same functionality in your plugin.

 

Hello Joachim,

The goal of my OnBeforeSerialize method was to ensure that a complete list of the open scenes are saved off with the MonoBehaviour. Having a scene save callback would be really nice, however, I would also need it to execute before entering playmode and builds as you're able to do both of these things without actually saving the scene. I think this is a bit tricky due to the temporary filenames (which is undesirable, see the bullet points below).

I could potentially use EditorApplication.hierarchyWindowChanged in order to try and detect changes to the Scene setup, but I'm worried this is also not its intended usage since it's named hierarchyWindowChanged as opposed to hierarchyChanged. The documentation would indicate this is safe, and acts more like hierarchyChanged. It appears to execute multiple times for simple operations like removing an unloaded scene which worries me slightly (almost like it's meant for OnGUI stuff), but serialization is similar so perhaps I shouldn't be alarmed. Can you confirm this would be a valid use of that callback, please?

If you're looking at plug-in usage of relatively new features, here's some more issues that complicate the code:

• A Scene is not considered loaded during the Awake() process (gameObject.scene.isLoaded returns false). Likewise, you can't call Scene.GetRootGameObjects() during Awake() because it says the scene is not loaded.

• During the build process, the Scene.name of the currently loaded scenes are the temporary file as opposed to the filename on disk. E.g. if you just print out gameObject.scene.name during a PostProcessScene callback, you'll get Temp/01.backup. My code in property AmsSceneReference.scene shows why this is a big problem for me (commented with // Welcome to my hell! ).

• Inconsistency for when objects are saved during the build process of a temporary scene. Sometimes a cross-scene reference is saved (i.e. an object is duplicated) before I get a callback to OnBeforeSerialize() with BuildPipeline.isBuildingPlayer set to true. If you are to introduce a before-scene-save callback I would need it to execute before scene builds.

• The PostProcessScene callbacks occur too late in the frame during a play session in the editor. For instance, you will receive your Awake() and OnEnable() before ever receiving a PostProcessScene which exhibits inconsistent behaviour when building a project vs. playing in the editor. This is why all of my plug-in code now attempts to live in Awake() or Start() and actually duplicates whatever PostProcessScene does (super messy).

• Executing Scene.LoadScene( path ) is inconsistent during runtime and the editor. The documentation says you should be omitting the ".unity" scene extension, but in the Editor that sometimes works, sometimes doesn't. For this reason I have to actually execute it twice (once with the extension and once without):

Code (CSharp):

1.  
2. // Try the editor path first, it works at least in the Editor
3. var editorScene = SceneManager.GetSceneByPath( editorPath );
4. if ( editorScene.IsValid() )
5.    return editorScene;
6.  
7. return SceneManager.GetSceneByPath( runtimePath );
8.  

• Executing a Scene.LoadScene(path) does not execute synchronously (e.g. is not considered loaded until the next frame). @SteenLund has told me there will be an EditorSceneManager version that will load synchronously which will solve my issues, but I haven't taken a look to see if it exists yet.

 

We will for the time being disable the script serialization errors in OnBeforeSerialize (e.g. when writing data), as this always happens on the main thread.

 

Any news on this?
Will disabling the script serialization errors in OnBeforeSerialize also apply to MonoBehaviour::OnValidate?

 

This change will disable the script serialization errors when saving. MonoBehaviour::OnValidate should be used to validate the data of your MonoBehaviours. What is your use case with OnValidate that triggers the errors? Can you provide a small repro project?

 

It looks like this feature snuck in to beta 18/19. By default, the "strict mode" talked about in this thread is disabled. I'm running tests on this now to see if it's still breaking our builds

 

As an update, I haven't been able to build on Windows 7 at all in batch mode. Keep getting a "Native extension not found" error. If I exit the build and rerun the same command, it builds fine.

Don't have time to log a report on this right now.

This was in beta 19.