Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Loading assemblies with Serializable classes at runtime results in null deserializations

Discussion in 'Scripting' started by Baawk, Jan 18, 2019.

  1. Baawk

    Baawk

    Joined:
    Nov 15, 2017
    Posts:
    3
    Hello,

    I've ran into some trouble with the Unity 2018.2.14f1 serializer (using .NET 2.0/3.5 on W7x64) while trying to load an assembly at runtime in the player (Windows Standalone 64). The basic idea is to allow the loading of new assets (using asset bundles) and new data/logic that they might contain. For example, a prefab uses a new behaviour that was not present in the game at compilation build, and re-building the game entirely for each added asset seems infeasible, especially if (vetted) user-created content is allowed.

    To do that, I'm first loading a normal assembly DLL using Assembly.LoadFile. After the assembly has been loaded, I load the asset bundle and fetch my resources as necessary. MonoBehaviours work flawlessly, but custom classes do not. Upon deserialization, the fields are always null.

    Here's a simple example of such a scenario. This code would be placed inside an own DLL, that gets then loaded in the game:

    Code (csharp):
    1. [System.Serializable]
    2. public class ExportedClass
    3. {
    4.    public string ClassText;
    5. }
    6.  
    7. public class ExportedBehaviour : MonoBehaviour
    8. {
    9.    public ExportedClass ExportedClass;
    10.    public string BehaviourText;
    11. }
    In another instance of Unity, add the dll as an asset (so it is available in the editor), create a new asset, add an ExportedBehaviour on it, and make sure that ExportedClass isn't null. BehaviourText can be set for reference. Bundle the asset up. For the consumer, attach the following behaviour on any GameObject and set the paths accordingly:

    Code (csharp):
    1. using System;
    2. using System.Linq;
    3. using System.Reflection;
    4. using UnityEngine;
    5.  
    6. public class TestBehaviour : MonoBehaviour
    7. {
    8.     public string AssemblyFile;
    9.     public string AssetBundleFile;
    10.  
    11.     private void Start()
    12.     {
    13.         Assembly asm = AppDomain.CurrentDomain.GetAssemblies().SingleOrDefault(s => s.GetType("ExportedBehaviour") != null);
    14.  
    15.         if (asm == null)
    16.         {
    17.             Debug.Log("ExportedBehaviour not found; load external essembly");
    18.             asm = Assembly.LoadFile(AssemblyFile);
    19.         }
    20.         else
    21.         {
    22.             Debug.Log("ExportedBehaviour already present; skip loading the assembly.");
    23.         }
    24.  
    25.         var exportedBehaviour = asm.GetType("ExportedBehaviour");
    26.         var exportedClass = asm.GetType("ExportedClass");
    27.  
    28.         var assetBundle = AssetBundle.LoadFromFile(AssetBundleFile);
    29.         var asset = assetBundle.LoadAsset<GameObject>(assetBundle.GetAllAssetNames()[0]);
    30.  
    31.         Debug.LogFormat("Asset: {0}", asset);
    32.         var behaviour = asset.GetComponent(exportedBehaviour);
    33.         Debug.LogFormat("Behaviour: {0}", behaviour);
    34.         var field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
    35.         Debug.LogFormat("ExportedClass: {0}", field);
    36.         field = exportedBehaviour.GetField("BehaviourText").GetValue(behaviour);
    37.         Debug.LogFormat("BehaviourText: {0}", field);
    38.  
    39.         Debug.Log("Assign field");
    40.         var newValue = Activator.CreateInstance(exportedClass);
    41.         exportedClass.GetField("ClassText").SetValue(newValue, "VALUE!");
    42.         exportedBehaviour.GetField("ExportedClass").SetValue(behaviour, newValue);
    43.  
    44.         field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
    45.         Debug.LogFormat("ExportedClass post-set: {0}", field);
    46.  
    47.         var copy = Instantiate(asset);
    48.         behaviour = asset.GetComponent(exportedBehaviour);
    49.         Debug.LogFormat("New Behaviour: {0}", behaviour);
    50.         field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
    51.         Debug.LogFormat("New ExportedClass: {0}", field);
    52.         field = exportedBehaviour.GetField("BehaviourText").GetValue(behaviour);
    53.         Debug.LogFormat("New BehaviourText: {0}", field);
    54.     }
    55. }
    The code does the following things:
    1. Loads the assembly.
    2. Loads the asset bundle.
    3. Fetches the only asset inside the bundle and prints it (which works)
    4. Gets the ExportedBehaviour component on said asset and prints it (which works)
    5. Gets the ExportedClass field of the component (which is null)
    6. Gets the BehaviourText field of the component (which works)
    7. Sets the ExportedClass field on the component.
    8. Instantiates the asset to create a copy.
    9. Gets the ExportedBehaviour on the copy (which works)
    10. Gets the ExportedClass on the copy (which is null)
    11. Gets the BehaviourText on the copy (which works)
    Now the thing that bugs me is the following: If the .dll was present when the player was built, the field is properly deserialized in the standalone player. If the player was built without the .dll, even though the ExportedBehaviour can be loaded, and is properly deserialized, the exported class itself is not. For the test, it does not seem to matter whether the object is instantiated into the world somewhere or simply inspected from the asset bundle.

    Loading the assembly at runtime in the editor does not seem to work, but this is not a serious issue as developers can be expected to add the DLL themselves. When the DLL is present, the Asset Bundle Browser can also be used to inspect the asset bundle in any editor instance and ExportedClass isn't null.

    This is kind of annoying for me, as I would like to keep some of my data organized as custom classes instead of MonoBehaviours, especially if there are multiple of the same time. Because the assets stored in the asset bundle may contain UnityEvents that are pointing towards the object itself, I'm not sure if ScriptableObjects would be a way to mitigate this issue, especially since UnityEvents with parameters have to be implemented as custom classes anyway.

    For me, it seems like the serializer has some kind of cache/list of serializable fields that is either generated during the player build or at the very start of the game. Since the assembly I am using is loaded later, the types are not included in this list, and therefore the serializer simply skips over them. This theory is somewhat solidified by the fact that if I add an empty DLL in the build process, then replace it with the full DLL afterwards, everything works as expected again. However, this means I can only support a finite, pre-defined number of DLLs, which is not quite what I am looking for.

    Is there any way to load additional code at runtime in a way that allows serializable-classes to be deserialized? I've attached a zip containing an example bundle, library that was used to create it, as well as the TestBehaviour as .cs file to reproduce the issue at hand. Maybe I am just doing something terribly wrong.

    Thanks in advance.
     

    Attached Files:

  2. doctorpangloss

    doctorpangloss

    Joined:
    Feb 20, 2013
    Posts:
    233
    Did you try exporting an asset bundle and loading that instead?

    Asset bundles!
     
  3. Baawk

    Baawk

    Joined:
    Nov 15, 2017
    Posts:
    3
    Hi,

    that's exactly what I was doing. In the zip in the first post you'll find such an asset bundle. The problem is that upon loading said asset bundle, Serializable-classes are null. MonoBehaviours and Unity-components load fine, but Serializables just stay null.
     
    MNNoxMortem likes this.
  4. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    394
  5. harrysjoerd

    harrysjoerd

    Joined:
    Jan 30, 2017
    Posts:
    5
    @Baawk Wow, finally someone with the EXACT same problem as I have. You did some better research than I did and I understand the problem a bit better now.

    Did you find a way to make this work yet?

    -Pascal
     
  6. Baawk

    Baawk

    Joined:
    Nov 15, 2017
    Posts:
    3
    Hey @harrysjoerd,

    I'm afraid I haven't. There are two things that I could imagine working, but haven't tried out so far:

    • Editing the globalgameassets file. I'm not too knowledgeable about Unity's player, but it seems like this file contains all the meta-information the player needs - including a list of DLLs. If the file format were known or reverse engineered, it might be possible to inject your own DLLs this way. This is still a bad solution, as it means there's a static kind of loading going on.
    • Using something like doorstop to see if loading the DLLs before/with Assembly-CSharp.dll and friends would help. If it is indeed a timing problem (i.e. serializer goes through all types at time X, then marks them as serializable), then loading the assemblies at the time the normal assemblies are loaded could help (assuming that the cache, if it exists, is built after all DLLs in the manifest have been loaded). This is a slightly better approach, as it would allow an dynamic loading process (i.e. as many DLLs as you wish, with whatever rule you wish to enforce), but still would not allow to read DLLs at runtime, so it's not a good solution.
    I'm not sure if this qualifies as a bug that could be filed with Unity, or whether this is expected behaviour. So far, the only real workaround that I have found is, sadly, hard-patching the Assembly-CSharp.dll using Mono.Cecil. I'm merging my serializable types into it, and in the editor just use them as normal scripts. This works, but of course needs to be redone with every player update, and needs to be done before the player runs, so it's not a viable solution either.
     
  7. harrysjoerd

    harrysjoerd

    Joined:
    Jan 30, 2017
    Posts:
    5
    Thanks for your elaborate answer. Man! This is a nuisance. None of these options work well with the desired modularity approach we're aiming at. I filed a bug report with Unity. I'll let you know when I hear something useful.
     
  8. slumtrimpet

    slumtrimpet

    Joined:
    Mar 18, 2014
    Posts:
    336
  9. mark_gr

    mark_gr

    Joined:
    Jan 16, 2015
    Posts:
    19
    I have a similar problem to this(if not the exact same problem)
    However the loading works 95 times out of 100.
    The other 5 times I get an error about miss serialization.

    I have managed to make some custom workarounds for my project but there are times where that does not work.
    Its a little hard to pinpoint the specifics, but essentially
    1.Try to load an asset with the script from bundle
    2. Occasionally the load will fail due to serialization error, this seems to cause all of the assets in the same bundle to not load properly (script or no script)
    3. We can detect the load error at runtime because the script will have null values attached to it
    4. Unload the assetbundle and reload it at runtime, which usually fixes the issue. However during this reload nothing else can be used from the same assetbundle because all the data gets unloaded.
    eg, if we have more than 2 assets from the same bundle loaded at the same time and one of them fails, most of the time we cant recover.

    Either way its not a great workaround. Just a tiny bandaid.

    Hopefully there is some kind of fix coming.

    Edit: Sorry, my scripts are not in a DLL, I have a serialized class defined in a monobehavior. Everything else is the same.
     
    Last edited: Jul 29, 2019
  10. slumtrimpet

    slumtrimpet

    Joined:
    Mar 18, 2014
    Posts:
    336
  11. doctorpangloss

    doctorpangloss

    Joined:
    Feb 20, 2013
    Posts:
    233
    Consider using JSONUtility.