Loading assemblies with Serializable classes at runtime results in null deserializations

Hello,

I've ran into some trouble with the Unity 2018.2.14f1 serializer (using .NET 2.0/3.5 on W7x64) while trying to load an assembly at runtime in the player (Windows Standalone 64). The basic idea is to allow the loading of new assets (using asset bundles) and new data/logic that they might contain. For example, a prefab uses a new behaviour that was not present in the game at compilation build, and re-building the game entirely for each added asset seems infeasible, especially if (vetted) user-created content is allowed.

To do that, I'm first loading a normal assembly DLL using Assembly.LoadFile. After the assembly has been loaded, I load the asset bundle and fetch my resources as necessary. MonoBehaviours work flawlessly, but custom classes do not. Upon deserialization, the fields are always null.

Here's a simple example of such a scenario. This code would be placed inside an own DLL, that gets then loaded in the game:

Code (csharp):

1. [System.Serializable]
2. public class ExportedClass
3. {
4.    public string ClassText;
5. }
6.  
7. public class ExportedBehaviour : MonoBehaviour
8. {
9.    public ExportedClass ExportedClass;
10.    public string BehaviourText;
11. }

In another instance of Unity, add the dll as an asset (so it is available in the editor), create a new asset, add an ExportedBehaviour on it, and make sure that ExportedClass isn't null. BehaviourText can be set for reference. Bundle the asset up. For the consumer, attach the following behaviour on any GameObject and set the paths accordingly:

Code (csharp):

1. using System;
2. using System.Linq;
3. using System.Reflection;
4. using UnityEngine;
5.  
6. public class TestBehaviour : MonoBehaviour
7. {
8.     public string AssemblyFile;
9.     public string AssetBundleFile;
10.  
11.     private void Start()
12.     {
13.         Assembly asm = AppDomain.CurrentDomain.GetAssemblies().SingleOrDefault(s => s.GetType("ExportedBehaviour") != null);
14.  
15.         if (asm == null)
16.         {
17.             Debug.Log("ExportedBehaviour not found; load external essembly");
18.             asm = Assembly.LoadFile(AssemblyFile);
19.         }
20.         else
21.         {
22.             Debug.Log("ExportedBehaviour already present; skip loading the assembly.");
23.         }
24.  
25.         var exportedBehaviour = asm.GetType("ExportedBehaviour");
26.         var exportedClass = asm.GetType("ExportedClass");
27.  
28.         var assetBundle = AssetBundle.LoadFromFile(AssetBundleFile);
29.         var asset = assetBundle.LoadAsset<GameObject>(assetBundle.GetAllAssetNames()[0]);
30.  
31.         Debug.LogFormat("Asset: {0}", asset);
32.         var behaviour = asset.GetComponent(exportedBehaviour);
33.         Debug.LogFormat("Behaviour: {0}", behaviour);
34.         var field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
35.         Debug.LogFormat("ExportedClass: {0}", field);
36.         field = exportedBehaviour.GetField("BehaviourText").GetValue(behaviour);
37.         Debug.LogFormat("BehaviourText: {0}", field);
38.  
39.         Debug.Log("Assign field");
40.         var newValue = Activator.CreateInstance(exportedClass);
41.         exportedClass.GetField("ClassText").SetValue(newValue, "VALUE!");
42.         exportedBehaviour.GetField("ExportedClass").SetValue(behaviour, newValue);
43.  
44.         field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
45.         Debug.LogFormat("ExportedClass post-set: {0}", field);
46.  
47.         var copy = Instantiate(asset);
48.         behaviour = asset.GetComponent(exportedBehaviour);
49.         Debug.LogFormat("New Behaviour: {0}", behaviour);
50.         field = exportedBehaviour.GetField("ExportedClass").GetValue(behaviour);
51.         Debug.LogFormat("New ExportedClass: {0}", field);
52.         field = exportedBehaviour.GetField("BehaviourText").GetValue(behaviour);
53.         Debug.LogFormat("New BehaviourText: {0}", field);
54.     }
55. }

The code does the following things:

1. Loads the assembly.
2. Loads the asset bundle.
3. Fetches the only asset inside the bundle and prints it (which works)
   
4. Gets the ExportedBehaviour component on said asset and prints it (which works)
   
5. Gets the ExportedClass field of the component (which is null)
6. Gets the BehaviourText field of the component (which works)
7. Sets the ExportedClass field on the component.
8. Instantiates the asset to create a copy.
9. Gets the ExportedBehaviour on the copy (which works)
10. Gets the ExportedClass on the copy (which is null)
11. Gets the BehaviourText on the copy (which works)

Now the thing that bugs me is the following: If the .dll was present when the player was built, the field is properly deserialized in the standalone player. If the player was built without the .dll, even though the ExportedBehaviour can be loaded, and is properly deserialized, the exported class itself is not. For the test, it does not seem to matter whether the object is instantiated into the world somewhere or simply inspected from the asset bundle.

Loading the assembly at runtime in the editor does not seem to work, but this is not a serious issue as developers can be expected to add the DLL themselves. When the DLL is present, the Asset Bundle Browser can also be used to inspect the asset bundle in any editor instance and ExportedClass isn't null.

This is kind of annoying for me, as I would like to keep some of my data organized as custom classes instead of MonoBehaviours, especially if there are multiple of the same time. Because the assets stored in the asset bundle may contain UnityEvents that are pointing towards the object itself, I'm not sure if ScriptableObjects would be a way to mitigate this issue, especially since UnityEvents with parameters have to be implemented as custom classes anyway.

For me, it seems like the serializer has some kind of cache/list of serializable fields that is either generated during the player build or at the very start of the game. Since the assembly I am using is loaded later, the types are not included in this list, and therefore the serializer simply skips over them. This theory is somewhat solidified by the fact that if I add an empty DLL in the build process, then replace it with the full DLL afterwards, everything works as expected again. However, this means I can only support a finite, pre-defined number of DLLs, which is not quite what I am looking for.

Is there any way to load additional code at runtime in a way that allows serializable-classes to be deserialized? I've attached a zip containing an example bundle, library that was used to create it, as well as the TestBehaviour as .cs file to reproduce the issue at hand. Maybe I am just doing something terribly wrong.

Thanks in advance.

 

Did you try exporting an asset bundle and loading that instead?

Asset bundles!

 

Hi,

that's exactly what I was doing. In the zip in the first post you'll find such an asset bundle. The problem is that upon loading said asset bundle, Serializable-classes are null. MonoBehaviours and Unity-components load fine, but Serializables just stay null.

 

@Baawk sorry. Mea culpa.

 

@Baawk Wow, finally someone with the EXACT same problem as I have. You did some better research than I did and I understand the problem a bit better now.

Did you find a way to make this work yet?

-Pascal

 

Hey @harrysjoerd,

I'm afraid I haven't. There are two things that I could imagine working, but haven't tried out so far:

• Editing the globalgameassets file. I'm not too knowledgeable about Unity's player, but it seems like this file contains all the meta-information the player needs - including a list of DLLs. If the file format were known or reverse engineered, it might be possible to inject your own DLLs this way. This is still a bad solution, as it means there's a static kind of loading going on.
• Using something like doorstop to see if loading the DLLs before/with Assembly-CSharp.dll and friends would help. If it is indeed a timing problem (i.e. serializer goes through all types at time X, then marks them as serializable), then loading the assemblies at the time the normal assemblies are loaded could help (assuming that the cache, if it exists, is built after all DLLs in the manifest have been loaded). This is a slightly better approach, as it would allow an dynamic loading process (i.e. as many DLLs as you wish, with whatever rule you wish to enforce), but still would not allow to read DLLs at runtime, so it's not a good solution.

I'm not sure if this qualifies as a bug that could be filed with Unity, or whether this is expected behaviour. So far, the only real workaround that I have found is, sadly, hard-patching the Assembly-CSharp.dll using Mono.Cecil. I'm merging my serializable types into it, and in the editor just use them as normal scripts. This works, but of course needs to be redone with every player update, and needs to be done before the player runs, so it's not a viable solution either.

 

Thanks for your elaborate answer. Man! This is a nuisance. None of these options work well with the desired modularity approach we're aiming at. I filed a bug report with Unity. I'll let you know when I hear something useful.

 

The described here and here:
https://forum.unity.com/threads/serialization-lost-when-loading-asset-bundles-in-runtime.587431/
Seem to be describing the same situation.

Still no solution, but linking them together in case it provides any value.

 

I have a similar problem to this(if not the exact same problem)
However the loading works 95 times out of 100.
The other 5 times I get an error about miss serialization.

I have managed to make some custom workarounds for my project but there are times where that does not work.
Its a little hard to pinpoint the specifics, but essentially
1.Try to load an asset with the script from bundle
2. Occasionally the load will fail due to serialization error, this seems to cause all of the assets in the same bundle to not load properly (script or no script)
3. We can detect the load error at runtime because the script will have null values attached to it
4. Unload the assetbundle and reload it at runtime, which usually fixes the issue. However during this reload nothing else can be used from the same assetbundle because all the data gets unloaded.
eg, if we have more than 2 assets from the same bundle loaded at the same time and one of them fails, most of the time we cant recover.

Either way its not a great workaround. Just a tiny bandaid.

Hopefully there is some kind of fix coming.

Edit: Sorry, my scripts are not in a DLL, I have a serialized class defined in a monobehavior. Everything else is the same.

 

Chiming back in that I resolved this issue (for now) by implementing customer serializers:
https://docs.unity3d.com/Manual/script-Serialization-Custom.html
for the custom serializable class that was being dropped. It's an ugly hack but it keeps us going for now.

 

Consider using JSONUtility.

 

how to use it ,i have try ISerializationCallbackReceiver ,but it is not run auto to read the Serialization file i have save...help me plz..

 

Just wanted to add that I've been banging my head against this issue for 2 weeks now. The first time, I found a workaround (I just saved the data I wanted as a comma separated string then split/used it at runtime). This time I spent 20 minutes wondering why my asset was broken until I remembered this issue from 2 weeks ago. But this time, I need to reference a specific component on a specific child of the gameobject. Json isn't any help - that will just try to serialize the child component.

So what I have is a MonoBehavior, with a List<S***ThatWontSerialize> in it. That S***ThatWontSerialize IS marked as Serializable, and is compiled in a DLL. That DLL is used in the editor to add a MonoBehavior, which exists ONLY IN the DLL, which contains that List field.

But when the game is run, and the DLL is loaded, everything in the MonoBehavior works great. Except the List, which is empty.

The monumentally stupid workaround I'm about to do is instead of a List<RefusesToSerialize>, is to make RefusesToSerialie a MonoBehavior itself, and just stick one on the GameObject for each item the List would have. THAT will serialize. It's also a hack, and makes my skin crawl that I cannot do it the right/intelligent way.

Just necroing this old thread because it is literally the first search I've done on this issue the multiple times it's come up where I found someone describing the issue I'm having.

My GUESS is the Serializable attribute is compiled out when its made into DLL form. Or, rather, it's 'remembered' in a way that only the DLL is aware of. Which is why if the game is built WITH the dll in place, that 'knowledge' carries over? Maybe? I don't know. But the reason it's a dll is because it can't just be built with the game.

Ugh. If I ever find some solution I'll come back here and post it.

 

Serializable isn't "compiled out", but I strongly suspect that the game is keeping a list of assemblies/types that can be deserialized somewhere - probably somewhere close to the serializer. If the assembly wasn't present when the game/the list was built, the type isn't deserializable using Unity's normal deserialization mechanics/asset bundles.

There's no real workaround that I know of, except just not using Serializable and either do custom serialization, or using MonoBehaviour/ScriptableObjects, which seem to work.

 

I have test assembly, where serialization works normally.
After decompiling / recompiling using VS, some fields wont deserialize any more.
So looks like issue in some attributes in dll.

 

I'm revisiting this issue, and I can confirm that JsonUtility doesn't help at all. Same issue occurs.

 

Same issue, what i resorted was writing a JSON file and loading it separately

 

Has there been any news on this issue? Writing a hacky workaround isn't really an option for me.
Shocking to see that this has been around for this many years now.