Loading and Unloading external DLL with Dots/Systems on it

TLDR: I am attempting to make a ingame mod loader like Rimworld. This is how I am attempting to do and my current issue with it.


I am getting started in making a mod manager for my game, and that includes new ECS Systems.
Currently I am managing to load a DLL and accessing it's classes through Reflection like this:

Code (CSharp):

1. //This runs before World Initialization
2. foreach (string mod in mods) {
3.     string[] assemblies = System.IO.Directory.GetFiles(mod, "*.dll", SearchOption.AllDirectories);
4.     try {
5.         foreach (string dll in assemblies) {
6.             //byte[] buffer = File.ReadAllBytes( dll );
7.  
8.             Assembly assm = Assembly.LoadFile(dll); //ModsDomain.Load( buffer );
9.  
10.             Type[] types = assm.GetTypes();
11.             foreach (Type type in types) {
12.                 Debug.Log(type.FullName);
13.             }
14.         }
15.     } catch (Exception e) {
16.         Debug.LogException(e);
17.     }
18. }

Code (CSharp):

1. //World Initialization
2.                 if (Application.isPlaying && ReloadWorld) {
3.                     World.DisposeAllWorlds();
4.                     world = DefaultWorldInitialization.Initialize("Default World", false);
5.                     DisableUnusedSystems(world);
6.                 }

OBS: I still haven't even tested if this is even capable of unloading said mod later on, probably it can't

Later on another part of my code I try to dynamically call one of it's classes:

Code (CSharp):

1.         Type[] SystemTypes =
2.             Utils.GetAllClassesExtending<ConfigSystem>("Unity", "Microsoft", "System", "Mono",
3.                                                        "UMotion"); //excludes type search in assemblies containing these strings
4.         Type[] Defaults = new Type[] { };
5.         World World = World.DefaultGameObjectInjectionWorld;
6.         SystemTypes = Defaults.Concat(SystemTypes.Where(T => ((ConfigSystem) World.GetExistingSystem(T))<<Null pointer here.Active).Where(T => !Defaults.Contains(T)))
7.                               .ToArray();

What I don't get is that my method of getting all types extending is the same as Worlds do to instantiate the world.

My Method:

Code (CSharp):

1. public static Type[] GetAllClassesExtending(Type t, params string[] exclude) {
2.         List<Type> types = new List<Type>();
3.         foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies()) {
4.             if (exclude.Any((E) => assembly.FullName.Contains(E))) {
5.                 continue;
6.             }
7. (...)

TypeManager method:

Code (CSharp):

1. #if !UNITY_DOTSRUNTIME
2.         internal static IEnumerable<Type> GetTypesDerivedFrom(Type type)
3.         {
4. #if UNITY_EDITOR
5.             return UnityEditor.TypeCache.GetTypesDerivedFrom(type);
6. #else
7.  
8.             var types = new List<Type>();
9.             foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
10.             {
11.                 if (!TypeManager.IsAssemblyReferencingEntities(assembly))
12.                     continue;
13. (...)

The World do not automatically instantiate loaded assembly systems neither on editor or runtime.
Could it be that assembly does not reference Entities? This is the only class on it:

Code (CSharp):

1. namespace TestMod {
2.     public class NewTestConfigScreen : ConfigSystem {
3.         public override VisualTreeAsset GetRootVisualAsset() {
4.             return Resources.Load<VisualTreeAsset>("AudioConfigSceneUXML");
5.         }
6.         public override string GetButtonName() {
7.             return "MyTestConfig";
8.         }
9.         public override void OnClose() { }
10.         public override void OnOpen(VisualElement root) {
11.             bool _false = false;
12.             AddButtonField(root, DatabaseLoader.Translate("UIHideIcons"), (T) => _false = T, _false, (true, "UIYes"), (false, "UINo"));
13.         }
14.     }
15. }

ConfigSystem extends SystemBase:

public abstract class ConfigSystem : SystemBase {

OBS: Another issue is that while TypeManager is loaded AFTER the mods assemblies are loaded, I don't think it can be initialized again if the mods change, even if the worlds are recreated.

 

If your mod is a dll then you can't unload it during runtime (this is a limitation of C# in general) but you can load and unload an AssetBundle. The only way to unload a dll is to unload the AppDomain it was loaded in. This gives you 2 options:
- load the dll in the main AppDomain, then reload the AppDomain (idk how to do this in unity without it breaking)
- load the dll in another AppDomain and have it communicate to the main AppDomain via MarshalByObjectRef or the SetData, GetData AppDomain methods.

For high performance, both methods sucks and I don't recommend them unless you don't mind the performance hit and annoying C# AppDomain stuff. here's a like of my suffering if you want to read:
https://forum.unity.com/posts/7793916/

As far as your world issue, ECS Worlds are.....interesting..... Here's some info:
https://gametorrahod.com/world-system-groups-update-order-and-the-player-loop/

but just to clarify

1. your world is not loading from the loaded dll?
2. you can't get any systems loaded from a dll to run in a world?

 

The systems added by the dll are not being automatically added by the world, even though the DLL loads before TypeManager/World initiates. Maybe its because they are in an assembly that references an assembly that references Entities, but they don't reference Entities themselves.
I could probably manually add them with GetOrCreateSystem, but it feels hacky.

About loading/unloading, I guess I can Application.Quit after applying the a mod load order, but I would prefer it it could just reload the whole game without closing the game window.

 

this is correct

you see in order to add types IComponentData, ISharedComponentData (other ecs data) you need to call TypeManager.AddNewComponentTypes(). my code specifically does this

Code (CSharp):

1.  
2.                 TypeManager.AddNewComponentTypes(Mods[modName].InstancedObjects[key].GetType());

i used an instanced object and grab the type from that. HOWEVER, the TypeManager isn't accessible during a build since it's in an #If UNITY_EDITOR block. I modified the entities package to enable this in a build and it worked fine though.

If you want a DOTSSystem to wun in a world you have to add it to the world. This will initalize it. to get it running you have to add it to an certain update group within the world

i use:

Code (CSharp):

1.   World.AddSystem(system.GetType().IsSystemBase() ? (SystemBase)system : system.GetType().IsComponentSystem() ? (ComponentSystem)system : (ComponentSystemBase)system);
2.                      

Code (CSharp):

1.    switch (systemGroup)
2.                         {
3.                             case SystemGroup.InitializationSystemGroup:
4.                                 var ISG = World.GetExistingSystem<InitializationSystemGroup>();
5.                                 ISG.AddSystemToUpdateList(system.GetType().IsSystemBase() ? (SystemBase)system : system.GetType().IsComponentSystem() ? (ComponentSystem)system : (ComponentSystemBase)system);
6.                                 ISG.SortSystems();
7. ...

World info stuff: https://gametorrahod.com/world-system-groups-update-order-and-the-player-loop/

 

After adding the system to the world it runs as if you made the system natively in the project. I currently am able to now only load entire ECS Systems and data from dll but AssetBundles and Bursted Libraries. Making a tutorial is on my todo list though...

 

How strange, back when I was compiling new code during runtime the type manager and world would recognize them even through they didn't exist at build time so long I did all the compilation before TypeManager.Initialize().
I assumed loading DLLs would work exactly the same. At least manually registering those types is not that hard.

 

I managed to get it work (So far no way to unload mods and didn't test adding IComponentDatas)
This is how I added systems and added them to upload queues:

Code (CSharp):

1.  
2. //Absolute very first thing the game runs
3.     public static void LoadModDLL() {
4.         AllMods = OrderMods(Directory.GetDirectories(ModsPath));
5.  
6.         List<Type> Systems = new List<Type>();
7.         foreach (string mod in AllMods) {
8.             string[] assemblies = System.IO.Directory.GetFiles(mod, "*.dll", SearchOption.AllDirectories);
9.             try {
10.                 foreach (string dll in assemblies) {
11.                     Assembly assm = Assembly.LoadFile(dll);
12.                     Debug.Log(dll);
13.  
14.                     Type[] types = assm.GetTypes();
15.                     foreach (Type type in types) {
16.                         if (typeof(ComponentSystemBase).IsAssignableFrom(type)) {
17.                             Systems.Add(type);
18.                         }
19.                         Debug.Log(type.FullName);
20.                     }
21.                 }
22.             } catch (Exception e) {
23.                 Debug.LogException(e);
24.             }
25.         }
26.         SystemsToLoad = Systems.ToArray();
27.     }
28.  
29. //On the class that reloads the world
30. if (Application.isPlaying && ReloadWorld) {
31.     World.DisposeAllWorlds();
32.     world = DefaultWorldInitialization.Initialize("Default World", false);
33.     DisableUnusedSystems(world);
34.     RegisterModSystems(world);
35. }
36.     private static void RegisterModSystems(World World) {
37.         foreach (Type Type in SystemsToLoad) {
38.             var System = World.GetOrCreateSystem(Type);
39.             Debug.Log($"Got Or Created: {Type}, {System.Enabled}");
40.         }
41.         DefaultWorldInitialization.AddSystemsToRootLevelSystemGroups(World, SystemsToLoad);
42.     }
43.