[Released] Roslyn C# - Runtime C# compiler

How do you get MethodInfo from a Roslyn ScriptType? I am running into a rewrite issue where the prior Compiler was looking for Type[] and then it would do foreach and

MethodInfo[] methods = type.GetMethods()

Edit: I'm mildly dumb, ScriptType.SystemType.GetMethods()..etc

 

Glad to hear it is working ok now. Sounds like something along that process triggered a refresh and allowed things to start working as expected. Only thing I would say is that I thought Unity required Net 4.6 and that should be the target for your class library, but that could have changed by now and I might have missed it. Anyway as long as it is working that is the main thing. Let me know if there is anything else.

 

Okay so in my project where I was rewriting the old Compile Routines to use Roslyn, there were 4 cases.

1. External scripts to Assembly, loaded into memory.
   
2. Files ending in ".cs" detected and made into an Assembly, loaded into memory.
   
3. Internal script fields collated and compiled into an Assembly, loaded into memory.
4. Files ending in ".dll" are done differently and it returns a string of the Assembly's Filename.

For #1 and #3, the routine is about like this, and I get a compiled Assembly and my application is now synced to all of its MethodInfo[] and its T<type>

Code (CSharp):

1.                                     //3.0 Roslyn Compiler External Code Assembly Routine here
2.                                     scriptDomain.RoslynCompilerService.GenerateInMemory = true;
3.                                     ScriptAssembly assemblyRoslyn = scriptDomain.CompileAndLoadSource(externalCode, ScriptSecurityMode.UseSettings, null);
4.                                     ScriptType[] roslynTypes = assemblyRoslyn.FindAllTypes();
5.                                     if (roslynTypes != null) {
6.                                         foreach (ScriptType scriptType in roslynTypes) {
7.                                             MethodInfo[] roslynMethods = scriptType.SystemType.GetMethods();
8.                                             if (roslynMethods != null) {
9.                                                 Object roslynAssemblyObject = null;
10.                                                 foreach (MethodInfo roslynMethod in roslynMethods) {
11.                                                     if (roslynMethod.Name == "Init") {
12.                                                         if (roslynAssemblyObject == null) {
13.                                                             roslynAssemblyObject = Activator.CreateInstance(scriptType.SystemType);
14.                                                         }
15.                                                         Object[] roslynArgs = new Object[1];
16.  
17.                                                         roslynArgs[0] = _core;
18.  
19.                                                         roslynMethod.Invoke(roslynAssemblyObject, roslynArgs);
20.                                                     } else if (roslynMethod.IsPublic && (roslynMethod.GetParameters().Length == 0) && (roslynMethod.ReturnType.Name == "Boolean")) {
21.                                                         if (roslynAssemblyObject == null) {
22.                                                             roslynAssemblyObject = Activator.CreateInstance(scriptType.SystemType);
23.                                                         }
24.                                                         _scriptMethodList.Add(scriptType.Name + "." + roslynMethod.Name);
25.                                                     }
26.  
27.                                                     // 3.0 AUTO-ADD THE INSTANTIATED CLASS TYPE FOR EXECUTION OF METHODS LATER IF ANY VALID METHODS
28.                                                     if (roslynAssemblyObject != null) {
29.                                                         _core.ApiBrain().SetScriptClassType(scriptType.SystemType, null);
30.                                                         // UNITY MUST MANUALLY SET THE SCRIPTING CLASS TYPES VIA myCore.ApiBrain().SetScriptClassType(Type type);
31.                                                         try {
32.                                                             _classTypes.Add(scriptType.SystemType);
33.                                                             Object o = Activator.CreateInstance(scriptType.SystemType);
34.                                                             _classObjects.Add(o);
35.                                                             // IF AN INITIALIZATION METHOD...
36.                                                             Object[] roslynArgs = new Object[1];
37.                                                             roslynArgs[0] = _core;
38.                                                             scriptType.SystemType.InvokeMember("Init", BindingFlags.Default | BindingFlags.InvokeMethod, null, o, roslynArgs);
39.                                                         } catch {
40.                                                             //Intentionally left blank.
41.                                                         }
42.                                                     }
43.                                                 }
44.                                             }
45.                                         }
46.                                     }

So that's all fine and the Parser in VS2022 is happy with everything.

What I don't understand is #4, the old CSScript routine was detecting for a file ending in ".dll", and it wants to write it to disk and return a string with... I cannot tell if I JUST need the string filename or the full path to the assembly or a relative path.

As follows:

Code (CSharp):

1. //Old 2.0 Internal Compile Routine
2.                         CSScript.CompileCode(code, assemblyFile, false, referenceAssemblies);
3.                         //^ this function ends up calling for this to happen as a return string:
4.                         /*
5.                             public string Compile(string scriptFile, string assemblyFile, bool debugBuild)
6.                             {
7.                                 if (assemblyFile != null)
8.                                     options.forceOutputAssembly = assemblyFile;
9.                                 else
10.                                 {
11.                                     string cacheFile = Path.Combine(CSExecutor.GetCacheDirectory(scriptFile), Path.GetFileNameWithoutExtension(scriptFile) + ".csc");
12.                                     options.forceOutputAssembly = cacheFile;
13.                                 }
14.                                 if (debugBuild)
15.                                     options.DBG = true;
16.                                 return Compile(scriptFile);
17.                             }
18.                          */

A couple of things: I want to totally transcend this routine in favor of Roslyn, and it seems possible. I've included the necessary UnityEngine.CoreModule.dll in my ClassLibrary references such that I can create those AssemblyReferenceAssets, but all of the Roslyn compile methods only accept reference Assemblies as IMetadataReferenceProviders and I've got a string[] array that has been passed into this function.



Is there a way to translate referenceAssemblies from string[] into something Roslyn can take? How can I do that via script from an unknown length string array? And further, the rest of my application needs to get a return string with the location of the assembly - so I've set the GenerateInMemory flag to false for this particular situation although the others go to memory. Should I just return the assemblyRoslyn.SystemAssembly.Location?

I don't want to break my app and I don't think I'll have to, but I'm in over my head just a tad. I also am looking at using EmitCompilationObject but I'm unsure what to do after producing one.

Thanks so much for your help. This is my last hurdle integrating Roslyn as best I can figure. It set right in perfectly with my Activator routines for subscribing methods so that was great.

 

Researching further into that IMetadataReferenceProvider you have set up, it uses Microsoft.CodeAnalysis and a MetadataReference. I have a string name for an Assembly reference I need to pass that corresponds to a DLL file, and I'm in a pre-unity environment trying to provide that to one of the Roslyn Compile Functions. I can't add it in the project. Tried already. The DLL I am referencing won't exist before runtime anyway because this is the Core project before I build the Core DLL and take it over to our application. The Compile function is supposed to prepare a library compiled at runtime to backreference the Core.

Here's the MetadataReference parameter I'm trying to get to ultimately. There aren't any constructor fields for me to use when creating a new AssemblyReferenceAsset by script, and


I just need to pass one string but Roslyn wants the new Interface type in the overload, great in all other cases because every other case was null, but no good in this instance:

 

Sorry for the flood of questions as I get to the bottom of this but, returning to CreateCompilationObject:

How could I get a proper Syntax tree and options to send to this command? It had the string array references I will want to use as an input since the IMetadataReferenceProvider style can't help me set this up prior to deploy-runtime.



Suppose I ran CreateCompilationObject as follows:

//3.0 Roslyn Internal Compile Routine //NOT DONE

                        Microsoft.CodeAnalysis.Compilation roslynCompilationDLL = RoslynCSharpCompiler.CreateCompilationObject(assemblyFile, referenceAssemblies, RoslynCSharpCompiler.ParseSource(code), null);

                    

                        scriptDomain.LoadAssembly(/*... as byte[]? Not sure, can't seem to get fullpath from the compilation object*/);

How would I actually load the Compilation object, or locate it.. or is it not loaded yet? Should I turn it into a Byte array and send it to scriptDomain.LoadAssembly()?

 

Hi again,
You can use the 'RoslynCSharp.Compiler.AssemblyReference' class included with our asset as it makes it much easier to add references via code. It has a number of static methods that will return an IMetadataReferenceProvider implementation, but it sounds like you will want the following in this case:

Code (CSharp):

1. IMetadataReferenceProvider myReference = AssemblyReference.FromNameOrFile("my/assembly/path.dll");

There are also a number of similar helper methods for creating reference assemblies from other sources. Hope that helps you.

 

Once you have a Compilation object as returned by the CreateCompilationObject method, there is then an additional step before you can access the compiler output (The assembly image) which is the emit stage. In a similar style you will also find a static EmitCompilationObject that will allow you to complete the process and will actually generate the assembly if there are no errors. Hope that helps. Let me know if there is anything else.

 

Got the functions written inside the Core and built the new DLL
I get a namespace missing error re: My Own Core -- it does attempt the compile though. Thought I'd try doing the assembly reference asset from your Settings in the Studio project so...

Back over in my Studio Project, still have this bug when I try to add any assembly references, Unity crashes. Tried reimporting the package from scratch. Any edit/addition to this list automatically crashes unity. Not doing it during playback. I don't know if anyone else has this error but I'm reporting it for Unity 2021.3.2f1 and I have had it in two separate projects, one Unity-Managed, and the other only loosely maintained by Unity but mostly VS2022.

No matter ultimately for me, because I just supplied a custom reference to the same DLL name at compile time and got away with it. I think I'm up and running on the compiler side now. Huge thanks to you. Good wizardry.

 

What's the best way to subscribe to the output of the Compiler Success/Failure, and are there any flags or settings I can set for the stacktrace depth?

 

Got the compiler messages. Question about interaction between Hot reload settings and the allow concurrent compile setting.. if allow concurrent compile is set to false, will re-compiling replace the assembly automatically?

 

Hey, glad you were able to get it working in the end.
Not sure what is causing the crash. We have had some users in the past report a similar issue but we have never been able to reproduce it on our end so couldn't really do anything to fix it. I will check it out thouse on 2021.3.2 to see if I can have better luck replicating it.

Allow concurrent setting has no relation to hot reloading. It simply allows the compiler to use multiple threads while compiling the source code. Recompiling will always replace the assembly on disk as long as the compilation was successful and assuming that the output file name is identical. It will not however cause the assembly changes to have any effect if you try to reload it. The CLR will simply return the already loaded version in memory. To get around this, the default behaviour of Roslyn C# is that it will always give easy assembly a unique name while will always allow them to load as a unique instance even if there are no changes.

 

Can you give a video tutorial to demonstrate the use process of connecting to a project?

I try to integrate the plug-in into the project. First, I use a simple unity project single solution to test,

But only scripts that inherit monobehavior can take effect. No error is reported, but the modified code in the rider cannot take effect, and the code has been prompted to be reloaded.

 

Hi, Not 100% sure what you mean but it sounds like you might want to attach the debugger to some of your runtime compiled code? If so, this is not something we have been able to get working reliably and is quite difficult to setup in any case. We have only tested with Visual Studio though so you may have better luck with rider if you wanted to try that. In fact I believe there was a user who posted in this thread that may have been able to get it working, might be worth checking for that post for any tips.
Let me know if I have misunderstood, but if you mean attaching the debugger then it is not something we can officially support at the moment due to the unreliable results we have had and the difficult setup.

 

Sorry, I just want to modify the code in rider and see the effect at runtime. But after I tried, only scripts that inherit monobehavior can take effect. I have tried all the replies here. I modified the common c# script, and the unity console prompted that the code had been reloaded without any errors.But it has no effect.ScriptDomain domain = ScriptDomain. CreateDomain("Example"); I attached the script of creating scriptdomain to the scene object, and also tried to add assembly csharp dll in RoslynSettings.

 

Oh my mistake, so it sounds like you are talking about the hot reload feature instead. Unfortunately this is a know limitation of the hot reloading at the moment where Roslyn C# is only able to detect and apply hot reload changes to monobehaviour scripts. If a source file containing a non-monobehaviour is changed, then Roslyn C# cannot simply recompile that script only due to how the CLR works. Instead it would need to recompile all parent scripts that reference that particular type, and that has not been implemented yet due to the complexity involved. At the moment the only real workaround is to define non-monobehaviour types inside the same source file as a monobehaviour that will make use of it, that way changing the source file will also mean that the monobehaviour script needs to be hot reloaded. Not an ideal workaround I know, but that is the only way to do that for the moment unfortunately as the hot reload feature has some limitations.

 

Is there a bare minimum example anywhere? The maze example is simple but I am still not understanding what is a domain and proxy, and trying to copy out all the nescessary code from maze when i only want a script to print a custom Debug.Log call is taking a lot of work.

Is there any way I could get a very simplistic code example/scene that compiles only a single Debug.Log("Hello World") line and prints to the console??

 

Hi,
You can find some minimal example scripts in the following folder 'Assets/Roslyn C#/Examples/ExampleScripts/' which demonstrate how to achieve some common use cases with minimal code.

A script domain is always required and you can just think of it as a collection where all loaded script assemblies are stored. Generally you will create one at startup and only ever need one instance in your application unless you are an advanced user. A proxy is simply a wrapper around an instance of an pobject which contains some convenience features to make calling methods and accessing fields as simple as possible without knowing the type before hand.

Here is a quick example snippet showing probably the minimum required amout of code required:

Code (CSharp):

1. class Example : MonoBehaviour
2. {
3.     ScriptDomain domain;
4.  
5.     // Assuming that this string contains vsome valid C# source code
6.     public string cSharpSource = ...
7.  
8.     void Start()
9.     {
10.         // Create the script domain
11.          domain = ScriptDomain.CreateDomain("Example");
12.  
13.         // Compile the code
14.         ScriptAssembly asm = domain.CompileAndLoadSource(cSharpSource);
15.     }
16. }

Then if the compiled C# code contains a mono behaviour component, you might like to add that component to a partucular game object which can be done like so:

Code (CSharp):

1. // Create instance which will call 'AddComponent' if the type is a MonoBehaviour
2. asm.MainType.CreateInstance(gameObject);

I hope that helps you. Let me know if anything is unclear of if there is anything else.

 

Thanks for the explanation it was very helpful and I was able to create a minimalistic example and have started developing a API concept for my game

I am interested to know how to properly sandbox the complier to prevent usage of network sockets, reflection, evals or other potentially harmful functions? Is there any recommended approaches to using Roslyn compiler for user customizable multiplayer games?

Are there any examples or anything you would recommend for implementing security?

Thanks!!

 

Hi again,
Glad to hear that you we able to get things working.
Sandboxing is a difficult thing to achieve as we have not had any success in using C# AppDomains to segregate compiled code for security purposes. Instead the only suitable solution we have found is to provide a code security system that can check the compiled code before it is ever loaded for usage of potantielly harmful api's for example such as IO. That system is included with Roslyn C# and can be configured from the settings window, but you can certainly prevent access to risky api's or namespaces by blacklisting them. The code checks run automatically after every successful compile, so only setup via the settings window is required. We would recommend blacklisting the following as a good starting point, but that list can be expanded to suit your needs:

Assemblies:

• Mono.Cecil.dll

Namespaces:

• System.IO.*
• System.Reflection.*

Types

• System.AppDomain

Also make sure that 'Allow PInvoke' option is disabled to disallow external calls to native code.

Hope that helps. Let me know if you need any more details or if there is anything else.

 

Will dotnow-interpreter work on iOS ?

 

Hi, dotnow should work just fine on IOS, although I have not tested since I have no devices. I believe atleast one user has been able to get it working successfully though if I remember correctly, and I will be happy to help if there are any issues.

 

Good to hear. Are there any issues with it getting approved on Apple store?

 

No idea I am afraid and it not something I could advise about. dotnow is freely available to use however you like, and on whatever platform you like, but it will ultimately be up to you in the end to ensure that you are complying with Apple and other platform policies for example.

 

IL2CPP Update

There is a new setup guide available to get Roslyn C# working with dotnow interpreter to run the Roslyn C# demo game and other runtime compiled code on IL2CPP platforms such as WebGL. You can find the setup guide and other useful resources on the github wiki.

Thanks to Dustin_00 for creating this guide.

 

if a script loaded in dotnow references assemblies, are those assemblies compiled to IL2CPP or Mono?

 

Hi, it is a good question.
dotnow does not handle loading of any reference assemblies. They must be loaded before hand (usually by Unity) so they will exist either in mono land or IL2CPP depending upon your backend. You can use Assembly.Load on mono to load reference assemblies, but for IL2CPP they must be AOT compiled to native code already. dotnow will then call into those reference assemblies via an interop call when required.
Hope that clears things up. Let me know if there is anything else.

 

Thanks. My app with dotnow compiled successfully as a Windows il2cpp build. I simply switched over to Dedicated Server and tried to compile it as a Windows 10 dedicated server and got this message:

error CS1061: 'ScriptDomain' does not contain a definition for 'CompileAndLoadMainFileInterpreted' and no accessible extension method 'CompileAndLoadMainFileInterpreted' accepting a first argument of type 'ScriptDomain' could be found (are you missing a using directive or an assembly reference?)

 

It sounds like you are missing 'ROSLYNCSHARP' define in the player settings based on that error. I think if I remember correctly that Unity is using define symbols per platform, so it makes sense that switching build targets could bring up this issue. I think it should just be a case of re-adding the define to the player settings for the new build target and hopefully everything should be working again then. Let me know if that is not the case.

 

Whoops. You are correct - added the define.. it works

 

Roslyn C# 1.7.2 has been released on the asset store. This version includes the following changes and bug fixes:

• Fix an issue where the demo scene could fail to run in some newer Unity version due to types defined in multiple assemblies.
• Added legacy demo scene for older or LTS Unity versions that works as is.
• Fix asmdef 'RoslynCSharp.Examples' with references to undefined platforms 'LinuxStandaloneUniveral' and 'LinuxStandalone32'
• Fix System.ValueTuple compiler errors on import in newer unity versions.
• System.ValueTuple support for legacy or LTS Unity versions is now provided via compatibility unity package included in `/Plugins`.
• Update to 2019.4 LTS as minimum supported version

 

Roslyn C# version 1.7.3 has been released on the asset store. This version includes the following bug fixes:

• Fix an issue in some later Unity versions where the settings InputDialog could cause an exception to be thrown.
• Fix an issue in some later Unity versions where the settings InputDialog could cause a hard crash of the editor when using 'Accept' or 'Cancel' buttons.
• Minor bug fixes.

 

So from what I understand - WebGL can dynamically load pre-compiled scripts with dotnow, but Roslyn cannot be used in WebGL to re-compile an edited script because of an IL2CPP bug in Unity when building with Roslyn? or is it possible?

 

Hi,
Yes dotnow can indeed load and execute dynamically loaded code on WebGL and other IL2CPP platforms. There is also good news that Unity was able to fix that particular bug a couple of years ago so it is indeed possible to use Roslyn C# and dotnow combined on WebGL and other IL2CPP platforms, so you can load pure C# code and execute it in game. With that said, you should be aware that dotnow is in early beta and does have some limitations due to the AOT nature, but please do have a play around with it and see what cool things you can do.
Another user has kindly created a detailed setup guide to use Roslyn C# with dotnow on such platforms which you can find here. With that you will be able to run the include Roslyn C# maze crawler demo, the dotnow snake demo, and also any custom code of course (Within limitations).

Hope that helps you. Let me know if you have any more questions or if there is anything else.

 

Thanks. I tried that guide a couple days ago & the Maze Crawler demo froze in WebGL. I may have missed a step, will try again. Does the Unity version matter? I am trying it with version 2021.3.17

 

I think the freeze issue can show up if 'Allow Concurrent Compile' is enabled in the Roslyn C# settings, so make sure that is disabled and things should work then I believe. Unity version should not matter although I know that some 2021 versions have trouble building for WebGL, but if it is building fine there should be no problems with newer versions.

 

That fixed it. thanks!

 

Glad to hear it. I checked the guide again and noticed that some important steps were missing regarding Roslyn C# settings, so I have updated that to avoid confusion for other users.
Let me know if there is anything else.

 

Roslyn C# 1.7.4 has been released on the asset store. This version includes the following bug fixes:

• Fix a bug where loading an assembly using AssemblyName would not update the asembly load location.
• Fix an issue where the code security engine could fail to run on an assembly loaded by AssemblyName causing the assembly to be unloadable.
• An exception will now be thrown if an assembly load by AssemblyName that does not have any load location set is requested, since the code security engine will not be able to determine the assembly source location.
• Minor bug fixes.

 

Roslyn C# 1.7.5 has been released on the asset store. This version includes the following features and changes:

• Add embedded resources api's for reading resource data from ScriptAssembly.
• Update minimum supported version to 2021 LTS.
• Minor bug fixes.

 

Rolsyn C# 1.8.0 has been released on the asset store. In this version due to popular demand we have finally updated the Roslyn binaries to version 3.8 which now brings support for the latest language features up to C# 9.

 

The asset store description still says Android is not officially supported. To make things even more challenging, I have an Android 10.0 device with an x86 CPU (not ARM). Would that be a deal breaker for your plugin?

 

Hey, We still cannot commit to saying Android is supported since we don't have any test devices, but it is pretty clear from user feedback that it is working just fine if the correct steps are taken and that you are aware of the limitations.
I don't think architecture will make any difference as long as you can get Unity to build for that platform, since all code we implement is pure C# and will run anywhere.

First thing to be aware of is that there are special setup steps required for android due to being APK. You can find a brief summary here but essentially you need to use the assembly reference asset feature exclusivley plus toggle some options in the settings window.
Second thing is that building for Android will mean IL2CPP which means no loading of code supported by Unity. That problem has been solved too using dotnow interpreter (Open source), but there are some limtations to be aware of (Check the readme for details). There is a setup guide to get Roslyn C# and dotnow up and running together here.

After that you should be able to load and execute code on Android IL2CPP, but as always, let me know if you have any issues and I am happy to help.

 

Unity 2020.3.10 LTS + IL2CPP
Added dotnow and it works fine in Editor but on runtime getting this:

AssemblyReferenceException: One or more assembly references could not be resolved. See ReferenceExceptions for more information
at RoslynCSharp.Compiler.RoslynCSharpCompiler.CompileFromSyntaxTree (Microsoft.CodeAnalysis.SyntaxTree[] syntaxTrees, RoslynCSharp.Compiler.IMetadataReferenceProvider[] additionalAsemblyReferences) [0x00000] in <00000000000000000000000000000000>:0
at RoslynCSharp.ScriptDomain.CompileFromSources (System.String[] cSharpSources, RoslynCSharp.Compiler.IMetadataReferenceProvider[] additionalReferenceAssemblies) [0x00000] in <00000000000000000000000000000000>:0

 

Update forget to mention : it works fine with Mono in Editor and runtime

 

Hey, Are you using assembly reference assets as mentioned in this guide, and are all the references in `Tools -> Roslyn C# -> Settings` cleared as they can causing issues on Il2CPP platforms?
If that all looks fine then can you provide a list of assembly references you have added via assembly reference assets, and an example source file that would fail to compile and I can take a look.

 

I probably have not clear Roslyn references, is it because of that?

I am waiting now for the light build to finish and will try ASAP:

 

Yes that is probably the cause of the issue then. Let me know if not, but I think you should be fine after that.

 

When I delete references in Roslyn Resouces asset it spams with error messages in Editor...

 

Hey, sorry I may have not been clear enough with my instructions.
It is not the asset itself that needs to be deleted, but instead it is all the references added under the `Assembly References` list section. Those represet assemblies loaded by path which is not supported on IL2CPP platforms.
You will need to restore the settings asset from source control or the package manager and then delete only the references listed in the setting asset, and hopefully you should then be up and running.
Sorry for not being clear enough and let me know if that stil does not solve the problem.

 

I did delete references sure not "RoslynCSharpSettings.asset. "



Imported, made defines. Maybe will start again to see. I have also my scripts divided into several dll-s for
optimisation and faster compile times. Added references there as guided and it compiles fine.

I am using one of the examples which loads those 2 assemblies at runtime:



and it works fine in the editor. Without those references set it was complaining not know "Object" reference.