Are C# Expression Trees (or ILGenerator) allowed on iOS

So after switching to the 2017.1 experimental .NET 4.6 profile, I could finally leverage the full power of C# expression trees in my serialization framework. However, a good question arises: are C# expression trees (and thus ILGenerator) considered dynamic code generation by iOS, and thus not really allowed for iOS publishing?

Anyone has any insights into this?

Cheers!

 

as far as I know, everything that does dynamic code generation will either not compile or throw an exception on IL2CPP. you can test it on a build and see.

I don't know if there is something that IL2CPP allows but is forbidden by Apple policies (short of writing your own bytecode interpreter)

 

You can parse, compile and execute expression trees on iOS with this asset using .CompileAot() extension on System.Linq.Expression.Expression<T> instance.

 

Actually Unity team could implement IL interpreter for AOT environment. It would be in 'gray' area of Apple policy.

 

Hey, link just goes to assert store front page for me...?

 

It is strange. I just copied it from browser address bar.
http://u3d.as/oCH <- link in old Asset Store.

 

How about things under System.Linq.Expressions only? I might be a little misleading, in that I have no explicit usage of anything under the System.Reflection.Emit namespace, I exclusively use the Expression Trees (System.Linq.Expressions); but I somehow got the impression that under the hood, Expression Trees actually generates IL code during the "Compile()" phase?

So just to make the question exactly lean and mean: are C# Expression Trees (System.Linq.Expressions) allowed for iOS?

 

Hmm, I've actually got an idea: since my usage case doesn't really involve "dynamic code generation", even though the Expression Tree is used (with reflection) to inspect existing types and generate "logic" to process the data with the known types, it might actually be possible for me to take the compiled expression trees and create an assembly in a separate process (sort of a AOT process), and at the publishing or runtime, the generated assembly is shipped (or loaded at runtime), so there would be no runtime code generation whatsoever.

The bits I'm looking now is: LambdaExpression.CompileToMethod based on this SO post: https://stackoverflow.com/questions...the-main-entry-point-to-a-new-executable-disk

Please share your insights on this. Thanks again!

 

that can be done in the editor though. I have something similar for protobuf.net: I generate the DLL from my data classes via an editor MenuItem and put it into the project.

 

Exactly as M_R suggested, the assembly generation would be done in the editor (a separate phase), and the generated assembly would be put in the Unity project, and gets referenced from there.

 

A follow up question: does IL2CPP automatically process an (Unity compatible) assembly that resides in the current Unity project (e.g. under "Assets/DLL" folder, so it looks like: "Assets/DLL/MyAssembly.dll")? Does IL2CPP work against "Assembly.Load(AssemblyName("MyAssembly"))"?

Note that I'm aware that Assembly.LoadFrom(assemblyFilePath) would not work (see: https://forum.unity3d.com/threads/il2cpp-loading-c-assembly-at-runtime.417553/), but Assembly.Load(AssemblyName) seems only needing to find a "reference", and I'm assuming it should work with IL2CPP (or with an IL2CPP processed assembly)?

 

Unfortunately, Assembly.Load(AssemblyName) does not work against IL2CPP ecosystem.

Based on my test (IL2CPP with the Xcode iOS simulator), if an assembly is referenced explicitly (such as invoking a function as defined inside the assembly in question), IL2CPP would pick the assembly up and does all the necessary transformation, and the generated C++ code is guaranteed to be referenced correctly, while if there are no explicit references to the assembly in question, the entire assembly would be stripped, and any runtime binding to the assembly would fail (even an in memory binding would fail, if implemented at all) - similar to the C++ linker that unreferenced symbols are stripped.

 

https://docs.unity3d.com/Manual/IL2CPP-BytecodeStripping.html

 

I have heard that in C# recent version `Expression.Compile` has 2 implementation. It would fallback to use reflection interpreter in any platform that not support dynamic codegen

https://stackoverflow.com/a/47476886/1513380

If it's true then it would be useful in some case that not really need performance. Such as let user add math script to control object movement

Would be anyone pleased to help test that is it really work on iOS and WebGL?

 

According to my investigation the reflection interpreter is natively supported in .NET framework since before 4.0. If unity was brought dotnet framework as is I think it would also contain the interpreter in the 4.6 engine

Problem is I have no iOS device to test on my own. And the API for ensure interpreter usage is just added in 4.7.1

 

@JoshPeterson Doesn't dotnet 4.6 experimental use corefx?

The implementation of Expression.Compile which also include reflection interpreter is in the corefx

Could you please let me have information about what mono people trying to do about this case?

 

I still suspect that mono branch was cloning implementation from corefx

https://github.com/mono/mono/tree/9...d63c8db3034228fe/mcs/class/System.Core/corefx

Seem like it was introduced the interpreter last year

 

@JoshPeterson I was having conversation with mono people in gitter and this is what they answer me

 

@JoshPeterson Could we see a sample of code that would run into AoT limitation?

 

Thank you. But this is really sad because math operation is what I really need for this feature

 

@JoshPeterson if I understood correctly, an expression that returns a bool wouldn't work, would it?

I'm "baking" a MethodInfo.Invoke (a function that returns a bool) with an Expression.Call to avoid allocs.

In this scenario I'm getting:
ExecutionEngineException: Attempting to call method 'System.Linq.Expressions.Interpreter.LightLambda::MakeRun0<System.Boolean>' for which no ahead of time (AOT) code was generated.

 

Thank you for your answer.

I read in your previous posts that in the future IL2CPP will have the capability of handling value types. Is it possible to have an estimate of when it will be released?

 

Any update on this @JoshPeterson ? eta?

 

@JoshPeterson
Can we track this feature somewhere?

 

Is there any kind of work around to this such as perhaps boxing the values?

Essentially I'm automatically supporting operators, but primitives don't have defined operators so I was using this to handle them (whereas non primitives I can just execute the operator method)

 

Hi, resuming this.

Where can I read more about System.Linq.Expressions.Interpreter.LightLambda? Am I supposed to use it directly in code? Also @JoshPeterson maybe you can tell me what's going on here: https://pastebin.com/raw/vJYMdzSK (this is a bit OT as it's NOT IL2CPP, it happens with mono)

 

I just realised that the exception does say System.Linq.Expressions.Interpreter.LightLambda, so LightLamba must be an internal type. Are you able to tell me if the code in the pastebin would work with IL2CPP? It dynamically generates new IL, so it shouldn't work in an AOT scenario, but then I am not sure what I read in this thread (talking about: https://forum.unity.com/threads/are...generator-allowed-on-ios.489498/#post-3665089)

 

thanks I will just for curiosity. For IL2CPP I actually fall back to a simple cast, as it seems it doesn't incur into boxing on that platform. The real issue remains the .net standard 2.0 crash, I will open a bug in case.

 

I'm getting a NullReferenceException when using this with high stripping enabled

Spoiler: Error

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object
2.   at System.Linq.Expressions.Interpreter.LightLambda.MakeRunDelegateCtor (System.Type delegateType) [0x00190] in <a52579e0afd14d309e8c16ce7ae6c478>:0
3.   at System.Linq.Expressions.Interpreter.LightLambda.GetRunDelegateCtor (System.Type delegateType) [0x00023] in <a52579e0afd14d309e8c16ce7ae6c478>:0
4.   at System.Linq.Expressions.Interpreter.LightLambda.MakeDelegate (System.Type delegateType) [0x00000] in <a52579e0afd14d309e8c16ce7ae6c478>:0
5.   at System.Linq.Expressions.Interpreter.LightDelegateCreator.CreateDelegate (System.Runtime.CompilerServices.IStrongBox[] closure) [0x00012] in <a52579e0afd14d309e8c16ce7ae6c478>:0
6.   at System.Linq.Expressions.Interpreter.LightDelegateCreator.CreateDelegate () [0x00000] in <a52579e0afd14d309e8c16ce7ae6c478>:0
7.   at System.Linq.Expressions.LambdaExpression.Compile (System.Boolean preferInterpretation) [0x0000b] in <a52579e0afd14d309e8c16ce7ae6c478>:0
8.   at System.Linq.Expressions.LambdaExpression.Compile () [0x00000] in <a52579e0afd14d309e8c16ce7ae6c478>:0

Spoiler: Code

Code (CSharp):

1. ParameterExpression leftParam = Expression.Parameter(type, "left");
2.             ParameterExpression rightParam = Expression.Parameter(type, "right");
3.             BinaryExpression body;
4.  
5.             try
6.             {
7.                 body = PrimitiveExpressionGenerator(leftParam, rightParam);
8.             }
9.             catch (InvalidOperationException)
10.             {
11.                 return null;
12.             }
13.  
14.             Delegate expr = Expression.Lambda(body, true, leftParam, rightParam).Compile();
15.  
16.             return new DynamicBinaryOperator(expr, type, type, type);

Any ideas what I need to preserve with the link.xml to stop this from happening @JoshPeterson? It's being used to add together primitive values

 

Not engine code stripping but the managed stripping option
It works in Editor, Disabled
But throws in Low, Medium, High
Tested on 2019.3.0f4 (if you believe it is a beta bug then I can move to there and repro it)

The code I included in my spoiler is the c# code that uses the expression tree and it works without the stripping on, or would you like something else?

EDIT: wanted to add this is the identical error that I would get on IL2CPP (without my own error handling)

 

That did the trick, thanks!

 

Hi there, I'm running into this AOT issue too with Expression trees.
Could the link.xml tell the compiler to generate the use cases ?
If not, is there any other way ? Another file or a console argument ^^'

We solved this issue in Newtonsoft Json by using a branched version allowing us to generate use cases (AotHelper).
But we can't here as we have no access in Expressions.Interpreter.LightLambda ~~