Mono Upgrade Modern .NET Class Library Profiles

This is kind of a hard question to answer as we don't have all the data - The main reason why one would choose .NET Framework over Standard is because, as you say, one needs some of the more esoteric namespaces that Standard doesn't (yet) support. However you haven't provided what is to be cut or why, from the subset.

As a developer I would always try to aim for Standard, but I would hate to be put into a position where I really require some esoteric namespace only .NET Framework provides and it has been arbitrarily cut from the Unity subset.

On the other hand I would also be for anything that helps speed up how quickly Unity can keep up to date with the latest .NET Standard API and C# version - Being 6 months or more behind is unacceptable, for example.

 

Surely the onus is on the developer to write code that works on the platforms they intend to support? It is no different to setting up and maintaining a graphics and art asset budget, shader complexity, etc.

I don't agree that Unity should waste resources curating and maintaining an API subset for .NET Framework, specially if it could cause equal and oppose pain in devs who use the subset and find out it doesn't contain the APIs they need. I would say dump the subset option, allow only .NET Standard (And any micro subsets you think are a good idea) and the full .NET Framework. The full .NET Framework is a more 'use at your own risk' option. Maybe you could document major namespaces that most assuredly do not work on specific platforms to help guide careless developers?

However you do mention that supporting the full .NET Framework would cause some pains throughout the internal ecosytem and I don't know what those are, so maybe I'm misjudging the time wasted in doing so?

 

There is an option in that poll that is missing. Only NET STANDARD.

.NET Standard is what everyone should target because 2.0 version has a huge API that was added comparing to 1.6.

A big problem is that frameworks/libraries that are old and don't target yet (or never) .NET Standard. I know that in 2.0 there is a compatibility chance that Microsoft said that in nuget repository a 70% of packages should work but it's theorically.

In my case this is what is preventing on some non Unity projects to target NET Standard, the dependency on third party libraries.

Also for platform specific APIs there are nuget packages (.NET Standard compatible) like this:

https://www.nuget.org/packages/Microsoft.Win32.Registry/

I assume that this package would throw a PlatformNotSupportedException or something like that in non compatible platforms.

There are and will be more in the future.

Also I think that Unity should put it's resources in what is more promising and it's .NET Standard.

I want in the future to be sure that if I import some .NET Standard compatible package will work in any Unity platform (if it's not platform dependant like Windows registry). Know we drag a dll to Unity and it's really random to know that for sure.

Also I want to be using the latest version and not being like we were until some months ago, locked in a .NET 2.0/3.5 profile. If you put your efforts only on .NET Standard it should be easier.

And finally and really important, without bugs that only appear on specific platforms like we have been suffering with iOS and IL2CPP...

 

May I suggest maybe collecting some data on this - analysing the methods used - I'd be happy to run a tool to profile our codebase, which contains bits up to 7 years old (been using Unity for a while...).

I know in the 5.6 version we're currently on, we do have to use the "Full" setting in order to compile our standalone clients, and there was some good reasons we needed to do that (I think something esoteric to do with the XmlSerializer...) - if the version you're proposing to include would be less than that, it would be nice to at least get a long (i.e. 6-12 months) depreciation warning on the libraries that you are planning to drop, so we can do workarounds.

 

So. What does maintaining a .NET Framework subset do for anyone? Seems like its most important feature is build size. However I would posit that if that is important to you as a dev, do the work to move to Standard. If maximum platform support is most important, then Standard is the way to go. If old code is most important, Framework.

 

Why not do .NET Standard 2.0 (Default, officially supported) + .NET Standard Full (You're on your own)

Seems like you kind of get the best of both worlds. If you're doing something that the .NET Standard 2.0 profile doesn't support then the onus should be on you to ensure that what you're doing works on your target platforms?

Is there a list of what features the Subset might provide that .NET Standard 2.0 doesn't?

Is there any way to provide those features on top of .NET Standard 2.0 rather than swapping it out? Would that make your lives easier or harder?

 

Regarding .NET Framework (Full) ...

This is the kicker, and an overwhelming reason to not ship this option. All this does is give people ways to accidentally shoot themselves in the foot, cause confusion in the community, and waste everyone's time (yours and ours) trying to figure out what actually does work, especially once IL2CPP comes into play.

I'd much rather Unity spend energy shaping future .NET Standard versions to fill the needs missed by not adding the .NET Framework (Full) option. It's pretty clear that .NET Standard is the future of cross-platform .NET development from Microsoft's perspective, so to me it doesn't many sense to invest in anything else.

If there was a .NET Standard 2.0 only option in your poll, I would have voted for that. (Edit: But I'm guessing that route would make the upgrade/migration path for old codebases a much larger problem to solve?)

 

Absolutely agree. At least while packaging DLL into Unity plugin this must be the only option for library authors. (Probably even more, library authors should be encouraged to target as low version of .netstandard as possible; the one that is sufficiently enough.)

 

@joncham

If — theoretically — .NET Standard is the only option: is it possible for Unity in this case to target actual existent .NET Core implementation from Microsoft and community (at least on Windows, Mac, Linux), instead of custom Mono/Xamarin branch?

 

No matter what you do, we need #Defines for profiles. If there's something that's faster on Subset than Standard, I want to be able to write:

Code (csharp):

1. #if UNITY_PROFILE_SUBSET
2.     Fast_Code();
3. #else
4.     Standard_Compliant_Code();
5. #endif

That's especially important for libraries. I think all asset store packages should compile under Standard, but that doesn't mean that they shouldn't use the subset when able, if using the subset improves performance.


Also, are you looking into ways to better handle AOT platforms? There's things that's .NET Standard compliant that will crash with JIT-exceptions on AOT platforms. As an example, EqualityComparer<T>.Default is a part of .NET Standard 2.0, but needs to JIT-compile the implementation of the constructor.
This is a problem not only for us, but for Unity internally. You made several consoles crash whenever user code set a UI elements' .navigation due to using EqualityComparer<T>.Default somewhere internally whenever we set that.

Note that IL2CPP is an AOT platform, so by extension IL2CPP does not support the full .NET Standard, unless you manage to work around every single part of the .NET standard platform that might trigger the JIT compiler.

 

No, it does not. Netstandard is just a shape of API. Specific platform vendor can choose actual implementation.

In Remarks section in your link: «The Default property checks whether type T implements the System.IEquatable<T> interface and, if so, returns an EqualityComparer<T> that uses that implementation. Otherwise, it returns an EqualityComparer<T> that uses the overrides of System.Object.Equals and System.Object.GetHashCode provided by T.»

In case of AOT-platform vendor can skip first check and just return “an EqualityComparer<T> that uses the overrides of System.Object.Equals and System.Object.GetHashCode provided by T.”

For example in .NET Native:

“Unlike the full desktop, we make no attempt to use the IEquatable interface on T. Because we can't generate code at runtime, we derive no performance benefit from using the type-specific Equals(). We can't even perform the check for IEquatable<> at the time the type-specific constructor is created (due to the removable of Type.IsAssignableFrom). We would thus be incurring an interface cast check on each call to Equals() for no performance gain.”

https://github.com/dotnet/corefx/issues/8844

 

This is kind of what I was getting at. Why not .NET Standard 2.0 exclusively, and then additional checkboxes for other .NET framework/future .NET Standard features on a fine grain basis? Essentially .NET Framework (Subset) (The good parts), but just the parts that aren't also included in .NET Standard 2.0.

 

@joncham what about this?

https://forum.unity3d.com/threads/unity-adopting-net-standard-libraries-by-default.483441/

What about Unity adopting System.Numerics? We want Unity using more .NET API and not reinventing the wheel.

 

Is there any way to enforce this restriction? I'm sick of all these plugins that are tested on one platform, and completely ruin compilation or (suddenly) run time on the other.

 

the problem with subset is that whatever API you decide to cut, there may be a plugin / legacy code that uses it, so a fallback to "Full" is still needed.

.NET Standard can be enforced by Unity on asset store (rejecting non-complaining stuff), if you import a DLL yourself you are on your own (hopefully it will give a meaningful error at import time if the target is wrong)

 

Can you give an example of something that anyone is using that is NOT in the plans for .NET Standard 2.0? I can't really think of anything that anyone might want to use in Unity that's not in .NET Standard 2.0. From what I understand, the non-standard API contains things like WPF, ASP.NET, WinForms, etc., which have never really been usable in Unity anyway. I don't understand what would be in "Subset" or "Full" that is not in .NET Standard 2.0.

 

They talk about Reflection.Emit here: https://blogs.msdn.microsoft.com/dotnet/2016/09/26/introducing-net-standard/
and here: https://github.com/dotnet/corefx/issues/4491
Admittedly I've not done a ton of research, but from what I've seen on github it seems like most of Reflection.Emit is there and they're working on adding more, but that on AOT platforms, it will just throw a runtime error, which seems like a good solution to me. I'd assume they will have some sort of Reflection.Emit.DoesThisWorkOnMyPlatform() check that you could do at runtime (worst case, you could try an Emit call and if it throws an error, catch it and assume you're on an AOT platform). Maybe I'm misunderstanding?

 

The part I'm talking about is this in the first link:

---------------------
Let’s look at a few examples and how we plan on modelling them:

• Registry. The Windows registry is a self-contained component that will be provided as a separate NuGet package (e.g. Microsoft.Win32.Registry). You’ll be able to consume it from .NET Core, but it will only work on Windows. Calling registry APIs from any other OS will result in PlatformNotSupportedException. You’re expected to guard your calls appropriately or making sure your code will only ever run on Windows. We’re considering improving our tooling to help you with detecting these cases.
• AppDomain. The AppDomain type has many APIs that aren’t tied to creating app domains, such as getting the list of loaded assemblies or registering an unhandled exception handler. These APIs are heavily used throughout the .NET library ecosystem. For this case, we decided it’s much better to add this type to .NET Standard and let the few APIs that deal with app domain creation throw exceptions on platforms that don’t support that, such as .NET Core.
• Reflection Emit. Reflection emit is reasonably self-contained and thus we plan on following the model as Registry, above. There are other APIs that logically depend on being able to emit code, such as the expression tree’s Compile method or the ability to compile regexes. In some cases we’ll emulate their behavior (e.g. interpreting expression trees instead of compiling them) while in other cases we’ll throw (e.g. when compiling regexes).

-----------------------------

This makes me think they're planning on making a "separate NuGet package" eventually for Reflection.Emit that will compile on all platforms, but only work on JIT platforms and throw a "PlatformNotSupportedException" on AOT platforms. And that they want library developers to move away from Core and instead use Standard with a dependency on the new Emit package if they need it. So if it's just a matter of waiting until they finish that, then I'd say .NET Standard 2.0 should be the focus; support for Full or Subset just sounds like a band-aid until .NET Standard gets fully up to speed? Or have the .NET Standard's team's plans changed since they wrote that?

 

Wait I think I understand now, I'm dumb.
The Emit package for Windows for example wouldn't work from within Unity on Windows because Unity is running its own runtime. So you'd have to write your own libraries for everything not supported anyway, which is basically the same as just supporting a .NET Subset.

 

Well, my main feedback I guess would be that, to me, it seems like the .NET Standard team needs to figure this out more than Unity does. The main thing I'd want from Unity as far as .NET compatibility is to have decent support for 3rd party .NET libraries. And it seems like the future of .NET libraries is to use .NET Standard, but it doesn't look like most of them have a clear way of dealing with the JIT/AOT divide, but that's more the .NET Standard team's problem, not Unity's.

So, it sounds like you're all on the right track. I agree with making .NET Standard the recommended and primary focus, and having .NET Subset there for people who want the things it doesn't support yet, and that deprecating Full is probably fine since a lot of it won't really work anyway.

 

What magic are you referring to? NuGet is just a package management system, just like npm in the JavaScript world or apt in the Debian/Ubuntu world. You build up a dependency tree and download the package and it's dependencies.

 

I assumed he just meant that there would have to be Unity-specific NuGet packages for all the .NET extensions and they don't want to make those. Since the Unity runtime is itself a "platform" that also has to support other platforms within itself. They'd rather just have one single uber-pack bundled into the core Unity download, in other words, .NET Subset.

 

There is one use case regarding Reflection.Emit / ExpressionTree Compile (that I'm currently relying on under the .NET 4.6 experimental): I'm relying on the fact that I am able to do Reflection.Emit / Expression.CompileToMethod to generate an assembly within the Editor environment, and then use the generated assembly at runtime via IL2CPP (*without* any runtime Reflection.Emit or ExpressionTree Compile stuff). So it seems that the dotnet profile used in Editor *should* be "broader" than the profile used at runtime as far as the API surface area is concerned. Like the current .NET 4.6 experimental profile is mostly *fully* available in the editor, but runs in a "subset" fashion under the IL2CPP iOS profile - but the two profiles should be "binary compatible" (I know it's not exactly "binary compatible" when IL2CPP is involved, but we know what it means).

So to conclude: Editor should be using a fuller set of profile (ideally full .NET 4.7.1+ or netstandard 2.0 that supports Reflection.Emit or ExpressionTree.Compile); at runtime (specifically IL2CPP), a "binary compatible" subset of the profile (without the need to support Reflection.Emit / ExpressionTree, since I have pre-generated the assembly already) as used in Editor. Makes sense?

@JoshPeterson @joncham

 

Am I right to infer that .NET Subset would only include Framework features in a fully Framework-compatible fashion? In other words, we won't see things that are "similar" to the Framework APIs in question, but have Unity-specific tweaks, additions, or worst of all, re-thinks? I would expect features or capabilities to be (potentially) left out, which seems consistent with the "subset" concept, but to me it would feel very wrong to expand on Framework APIs in any way. In that case I'd think Subset would do more harm than good -- just put those things over in the Unity namespace.

 

Excellent. Subset will be an "interesting" twist for your documentation team to address clearly...

Full Framework is experimental now, but it appears to me that Unity expects to stabilize it for production-level usage long before Standard and Subset work will appear as experimental features. How much stability risk does Unity anticipate from adding Standard and Subset .NET targets once full Framework is declared stable? I can see how full Framework implementation possibly makes them easier to add later, but I'm wondering how much one has the potential to impact the other, if at all.

 

But Microsoft is not considering this for .NET Core/.NET Standard already? Unity should avoid to modify .NET/Mono whenever it's possible.

 

What they're proposing is completely in line with what Microsoft has planned with the new .NET Standard concept.

.NET Standard isn't a thing you can use, it's strictly a specification.

.NET Core is just one implementation of that specification, as is .NET Framework. Anyone can implement more than the specification (as Framework does and Unity's Subset would), but you can't implement less than the .NET Standard API specification and still call it .NET.

Even though my preference is that Unity's Subset wouldn't deviate from .NET Framework, as far as Microsoft is concerned that would be fine, too. If the .NET Standard APIs are there, then it's a valid .NET implementation as far as Microsoft is concerned.

 

Hello,

Is there any timeframe to when it will be possible to use .Net Standard 2.0? Will it be in beta 2017.3 ?
I'm just very curious and exited for this.

Thank you!

 

@joncham In my opinion. I think you should always go full profile and deprecate subset

But then you could only test the subset

In other word, standard is the new subset but the real subset will hidden in full just for backward compatible, testing only what needed. And expose this information for enthusiasm to do checking. And use at their own risk. Maybe have a warning when using thing that not tested or not support all platform

If someone care about size they should use only netstandard

 

I don't seem to find any response to this specific case, any takers?

 

First off I consider the loss of Reflection.Emit a serious blow, I understand the AOT implications that makes this necessary on some platforms. So that means I would have to change my workflow, well all the stuff I do with Reflection.Emit at runtime, I could theoretically be doing as a build step generating an assembly with all the stuff I used to emit at runtime. However my initial attempts at this hit the System.FieldAccessException.

Obviously I can go unsafe and read the fields myself, however then I have to worry about differences in runtime implementations, not to mention that it is terribly messy.

@joncham and @JoshPeterson, any advice on how to progress? The microsoft documentation is very vague about a concept of full trust, which I should be able to obtain if I fire up another appdomain, but accessing private fields in an empty appdomain is kinda pointless.