Unity Future .NET Development Status

As I said in a previous post if you try for example System.Text.Json NuGet package that targets 2.0 (no 2.1) and depends on packages that are assemblies that already come with 2.1 it would fail and there is no possible way to make it work. This has nothing to do with UnityNuget.

 

A suggested solution would be for Unity to automatically exclude all NuGet packages that are in the asset folder and that were added in .NET Standard 2.1 from the build process and only use those from Mono.
I tested this with the .NET5 app to see how they behave, and they do just that. In the case of System.Memory, they ignore the NuGet dependency and use the .NET5 version.

 

.NET 5 doesn't ignore System.Memory NuGet package.
System.Memory has multiple targets: net461, netcoreapp2.1, netstandard1.1, netstandard2.0.
When you use it in .NET 5 application (directly or as a dependency of another package) it is resolved using netcoreapp2.1 target (closer to net5 target).
And fun thin that System.Memory for netcoreapp2.1 is empty: NuGet resolve it but no dll added to the compilation.


With Microsoft.Bcl.AsyncInterfaces situation is similar but trickier. It has netstandard2.1 but dll in that target is empty, it has TypeForwardedTo only so it do not introduce duplicated types.

 

The System.Memory NuGet has current version 4.5.4, but when i published the test app as a self-contained app, the System.Memory.dll has the version 5.0.x.

I tested it with a 2.1 library, which refers to a 2.0 library, which in turn refers to System.Memory.
In the end it uses the .NET 5 implementation, not the NuGet implementation.

 

Yeah. I just write what happens from the NuGet side and how dependencies are resolved.

You are right System.Memory was taken from the framework itself.

What I want to say:
- no NuGet packages never get ignored;
- if some dll is part of some framework than NuGet just contains stub for that target(no dll or dll with redirects to another assembly)

So to simulate this behaviour in unity we need to have multiple dlls in package with different define constraints to match different target framework.

For example to port System.Memory as unity package that will work in pre 2021.2 and in 2021.2 we need to have a package with one System.Memory and define constraints that will disable it in 2021.2 and higher (line !UNITY_2021_2_OR_NEWER). so dll will be just ignored on 2021.2 what will be equal to using stub target in nuget package.

For Microsoft.Bcl.AsyncInterfaces we will need to have 2 dll in package:
- one from netstandard2.0 target and with !UNITY_2021_2_OR_NEWER constrain
- second from netstandard2.1 target with UNITY_2021_2_OR_NEWER constrain

 

If you need help, you know where to reach me.

 

Do you have any UnityNuget fork that targets net standard 2.1? I don't think NET_STANDARD_2_0 does anything (for non-Unity libraries) as it's incorrect define in NET world, it would have to be NETSTANDARD2_0 but I guess they don't do much in packages resolution.

 

Mmm but what happens if say, an asset store plugin depends on System.Memory so it comes with the dll? Would such packages have to release a new version or can unity deal with that from inside?

 

Proper packages change public key of libraries to not collide with others (for ex. Mirror with Mono.Cecil).

 

How does that work? When unity loads the now "changed" dll wouldn't it still find 2 copies of System.Memory.Span<> and fail?

 

Yeah I agree, I think this is an issue with UnityNuget, not with Unity.

How difficult would it be to update it for versions 2021.2 + to use a different version?

 

Forgot to mention, they also change namespace. Unity did the same in burst with Unity.Burst.Unsafe (original
System.Runtime.CompilerServices.Unsafe).

 

Oh, I didn't know u could force change a namespace in an external/framework dll. Makes sense.

 

But I think you are right that in more complex cases no one does it with System.Memory, System.Buffers etc. Before everyone ended up removing duplicate dlls but now it's impossible to do it with these included in Unity. Deleting project dlls may result in missing members as they were newer than included in net standard 2.1.

Few solutions I can think of:
1. Unity makes references resolution that generate proper csprojs and build with proper csc commands.
2. Users do it themselves by creating asmdefs and overriding references.
3. Unity makes proper nuget support and ignore common dlls in project.
4. Unity adds support for net standard 2.0 and net framework 4.8 (in other words option to hide/remove all additional included libraries).

 

Hey there

Do we have change to get compiler update to .net 6 before 2021 LTS so we can get benefits of C# 10 syntax sugar?
- Improved interpolated strings
- global using directives
- source generators 2.0
- etc.

 

As far as I know, the packages like System.Memory .NET Standard 2.1 are implementations without newer members, otherwise such libraries can no longer be used in conjunction with Standard 2.1. As example you have a 2.0 library with a reference to the System.Memory package, and if you use a member that does not exist in the 2.1 standard, another library or program that has its own implementation cannot find this member.

 

With the incorporation of System.Memory into the BCL, .net core included a lot of optimization around internal string manipulation in other APIs (by using Span, etc).

Josh, do u know if we also got some of those optimizations in the current version of mono that unity (beta/alpha) uses??

 

And i also hope, that NativeArray supports Span<T>. So that native arrays can also be used with libraries that support Span<T> without first having to convert them into normal C# arrays.

 

I think you should be able to do this quite easily.

Span<T>

has a constructor that takes a pointer, so this is just a matter of an unsafe utility method. It would be nice if this were added to the API, but it's a one line extension method to add yourself.

 

@JoshPeterson

hello sir
i am planning to update my install wizard short cuts and like to verify if I am correct to what I should include in options
did i miss any?


UNITY_2019_4 || 2020_1 || 2020_2 || 2020_3
.Net 4.x
.Net Standard 2.0

UNITY_2021_1
.Net Framework
.Net Standard

UNITY_2022_2
.Net Framework
.Net Standard
.NET Standard 2.1

 

It has already been said several times but:

2021.2
.Net Framework => Framwork 4.8 + Standard 2.1
.Net Standard => Standard 2.1

For Unity 2022.1 current the same as 2021.2 but maybe that could change, as there is still more than half a year until the release.

 

That was a typo on my side.

 

> There have been a few questions on this thread (and likely others) about "fast span" support - that is, intrinsics for parts of the Span API. Specifically, Mono has has used intrinsics to improve the performance of Span's get indexer since the .NET Standard 2.1 support shipped in Unity.

@JoshPeterson. Could you please clarify?

"fast span" has little to do with methods being specially handled by the runtime. Instead, it is entirely based around whether the span tracks an interior reference (that is a byref field) or if it explicitly tracks the backing array/object/pointer + an offset and a length.

This makes a big difference as to the size and overall performance due to needing less operations for any individual access. It, however, requires new runtime support for interior pointers and the lifetime tracking that goes along with that.

We also have methods that are special-cased in (intrinsic to) the runtime to help with bounds check elimination and other features to make it more on par with `array`, but that is possible with slow or fast span and is largely independent.

 

Adam Sitnik went into some details in his blog post here: https://adamsitnik.com/Span/
as does the original design doc:https://github.com/dotnet/corefxlab/blob/archive/docs/specs/span.md

-----------------

The summary is that "slow span" (sometimes also called "portable span") is the span used on runtimes without any byref field support. Its the implementation you'll find in `System.Memory` and is used on full framework. The last shipped implementation of it is: https://github.com/dotnet/corefx/blob/release/2.1/src/System.Memory/src/System/Span.Portable.cs

You'll see that it has 3 fields:
```
private readonly Pinnable<T> _pinnable;
private readonly IntPtr _byteOffset;
private readonly int _length;
```

`Pinnable<T>` is a special sealed class that arrays, strings, and other types which can be the backing GC reference are converted to to easily give access to the "first field". For pointers, this field is `null`. `_byteOffset` represents the offset of the first element of the span (or the raw pointer, for spans over pointers) and `_length` represents the `length` of the span. It will be `12` bytes on 32-bit systems and `24` bytes on 64-bit systems. Any access requires an initial bounds check, then checking if `_pinnable` is null and either returning `_byteOffset + index` if it is or getting a byref to `_pinnable.Data` and then adding `_byteOffset` and `index`. There are also limitations on what `Pinnable<T>` can wrap and its largely limited to arrays/strings.

-----------------

"Fast span" on the other hand is quite a bit different: https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Span.cs

This instead adds runtimes support for `byref fields` which were previously "illegal". It doesn't fully add this support such that they can be used anywhere, but instead defines a special internal `ByReference<T>` (https://github.com/dotnet/runtime/b...tem.Private.CoreLib/src/System/ByReference.cs) type which the runtime has special handling for (this type is intrinsic). This type is only used in `Span<T>`/`ReadOnlySpan<T>` and allows you to directly track either a pointer or a `byref T`. This allows you to skip storing the backing array altogether and allows you to completely ignore whether the span tracks a pointer or array/string/gc ref. Instead you simply bounds check and then add `byref + index` regardless. This also means APIs like `System.Runtime.InteropServices.MemoryMarshal.CreateSpan<T>(ref T reference, int length)` can exist, allowing you to create a span over any byref, not just arrays/strings.

In particular, MemoryMarshal.CreateSpan requires proper interior pointer tracking support (and therefore "fast span") and given its in NS2.1, this means that Unity will require the support for this to work as expected.

-----------------

If you need any other details, I'd be happy to provide them, as would I expect most of the rest of the .NET team.

We're generally happy for issues/discussions/questions to be logged on dotnet/runtime and we're available on other avenues as well, such as over e-mail.

 

> If you need any other details, I'd be happy to provide them, as would I expect most of the rest of the .NET team.

The same goes for any questions, asks, etc about System.Numerics or System.Runtime.Intrinsics in particular since I'm one of the area owners there. We've added many new APIs and perf improvements here and intrinsics in particular is a very bulky set of APIs once you start considering .NET Core 3.1+ support (they are not in .NET Standard 2.1, unfortunately).

 

So the fork of Unity's mono runtime has support for so-called `byref fields` ?

 

Chiming in super late, but to be honest, Domain Reload is a vestigial artifact of a time long gone by.
I'd really like it removed, as it was removed from .NET 6 with good reason.

Time for a cleaner way to do it, Unity. You had a good run,15 years isn't bad.
(honestly, Enter Play Mode options itself showed just how not needed Domain Reload really ought to be)

In fact, I am questioning the long term viability and sensibility of IL2CPP, but I am not only a Transpilerphobe, I am also a JIT fanatic (I would always trust a JIT compiler to do a better job at adapting to the target environment and performance envelopes than just about any ahead of time compiled binary and runtimes - if not now, then at least later, you know. Just in Time )

 

dotnet 6 RC1

https://devblogs.microsoft.com/dotnet/announcing-net-6-release-candidate-1/

 

From what I know the only alternatives are: MonoAOT and NativeAOT. First is used in Maui (xamarian ios), 2nd is experimental and works only on win-x64, linux-x64, osx-x64.

Unity could try MonoAOT but I guess it will be slower than il2cpp. Only thing that makes sense is CoreCLR instead of Mono on supported platforms and for editor.

 

I would prefer MonoAoT if it allow us to use existing Xamarin library on all platform that support Xamarin

 

Your information is a bit out-of-date. NativeAOT now can already work on win-x64, win-arm64, linux-x64, linux-musl-x64, linux-arm64 and osx-x64, and will work on linux-arm and win-x86 in near future. Also some community folks have ported NativeAOT to PS5 and Switch platforms and use it in production.

 

This last sentence is very interesting. Do you have some link to source of this info?

 

Thanks, docs are outdated. I've tried musl but it doesn't support cross-platfrom builds.

Try to benchmark performance against unity il2cpp. Interesting to see that there's optional interpreter that makes Assembly.Load and System.Reflection.Emit work on ios. https://devblogs.microsoft.com/xamarin/introducing-xamarin-ios-interpreter/

 

That suppose to work in unity too as of later version (maybe since 2021.2)

 

That interpreter would work in il2cpp? Haven't seen anything mentioned.

 

https://forum.unity.com/threads/are...tor-allowed-on-ios.489498/page-2#post-7263739

This thread comment should be meant for interpreter and part of `Reflection.Emit`. What still left is only full JIT compiler and `Reflection.Emit` related to JIT execution as direct assembly. But if it just ILGenerator and using interpretor it should already work for many version and really work in all situation in 2021.2

 

I don't think this change directly relates to the interpreter.

The issue was that all generic type instances needed to be known and compiled at build time and couldn't be later constructed dynamically. With full generic sharing, the generic implementation becomes universal and the runtime can construct any generic type instance when needed (i.e. just the type metadata, the implementation is the same).

This means the interpreter now works in more cases, where it would previously fail when it needed to construct a new generic type implementation. But this also means it won't work in all situations, other limits of AOT compilation still apply.