Bug 2021.2.0b6 and System.Memory/ReadOnlySpan under .NET 4.8

Hello!

Referencing #1355846, it seems that b6 has some nasty behaviour changes (likely due to the update to .NET 4.8). System.Memory seems to be suddenly merged into the mscorlib even for 4.8, which means types that previously had to be supplied using assemblies (System.Span/System.ReadOnlySpan from System.Memory.dll) are suddenly available/can no longer be imported by an assembly. This is a bit odd because System.Memory is not a part of the .NET Framework 4.8.

In theory, this change could be nice, but sadly ReadOnlySpan seems to be broken. In particular, attempting to index it will throw an error:

Code (CSharp):

1.  
2. System.ReadOnlySpan<int> span = new System.ReadOnlySpan<int>(new int[10]);
3. UnityEngine.Debug.Log(span[5]);
4.  

will result in

This is a bit frustrating, because it effectively means that ReadOnlySpan is no longer usable (and, by extension, a big chunk of whoever was using it before).

Because it effectively bricks projects using ReadOnlySpan, I'll have to skip b6 and revert back to b5.

 

ReadOnlySpan<T> seems to not be even defined in mscorlib as I'm getting a ton of:

error CS0012: The type 'ReadOnlySpan<>' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.

Whenever I try to remove System.Memory to solve the double declaration.


Looking closely there are some:

error CS0433: The type 'IAsyncEnumerator<T>' exists in both 'Microsoft.Bcl.AsyncInterfaces, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' and 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

How did that class even get there?!?! b6 mscorlib is cursed as hell.

 

Hm, quite odd. For me, when I had System.Memory still there, it talked about a clash between mscorlib and System.Memory, and Visual Studio's F12 (after re-creating the csprojs) also led me to the mscorlib.

It seems that with their upgrade to 4.8 and more specifically netstandard2.1, somehow classes of the latter got thrown into the former.

 

The issue on my side is the same as @Baawk
Will submit a bug report soon.

 

Mmm part of my code is compiled as a dll in a separate project, those ReadOnlySpan errors are actually coming from there since that project references System.Memory.

 

Nice to hear that it's reproducible for others as well! I'm not too familiar with Unity bug reporting, but as (perhaps badly stated) in the first post, I've already reported it as 1355846. Not sure if multiple reports will help or hinder progress.

Ah, that's a good point actually. If you have third party dependencies which expect these types to be in System.Memory.dll, but they're actually now in mscorelib... How is that supposed to work? You could, probably, create an own System.Memory assembly that type forwards them to the mscorlib but... uh.

I'm beginning to think that it was probably a mistake that these types came in for 4.8, and should probably get removed again.

 

Oh yeah, I completely missed it!
Well, now we have two developer reporting in!

It seems so, since .net 4.8 does not support span at all, only 5.0+ (according to https://docs.microsoft.com/en-us/dotnet/api/system.span-1)
But, since it's mono, maybe it does, partially?

 

So far i know, mono implements .NET Standard 2.1 completely, including Span and Memory.
It would also make no sense for the .NET 4.8 profile to leave out the Span and Memory if the runtime (Mono) supports it anyway.

 

That entirely depends on how it is done, however. The current way (simply throwing it into mscorlib without providing type forwarding or similar) is just not working. Every dependency that relies on any type inside System.Memory (or other classes that got "merged" into mscorlib) will not work, therefore severely restricting the possible .NET libraries that one can use within Unity.

On top of that, it might make sense to allow users to ship their own System.Memory or similar. There might be special cases where a certain version is required (e.g. some odd bugs/library dependencies), and the b6 behaviour is a (major) breaking change from that.

 

Any news on this? Same problem in b7...

 

This change makes 90% of my game framework unusable. Really hope this will get resolved before the 2021.2 goes out of Beta

 

Yeah, it's chaotic for us too.
A lot of stuff we use depends on System.Memory.
I managed to switch to netstandard to test and got it working, but this is not a project where I want to just switch to standard like that.

I think this might be the first time I'm fully blocked out from a beta.

 

Same issue here. It also breaks so many libraries like System.IO.Pipelines depends on System.Memory.

 

Is this a bug in Unity that will be fixed, or a change which developers will have to adapt to?

 

Regardless of unity intention, this is definitely to be treated as a bug since it breaks a ton of 3rd party libraries including "official" .NET ones.
Avoiding this is exactly half of the reason why Unity won't change to .NET5 yet (the other half being the lack of app domains).

 

I suppose it could be doable to remove the types from the supplied mscorlib, but it would definitely be much nicer and less stressful if Unity could fix this obviously gone-bad migration.

In the issue system, my bug is still open. The lack of communication from Unity regarding this topic is quite disappointing, seeing as it's really severely crippling .NET 4.8 has backend, which still has some advantages over netstandard or is flat out required for some products.

 

I didn't get a reply either. Unity is in dire need of improving their communication with developers.

Let's hope it's not something that's harder to fix than it looks.

 

One of the assets we are using (Dissonance) has these errors in both 2021.0b6 & 0b7 - so perhaps you can test with this?

 

I'm one of the developers of Dissonance, if I can help debug this issue feel free to contact me at martin@placeholder-software.co.uk

To be honest right now I'm not sure why this issue would affect Dissonance - as far as I know we don't use Span/ReadOnlySpan anywhere (we use the older ArraySegment struct instead).

 

You're correct. If I create a new project in 2021.2.0b7, download and import the latest Dissonance, I get some compile errors, but it looks like only the syntax has changed (Specifically CopyTo no longer returns an ArraySegment).

 

I am also suddenly getting all this error :

Assets\Plugins\Ulid\Ulid.cs(188,62): error CS0433: The type 'ReadOnlySpan<T>' exists in both 'System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' and 'netstandard, Version=2.1.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'

 

Is this bug currently available at the Issue Tracker? So we can vote for it?

 

I just imported this into 2021.2 and I see what has happened. We defined our own

CopyTo

extension method on

ArraySegment

- now there's a built in

CopyTo

method which takes the same arguments but has a different return type. Fortunately this particular case should be easy to fix (I can just rename the extension) and will probably not affect other assets (not many people will have defined a

CopyTo

extension method that just happens to clash).

 

Don't copy System.Memory.dll manually to an .NET standard 2.1 Unity project.

 

Oh yeah, I see it... so it is now part of .net standard 2.1?

 

Yes
Edit: However, there still the bug, that Unity cannot find System.Memory, even if it is part of .NET Standard 2.1.

 

Beta8 just hit the hub but I don't see this issue in either known issues nor fixes.

 

This entire/thread revolves around .NET 4.8 as scripting backend however, not netstandard 2.1. There's no fix for this problem on .NET 4.8. For 4.8, these types should not exist, and if Unity shipped System.Memory themselves (which perhaps could be slightly problematic), they shouldn't do it in mscorlib (or unless they're also shipping a forwarding assembly to make sure external assemblies work fine).

My bug report is still open, so I suppose it hasn't been fixed yet.

 

My report is still open too, with no reply.
It's a bit concerning not seeing this issue in the "known issues".

I managed to hack my project to work on netstandard for the time being, just so I can continue participating in the beta testing, but this is a pretty serious issue and we'll have a lot of trouble if it's not fixed before release.

 

Mono has an Forwarder for System.Memory. And so far i know, Unity use the mscorlib from mono, not an custom fork.

 

You can check the b7 installation directory. There's no System.Memory assembly that could do the forwarding (unless they're doing this in some other, non-standard fashion) - plus the error messages that come up also point towards this not being the case for Unity.

Point being: Just removing the System.Memory.dll does not help; ReadOnlySpan is still broken, and external dependencies aren't working because they can't find System.Memory.dll. It might be possible to provide an own System.Memory.dll with those forwards, but frankly, that shouldn't be our job, and it still would not resolve the problem that at least some of the implementations that Unity provides don't work.

 

There is a System.Memory.dll in "MonoBleedingEdge\lib\mono\unityaot-win32\Facades" but not in "MonoBleedingEdge\lib\mono\unityjit-win32\Facades" Probably a bug in Unity's .NET profile?
https://github.com/Unity-Technologies/mono/tree/unity-master/mcs/class/Facades/System.Memory

 

The error in the tracker seems to be more of a problem with the wrong C# version. The indexer is an readonly ref return, an only work with C# >= 7.2.
The other problem many have here is that third party libraries with references to System.Memory don't work. If you add the dll manually then the members like ReadOnlySpan<T> collide with the existing ones. And if you remove the DLL, you get the message System.Memory cannot be found.

 

What are the benefits of waiting to switch to .Net Core instead of already doing the switch now?

 

Thanks!

 

@JoshPeterson Hi, thanks for detailed explanation. But about 3, removing System.Memory.dll simply throws missing reference error for many libraries from nuget. Do you mean that these libraries are unavailable until Unity completely move to .NET Core ecosystem?

 

I tried this with Unity 2021.2.0b9 today, and got the same bunch of errors :

Assets\Plugins\Ulid\Ulid.cs(188,62): error CS0433: The type 'ReadOnlySpan<T>' exists in both 'System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' and 'netstandard, Version=2.1.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'

I don't have any manual dll copies of System.Memory, but upon tracking it down a bit more reveals that:
There is one such as:
UnityProject\Library\PackageCache\com.unity.searcher@4.8.0\Editor\Lucene\System.Memory.dll

Which belongs to the Searcher package from package manager.

Wondering if Searcher isn't compatible? What gives?

 

The issue unsolved in 2022.1.0a7.1073.

 

If the fix made it to alpha7, u have to remove your own dlls from plugins folder and it should work

Oh nvm, seems like a package is still including it?