Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Enter the 2020.2 Beta Sweepstakes for a chance to win an Oculus Quest 2.
    Dismiss Notice

[Bug 1214643][2019.3.0f5] System.Runtime.CompilerServices.Unsafe clashes with imported dll

Discussion in 'Data Oriented Technology Stack' started by MNNoxMortem, Jan 26, 2020.

  1. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    600
    MessagePack-CSharp requires System.Runtime.CompilerServices.Unsafe 4.5.1

    Unity.Burst and Unity.Collections require 4.5.0

    This causes Unity to go cause havoc:
    • Breaks all assembly definitions
    • Causes Unity to Crash
    • Unable to import the dlls anymore
    • Unable to show the inspector for any

    Unity.2019.3.0f5
    Having the following asmdef in my project

    Code (CSharp):
    1. {
    2.     "name": "Plugins",
    3.     "references": [],
    4.     "includePlatforms": [],
    5.     "excludePlatforms": [],
    6.     "allowUnsafeCode": true,
    7.     "overrideReferences": true,
    8.     "precompiledReferences": [
    9.         "Zenject-usage.dll",
    10.         "Sirenix.OdinInspector.Attributes.dll",
    11.         "Sirenix.Utilities.dll",
    12.         "Sirenix.Serialization.Config.dll",
    13.         "Accord.dll",
    14.         "Accord.Math.dll",
    15.         "Accord.Math.Core.dll",
    16.         "Sirenix.Serialization.dll",
    17.         "System.Memory.dll",
    18.         "System.Threading.Tasks.Extensions.dll",
    19.         "System.Buffers.dll",
    20.         "System.Runtime.CompilerServices.Unsafe.dll",
    21.         "Ookii.Dialogs.dll",
    22.         "System.Windows.Forms.dll"
    23.     ],
    24.     "autoReferenced": false,
    25.     "defineConstraints": [],
    26.     "versionDefines": [],
    27.     "noEngineReferences": false
    28. }
    whcih references a "non auto reference" to System.Runtime.CompilerServices.Unsafe.dll and updating packages from
    Code (CSharp):
    1. {
    2.   "dependencies": {
    3.     "com.unity.burst": "1.2.0",
    4.     "com.unity.collections": "0.5.0-preview.9",
    5.     "com.unity.entities": "0.5.0-preview.17",
    6.     "com.unity.ext.nunit": "1.0.0",
    7.     "com.unity.ide.rider": "1.1.4",
    8.     "com.unity.inputsystem": "1.0.0-preview.3",
    9.     "com.unity.jobs": "0.2.3-preview.9",
    10.     "com.unity.mathematics": "1.1.0",
    11.     "com.unity.memoryprofiler": "0.2.0-preview.1",
    12.     "com.unity.performance.profile-analyzer": "0.5.0-preview.1",
    13.     "com.unity.physics": "0.2.5-preview.1",
    14.     "com.unity.quicksearch": "1.4.1",
    15.     "com.unity.rendering.hybrid": "0.3.2-preview.17",
    16.     "com.unity.searcher": "4.0.9",
    17.     "com.unity.test-framework.performance": "1.3.2-preview",
    18.     "com.unity.testtools.codecoverage": "0.2.2-preview",
    19.     "com.unity.textmeshpro": "2.0.1",
    20.     "nuget.mono-cecil": "0.1.6-preview",
    21.     "com.unity.modules.animation": "1.0.0",
    22.     "com.unity.modules.assetbundle": "1.0.0",
    23.     "com.unity.modules.audio": "1.0.0",
    24.     "com.unity.modules.imageconversion": "1.0.0",
    25.     "com.unity.modules.imgui": "1.0.0",
    26.     "com.unity.modules.jsonserialize": "1.0.0",
    27.     "com.unity.modules.physics": "1.0.0",
    28.     "com.unity.modules.physics2d": "1.0.0",
    29.     "com.unity.modules.terrain": "1.0.0",
    30.     "com.unity.modules.terrainphysics": "1.0.0",
    31.     "com.unity.modules.ui": "1.0.0",
    32.     "com.unity.modules.uielements": "1.0.0",
    33.     "com.unity.modules.umbra": "1.0.0",
    34.     "com.unity.modules.unitywebrequest": "1.0.0",
    35.     "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
    36.     "com.unity.modules.unitywebrequestaudio": "1.0.0",
    37.     "com.unity.modules.unitywebrequestwww": "1.0.0",
    38.     "com.unity.modules.vr": "1.0.0",
    39.     "com.unity.modules.xr": "1.0.0"
    40.   },
    41.   "testables": [
    42.     "com.unity.test-framework.performance"
    43.   ]
    44. }
    45.  
    to
    Code (CSharp):
    1. {
    2.   "dependencies": {
    3.     "com.unity.burst": "1.2.1",
    4.     "com.unity.collections": "0.5.0-preview.9",
    5.     "com.unity.entities": "0.5.0-preview.17",
    6.     "com.unity.ext.nunit": "1.0.0",
    7.     "com.unity.ide.rider": "1.1.4",
    8.     "com.unity.inputsystem": "1.0.0-preview.4",
    9.     "com.unity.jobs": "0.2.3-preview.9",
    10.     "com.unity.mathematics": "1.1.0",
    11.     "com.unity.memoryprofiler": "0.2.0-preview.1",
    12.     "com.unity.performance.profile-analyzer": "0.5.0-preview.1",
    13.     "com.unity.physics": "0.2.5-preview.1",
    14.     "com.unity.rendering.hybrid": "0.3.2-preview.17",
    15.     "com.unity.searcher": "4.0.9",
    16.     "com.unity.test-framework.performance": "1.3.2-preview",
    17.     "com.unity.testtools.codecoverage": "0.2.2-preview",
    18.     "com.unity.textmeshpro": "2.0.1",
    19.     "nuget.mono-cecil": "0.1.6-preview",
    20.     "com.unity.modules.animation": "1.0.0",
    21.     "com.unity.modules.assetbundle": "1.0.0",
    22.     "com.unity.modules.audio": "1.0.0",
    23.     "com.unity.modules.imageconversion": "1.0.0",
    24.     "com.unity.modules.imgui": "1.0.0",
    25.     "com.unity.modules.jsonserialize": "1.0.0",
    26.     "com.unity.modules.physics": "1.0.0",
    27.     "com.unity.modules.physics2d": "1.0.0",
    28.     "com.unity.modules.terrain": "1.0.0",
    29.     "com.unity.modules.terrainphysics": "1.0.0",
    30.     "com.unity.modules.ui": "1.0.0",
    31.     "com.unity.modules.uielements": "1.0.0",
    32.     "com.unity.modules.umbra": "1.0.0",
    33.     "com.unity.modules.unitywebrequest": "1.0.0",
    34.     "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
    35.     "com.unity.modules.unitywebrequestaudio": "1.0.0",
    36.     "com.unity.modules.unitywebrequestwww": "1.0.0",
    37.     "com.unity.modules.vr": "1.0.0",
    38.     "com.unity.modules.xr": "1.0.0"
    39.   },
    40.   "testables": [
    41.     "com.unity.test-framework.performance"
    42.   ]
    43. }
    44.  
    breaks Unity completly and it does not recover. As there is no way to control the dlls added via packages I am not able to remove the namespace clash as Unity will add it's own version via Burst and that causes Unity to break horribly. All that would be needed is to make sure packages use their own dll and don't auto import them into any other asmdef in the whole unity project.

    This currently means I am not able to add anything to a Unity project which requires a different version of a dll that is added via Unity packages.

    Furthermore unity does do the following extremly nasty things:
    • Create new asmdef files, which removes ALL the dependencies you would require to compile the code within this asmdef, which causes the asmdef to not compile anyway
    • Fallback to Assembly-CSharp.csproj, .... default projects although it is clear that those also can never compile as all the dependencies and folders are set up as defined in the asmdef files and therefore this breaks
    • Crashes regularly once this happens. The only way to get Unity into a non crashing state is to close unity, manually edit all asmdef, delete all the dlls involved from all package cache folders, then remove the dll you would need and import from your code and then once there is no such nameclash anymore then start Unity again and reimport everything.
    • Auto reference added dlls from within a package (Burst) to outside of that package (e.g. ANY other csproj)

    Workaround

    See also
     

    Attached Files:

    Last edited: Jan 26, 2020
  2. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,039
    Setup Collections as a development package. That seems to be the most correct approach to resolve this short of Collections doing the right thing, which you can't exactly wait around for.

    Howto:

    - Copy Collections from Library/PackageCache to %Project Folder%/Packages
    - Remove CompilerServices.Unsafe dll from said folder
    - Add the version of CompilerServices.Unsafe dll that you want to use in Assets folder.
     
  3. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,039
    Also just to be clear I don't think the issue is burst, it's the Unsafe dll in Collections that is the problem.

    The dll's in .Runtime are not loaded globally, quick peek it actually looks like burst creates it's own appdomain. Which actually answers a question I posted yesterday about why burst fails loading our custom Unsafe dll (it already had it's own version loaded).
     
    MNNoxMortem likes this.
  4. June1111

    June1111

    Joined:
    Aug 11, 2017
    Posts:
    33
    Ran into this same issue but even worse with the number of assembly definitions we have. The only way to resolve it was either reference exactly every single dll a file depended on, or rework the other dlls to use the one shipped with collections. The way dlls inside packages seem to disappear when compiling things in assembly definitions is super frustrating.
     
  5. xoofx

    xoofx

    Unity Technologies

    Joined:
    Nov 5, 2016
    Posts:
    296
    Yes, the issue is with the Collections package. The proper way to fix this would be that Unity is shipping an official package with this CompilerServices.Unsafe dll on its own, or it is shipping an internal version of it (as we do in Burst) to avoid clashing with end-user dll. I will ping back the team internally to make them aware of this issue.
     
  6. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    600
    Please do not loose sight on all the other bugs listed above. It is one thing to get a nameclash in the same domain. It is another to bring the project in a (without manually recreating everything) unrepairable state because Unity begins to overwrite asmdef files with empty reference sections and cascade into a crash to desktop.
     
  7. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    600
  8. xoofx

    xoofx

    Unity Technologies

    Joined:
    Nov 5, 2016
    Posts:
    296
  9. jdh5259

    jdh5259

    Joined:
    Sep 14, 2017
    Posts:
    10
    Hopefully this might be useful to someone...

    I was running into a similar issue with different libraries depending on different versions of System.Runtime.CompilerServices.Unsafe. I tried a variety of different approaches to include using bindingRedirects (which would have been fantastic if they worked with Unity) but most of them failed.

    After scouring the internet for clues (I don't remember the exact source that helped), the only approach that sort of worked was to tie into
    AppDomain.AssemblyResolve
    . That way, when an attempt is made to resolve to a specific version on the assembly, you can redirect it to the version that is forcibly embedded in the Collections package. Below are the scripts I used to try to work around the issue (not sure if this is a good approach or not...).

    The Editor portion doesn't actually appear to work as I intended. I had to give up on some of my attempts when dealing with assemblies not loading in the Editor because AssemblyResolve was not getting consistently called for some reason.

    ResolveAssemblies:
    Code (CSharp):
    1. using System;
    2. using System.Globalization;
    3. using System.Reflection;
    4. using UnityEngine;
    5.  
    6. /// <summary>
    7. /// Resolves required assemblies. Somewhat mimics the typical app.config/bindingRedirect configurations which
    8. /// don't appear to work with Unity.
    9. /// </summary>
    10. public class ResolveAssemblies
    11. {
    12.     private static Assembly cachedAssembly = null;
    13.  
    14.     static ResolveAssemblies()
    15.     {
    16.         RedirectAssemblies();
    17.     }
    18.  
    19.     /// <summary>
    20.     /// Method that forces a call to the static constructor.
    21.     /// </summary>
    22.     [RuntimeInitializeOnLoadMethod]
    23.     public static void StaticInitialize()
    24.     {
    25.         // force call to static constructor
    26.     }
    27.  
    28.     private static void RedirectAssemblies()
    29.     {
    30.         //// The Unity Collections packages currently embeds a specific version of the
    31.         //// "System.Runtime.CompilerServices.Unsafe" library. Because of this, you are unable to include other
    32.         //// versions of the same library resulting in version conflicts. For now, redirect to the version in
    33.         //// the Unity Collections package.
    34.  
    35.         AppDomain.CurrentDomain.AssemblyResolve += AssemblyResolve;
    36.         AppDomain.CurrentDomain.DomainUnload += DomainUnload;
    37.     }
    38.  
    39.     private static Assembly AssemblyResolve(object sender, ResolveEventArgs args)
    40.     {
    41.         var requestedAssembly = new AssemblyName(args.Name);
    42.         if (requestedAssembly.Name != "System.Runtime.CompilerServices.Unsafe")
    43.         {
    44.             return null;
    45.         }
    46.  
    47.         // Cache the assembly in case multiple attempts are made for different versions
    48.         if (cachedAssembly != null)
    49.         {
    50.             return cachedAssembly;
    51.         }
    52.  
    53.         // Redirect to a specific version of the assembly
    54.         var targetPublicKeyToken = new AssemblyName("x, PublicKeyToken=b03f5f7f11d50a3a").GetPublicKeyToken();
    55.         requestedAssembly.SetPublicKeyToken(targetPublicKeyToken);
    56.         requestedAssembly.Version = new Version("4.0.4.0");
    57.         requestedAssembly.CultureInfo = CultureInfo.InvariantCulture;
    58.         cachedAssembly = Assembly.Load(requestedAssembly);
    59.         return cachedAssembly;
    60.     }
    61.  
    62.     private static void DomainUnload(object sender, EventArgs e)
    63.     {
    64.         AppDomain.CurrentDomain.AssemblyResolve -= AssemblyResolve;
    65.         AppDomain.CurrentDomain.DomainUnload -= DomainUnload;
    66.     }
    67. }
    ResolveAssembliesEditor:

    Code (CSharp):
    1. using UnityEditor;
    2.     using UnityEngine;
    3.  
    4.     /// <summary>
    5.     /// Resolves required assemblies in the Unity Editor. See <see cref="ResolveAssemblies"/>.
    6.     /// </summary>
    7.     [InitializeOnLoad]
    8.     public class ResolveAssembliesEditor : MonoBehaviour
    9.     {
    10.         static ResolveAssembliesEditor()
    11.         {
    12.             ResolveAssemblies.StaticInitialize();
    13.         }
    14.     }
     
    MNNoxMortem and SugoiDev like this.
  10. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    600
  11. June1111

    June1111

    Joined:
    Aug 11, 2017
    Posts:
    33
    ending up using a simple workaround for now
    move packages from packagecache to the packages folder (unity auto finds ones in there as ones your editing)
    then delete the dll from inside it (we needed both collections and newtonsoft packages)
    then in your project use the versions you need for other things (like our signalr client)

    when you update packages, make sure you delete those old ones and get the latest and redelete the dll's till you can get your hard referenced versions to go away.
     
  12. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,039
    Install the package development package and it makes this easy. Single click to add/remove from development.
     
    June1111 and MNNoxMortem like this.
  13. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    600
    In case the clash is on second level dependencies (A=>C.v1,B =>C.v2) the above deletion and copy workaround does not work out that easy, as B might not be shipped by unity but the dependency C clashes.

    You can recompile the library with a proper reference. That works for dlls like System.Threading.Tasks.Extensions:
    • Decompile
    • Change reference
    • Recompile
    In cases where this is not so straight forward (actually compiling System.Threading.Tasks.Extensions from the dotnet core library is not as easy as opening the csproj file in there and simply updating the version - but that obviously is one way to do it!) you can simply disassemble the dll and re-assemble it with a diferent reference version as long as the called code is compatible (obviously a nasty hack, but that hack takes a few seconds of your time and might unblock you). This is in particular helpful where decompiling and recompiling is not as straight forward as well (if anyone has tried that with Jetbrains DotPeek on System.Buffer.dll and System.Memory.dll s/he will know what I mean)
    This essentially is close to a proper binding redirect which is not available in the dotnet core way in Unity.
     
    Last edited: May 6, 2020
    June1111 likes this.
  14. Haapavuo

    Haapavuo

    Joined:
    Sep 19, 2015
    Posts:
    54
    Still no update in the Collections package. Come on Unity. Remove the DLL from the package so we can get our project to work!
     
  15. xoofx

    xoofx

    Unity Technologies

    Joined:
    Nov 5, 2016
    Posts:
    296
    MNNoxMortem and Piefayth like this.
unityunity