Search Unity

Please update com.unity.collections reference of System.Runtime.CompilerServices.Unsafe

Discussion in 'Package Manager' started by SLGSimon, Aug 21, 2019.

  1. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    Currently com.unity.collections is referencing System.Runtime.CompilerServices.Unsafe 4.3

    System.Memory requires System.Runtime.CompilerServices.Unsafe (>= 4.5.0) so anything referencing that will cause an error that cannot be fixed.

    Please move this out to a "nuget.xxx" package with updated versions.
     
    andywatts and GeorgeAdamon like this.
  2. ethan_jl_unity

    ethan_jl_unity

    Unity Technologies

    Joined:
    Sep 27, 2018
    Posts:
    104
    Hi there,

    I have forwarded this issue to the team in charge of the `com.unity.collections`. Thanks for the report.
     
    GeorgeAdamon likes this.
  3. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    Thanks @ethan_jl_unity.

    Would it also be possible to get some info on how these situations will be handled in the future, with unity packaging dlls from nuget?
     
  4. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    Sorry to ping you again, but is there an update on this? It's blocking updates to our shared libraries.
     
  5. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    +1
    This would be greatly appreciated. We are having massive issues with the NumSharp library at the moment, which needs System.Runtime.CompilerServices.Unsafe (>= 4.5.2) and at the same time we need Unity.Collections which ships with 4.0.3.0
     
    Last edited: Oct 1, 2019
  6. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    To circumvent my problem, I had to:
    1. Copy the folders com.unity.collections, com.unity.jobs, com.unity.test-framework and com.unity.test-framework.performance from Library/PackageCache/
    2. Paste them inside my Assets folder
    3. Delete or Disable System.Runtime.CompilerServices.Unsafe.dll from the Assets/com.unity.collections/
    4. Uninstall Unity.Collections and Unity.Jobs from the Package Manager
    So far everything works, and both NumSharp and Collections use the 4.5.2 version of System.Runtime.CompilerServices.Unsafe
     
  7. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    @SLGSimon, @GeorgeAdamon,

    I am really sorry you're still having issues. The issues have been forwarded to the relevant teams and I'll follow-up on this. In the meantime, you can get unblocked in a less drastic manner by embedding the faulty packages - similar to the steps taken by @GeorgeAdamon, but under the
    Packages/
    directory. This allows the package manager to still see these packages and install their dependencies, and you don't need to remove them from the project manifest.

    So updated steps would look like:
    1. Copy only the folder com.unity.collections from Library/PackageCache/
    2. Paste it under
      Packages
      (optionally rename it to
      com.unity.collections
      so it has the same name as in its
      package.json
      file)
    3. Delete or Disable System.Runtime.CompilerServices.Unsafe.dll from the Packages/com.unity.collections/
     
  8. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    Thanks for the tip @maximeb_unity and we are looking forward for this issue to be resolved!
     
  9. Jeff_Blumenthal

    Jeff_Blumenthal

    Joined:
    Apr 7, 2017
    Posts:
    8
    Could I please get more detailed information on the locations of these folders? I have looked in my editor and project folders and cannot find them. I'm on 2019.2.3f1. Thank you.
     
    Last edited: Oct 28, 2019
  10. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    Both
    Library/PackageCache/
    and
    Packages
    referred to folders directly in your project directory. Say your Unity project is in
    C:\Users\me\Documents\Unity\MyProject
    , then
    C:\Users\me\Documents\Unity\MyProject\Library\PackageCache
    and
    C:\Users\me\Documents\Unity\MyProject\Packages
    are the directories I was referring to.
     
  11. Jeff_Blumenthal

    Jeff_Blumenthal

    Joined:
    Apr 7, 2017
    Posts:
    8
    Thank you for the quick reply.

    For my project I did not have com.unity.collections package.

    I did solve my compile problems by downloading the latest nuget packages and then moving the following the netstandard2.0 dlls to my Plugins directory:

    System.Buffers.4.5.0
    System.Memory.4.5.3
    System.Runtime.CompilerServices.Unsafe.4.6.0

    Hopes this helps others.
     
  12. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,053
  13. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    They already have some nuget packages replicated in their package manager, just not all they are using. Though I haven't heard if this is their official way of dealing with the issue...
     
  14. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,053
    Yes but that package is not the official Nuget package, it is recompiled by Unity and when you want to use an official Nuget package that depends on another official Nuget package, it won't work. Or if you need some specific version and not the one that Unity provides.
     
  15. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    That's exactly what the link you provided is doing, it converts a nuget package to an npm/unity package. Unity - for who knows what reasons - has decided on npm as their package technology and I don't see that changing, so we just have to work within that framework.

    To make it work they're going to have to officially maintain a number of packages (nuget.xxx in the package manager), though I haven't heard anything official about this.
     
  16. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    @maximeb_unity Six months later and this is still a problem, and there have been a number of package updates since with no fix.

    Also you stated you were going to follow up with the relevant teams, what was the result of that?
     
    richardkettlewell likes this.
  17. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    @xoofx as you're the author of of UnityNuGet and work on burst this is probably relevant to you.

    As I've been updating to 2019.3 I've noticed burst is now packing a whole bunch of nuget dlls. It even includes mono.cecil that is in the nuget.mono-cecil package?

    I'm getting two warnings when I load up unity with burst 1.2.1 but it doesn't seem to be affecting anything. If I try to remove the dlls burst breaks so I'm not sure how they relate to the unity editor, or what to do here.

    Could not load assembly System.Buffers, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    UnityEngine.Debug:LogWarning(Object)
    Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:260)
    Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:256)
    Unity.Burst.Editor.BurstReflection:GetAssemblyList(AssembliesType) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:239)
    Unity.Burst.Editor.BurstLoader:.cctor() (at Packages/com.unity.burst@1.2.1/Editor/BurstLoader.cs:69)
    UnityEditor.EditorAssemblies:processInitializeOnLoadAttributes(Type[])

    Could not load assembly System.Buffers, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    UnityEngine.Debug:LogWarning(Object)
    Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:260)
    Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:256)
    Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:256)
    Unity.Burst.Editor.BurstReflection:GetAssemblyList(AssembliesType) (at Packages/com.unity.burst@1.2.1/Editor/BurstReflection.cs:239)
    Unity.Burst.Editor.BurstLoader:.cctor() (at Packages/com.unity.burst@1.2.1/Editor/BurstLoader.cs:69)
    UnityEditor.EditorAssemblies:processInitializeOnLoadAttributes(Type[])
     
  18. xoofx

    xoofx

    Unity Technologies

    Joined:
    Nov 5, 2016
    Posts:
    417
    These DLLs are in a special folder .Runtime that is not supposed to be processed by Unity editor at all, they are not part of the user Editor AppDomain, so they can't be found by BurstReflection.
    I suspect that the issue you are seeing is likely coming from another package you are using that is actually using these DLLs. We should probably remove these warning in Burst because I'm not sure they are relevant when things can't be loaded (because that's likely not a code that will be bursted in the end)
     
  19. Mortuus17

    Mortuus17

    Joined:
    Jan 6, 2020
    Posts:
    105
    Not only has this behaviour not been fixed with the update of DOTS; things have gotten way worse.
    Previously, the answer provided by maximeb_unity helped.
    Now, updating DOTS, numerous CS0246 errors pop up (Namespace of individual packages couldn't be found, even between each other). When trying to import DOTS to a brand new HDRP project, with the second or third package already, infinite import loops occur (all dependencies shown in package manager installed IN "CORRECT" ORDER) - am I missing something? Am I the only one who observes this? How do I fix it? And most importantly: When won't we have to deal with this anymore?
     
  20. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    @Mortuus17 there was a versioning problem recently, I think you need to manually update burst? Something like that. I don't think preview packages are shown as updates normally so you will have to turn them on or manually check available versions
     
  21. Mortuus17

    Mortuus17

    Joined:
    Jan 6, 2020
    Posts:
    105
    Unfortunately not.
    As said, this happens with blank HDRP projects, too.
    Upgrading to 2020.1.X(beta) fixes most compilation errors, but now the Entities namespace uses types that have been removed from Unity. I don't know what to say.
     
  22. unity_o-ud8Xgk1mqn3w

    unity_o-ud8Xgk1mqn3w

    Joined:
    Feb 10, 2018
    Posts:
    1
    Please update it's really annoying, I can't get to fix this error !
     
  23. greenheartgames

    greenheartgames

    Joined:
    Jan 29, 2015
    Posts:
    15
    +1 for a proper fix please. couldn't get the suggested workaround to work while trying to use System.Text.Json nuget package.
     
  24. coldwarrior5

    coldwarrior5

    Joined:
    Nov 9, 2016
    Posts:
    12
    +1 I also experienced the issue as I am using Ros# and they need Newtonsoft.Json.dll and the System.Runtime.CompilerServices.Unsafe.dll for it to work properly, with the recent changes to the DOTS package (collections, entities, jobs, etc.) I am getting conflicts between the two, please resolve this issue.

    For the Newtonsoft.json.dll the solution was relatively easy since you did create a package for it com.unity.nuget.newtonsoft-json@2.0.0 but you forgot to add an entry to the manifest.json, located under Project folde\Packages, so I had to do it manually, which is fine and works normally once done, but you should do that automatically.

    For the System.Runtime.CompilerServices.Unsafe.dll the solution is much more hacky and reduces performance since the collections package is treated as in development, so it is not optimized, but it also isn't very elegant, you essentially have to do this every time the package gets updated. Please take care of this, it is almost a year since this has been reported.
     
  25. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    120
    I am having a similar conflict issue with System.Runtime.CompilerServices.Unsafe.dll where I am attempting to use the MessagePack library (with a copy of this dll pasted into Assets/Plugins) and that works, but when I then attempt to install DOTS Entities it borks.
     
  26. UbiAnthonyB

    UbiAnthonyB

    Joined:
    Jun 28, 2019
    Posts:
    4
    Still a problem unfortunately :(
     
  27. bkaradzic-unity3d

    bkaradzic-unity3d

    Unity Technologies

    Joined:
    Jan 17, 2020
    Posts:
    17
    System.Runtime.CompilerServices.Unsafe.dll is removed from Unity.Collections package (next release [0.11.0]).
     
    MNNoxMortem likes this.
  28. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    What was the solution? Is it just not being used or moved elsewhere?
     
  29. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,053
    +1

    I'm curious.
     
  30. jsm174

    jsm174

    Joined:
    Jul 22, 2015
    Posts:
    16
    Hello. Any estimate on when 0.11.0 will be released?
     
    ConAim likes this.
  31. bkaradzic-unity3d

    bkaradzic-unity3d

    Unity Technologies

    Joined:
    Jan 17, 2020
    Posts:
    17
    It's not being used anymore. All necessary functionality (UnsafeUtility.AsRef and similar functions) that we used from .dll was implemented inside Unity 2020.1, and then this .dll is not needed anymore. It was there for a long time, except DOTS minimum version was 2019.3 so we couldn't remove it until minimum version was updated to 2020.1.
     
    ConAim and Favo-Yang like this.
  32. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    So this is irrelevant for 2019.4 LTS?
     
  33. bkaradzic-unity3d

    bkaradzic-unity3d

    Unity Technologies

    Joined:
    Jan 17, 2020
    Posts:
    17
    2019.4 LTS still requires this .dll to be present, functionality is only available in 2020.1 and above (DOTS collections package since [0.10.0] - 2020-05-27 require 2020.1, .dll is removed in 0.11.0 which will be released soon).
     
    skent_unity and Haapavuo like this.
  34. Haapavuo

    Haapavuo

    Joined:
    Sep 19, 2015
    Posts:
    97
    When will 0.11.0 be available? We are in a huge need of this fix. Thank you.
     
    ConAim likes this.
  35. bsergeant

    bsergeant

    Joined:
    Jul 15, 2020
    Posts:
    1
    Hi there, we are also blocked by this, any update on when 0.11.0 will be coming ?

    We are trying to use the CSharp message pack library.
     
  36. UnityMaru

    UnityMaru

    Community Engagement Manager PSM

    Joined:
    Mar 16, 2016
    Posts:
    1,227
    It looks as so this was released last Wednesday :) Apologies for the delay in reply.
     
    Hazkin likes this.
  37. Glader

    Glader

    Joined:
    Aug 19, 2013
    Posts:
    456
    Except this entire class of problem remains UNSOLVED. Poor man's static linking and changing a namespace does not solve the general issue of assembly reference clashing in the Unity3D Editor from externally imported assemblies versus implicitly referenced assemblies contained in package dependencies. There are other assemblies that clash, this was just one of them.
     
  38. Hazkin

    Hazkin

    Joined:
    Sep 11, 2012
    Posts:
    3
    So, as far as I understand, there is no solution for version 2019.4 LTS?

    UPD. It turned out to be solved by moving the Collections package from Library / PackageCaсhe to Packages in the root of the project and renaming the "Unsafe" library in it
     
    Last edited: Jul 30, 2020
    UbiAnthonyB likes this.
  39. CianNoonan

    CianNoonan

    Joined:
    May 19, 2017
    Posts:
    139
    We've hit a similar issue, we use System.Memory for ReadonlySpan among others, this requires CompilerServices.Unsafe which then results in the below error from burst.

    Code (CSharp):
    1. Could not load assembly System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
    2. UnityEngine.Debug:LogWarning(Object)
    3. Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Library/PackageCache/com.unity.burst@1.2.3/Editor/BurstReflection.cs:260)
    4. Unity.Burst.Editor.BurstReflection:CollectAssembly(Assembly, HashSet`1) (at Library/PackageCache/com.unity.burst@1.2.3/Editor/BurstReflection.cs:256)
    5. Unity.Burst.Editor.BurstReflection:GetAssemblyList(AssembliesType) (at Library/PackageCache/com.unity.burst@1.2.3/Editor/BurstReflection.cs:239)
    6. Unity.Burst.Editor.BurstLoader:.cctor() (at Library/PackageCache/com.unity.burst@1.2.3/Editor/BurstLoader.cs:69)
    7. UnityEditor.EditorAssemblies:ProcessInitializeOnLoadAttributes(Type[])
    8.  
     
  40. SLGSimon

    SLGSimon

    Joined:
    Jul 23, 2019
    Posts:
    80
    This is just a warning, and you can ignore it - though I agree it would be nice if it didn't happen.
    You can ignore the exception in visual studio if you are getting it when debugging.
     
  41. CianNoonan

    CianNoonan

    Joined:
    May 19, 2017
    Posts:
    139
  42. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    Updating to the latest version of burst in 2019.4.9f1 I still have this issue.
    Moving the folder as mentioned above does work, but that doesn't feel like a great fix.
     
  43. elfnik

    elfnik

    Joined:
    Feb 3, 2018
    Posts:
    8
    Solution:

    Update all 4 packages 1 by 1. Somehow when you update only one, some of the others still stay on the older version (from package manager):

    shader graph
    core rp libraries
    high definition rp
    high definition rp config.
     
  44. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    And as the root problem is not solved, the same problem is popping up over and over again. This time by com.unity.search. I am more than mildly annoyed by now.
     
  45. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    996
    @MNNoxMortem what did you end up doing to resolve the conflicts? Is the core issue just swept under the rug again?
     
  46. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    I deassemblied the dll, modified the header and reassembled it. This is not a recommended solution.

    So far Unity has not solved management of external libraries and dll depenency hell and the classic binding redirects are not really available. However, please not forget, that this is understandable. A lot of things can go wrong with version incompatibilities. By moving towards a .NET scripting runtime they are moving in the right direction and I would assume binding redirects are also available as a byproduct. From my perspective Unity is not sweeping this under the rug, but have considered that a proper solution is better than another self-made hotfix. To be honest I support that. Obviously I can only guess, but this is definetly how I believe Joachim tries to move forward: Better long term solutions over quick fixes. I might be wrong, so take this with a grain of salt.
     
    bdovaz likes this.
  47. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    996
    In the end, I discovered open source UPM's - from plugins, using package.json - from end user, using manifest.json

    Check out this site for the biggest lead (includes system.buffers, newtonsoft, etc):

    https://www.openupm.com

    This seems to be the long-term solution (and a bit annoying that someone else has to do it before Unity, but at least it's open src because Unity delayed).

    This should honestly be pinned because there's 0 resources how to properly manage this - took me hours research.
     
  48. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,053
    https://github.com/xoofx/UnityNuGet
     
  49. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    It is not a solution at all. openupm is just a package manager (or more precisely, a package registry) and nothing else. It does not do any magic. It can help you maintain and upgrade versions easier or give you access to a wider range of packages, but that's it. If you have a dependency conflict, you will have a dependency conflict loading the packages via any other source. In .NET you have exactly two options: 1) Resolve the conflict or 2) use binding redirects.