Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Built In NuGet/Package Manager Support

Discussion in 'Experimental Scripting Previews' started by james7132, Nov 10, 2017.

  1. james7132

    james7132

    Joined:
    Mar 6, 2015
    Posts:
    166
    With a more modernized runtime and platform, we are afforded use of the rest of the .NET ecosystem. It's great to be able to just download a DLL or extract it from a NuGet package and have it work in a Unity project.

    This is a great step forward in making developers lives easier. However there are still a few pain points that should be addressed:
    • Including third-party DLLs still requires you to download and hand configure dependencies
    • Plugins need to be included in source control as binaries. This doesn't work well with DVCS like git.
    • There's no validation of compatibility between DLLs included in a project.
    All of which can be resolved with first party integration with NuGet. I see in 2017.2 there's the introduction of PacMan as a Unity package manager, but this seems to be largely for engine components and potentially specially tailored packages for Unity, without anything regarding potential integration with NuGet for scripting.

    Is there any plans for this in the future? If there is, is there an ETA on when this will be available?
     
    rdcm, Nanie, ModLunar and 2 others like this.
  2. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,051
    I also want to know this.
     
  3. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    Eh, not really. Some of my Asset Store products have dozens or conditional defines like #if UNITY_5_4_OR_NEWER in order to support all possible versions of Unity in the source code. Having the requirement to build a .dll would probably mean having to build 10 or so .dll's to make it compatible with all supported Unity versions. This is less than ideal, especially since projects do sometimes get upgraded to newer Unity versions.
    Including plugins as source code solves a lot of problems like this one.
     
  4. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,051
    I think that he meant nuget dlls that obviously don't use Unity API. Asset Store publishers still need to provide source code (preferaly with assembly definition files) to deal more easily with conditional defines or provide multiple dlls.

    Soon Unity it's going to support .NET Standard 2.0 so all libraries compatible with .NET Standard should be compatible with Unity.
     
  5. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    Unity is never going to fully support .NET Standard 2.0. Why? Because of Unity moving towards IL2CPP, which is an AOT platform. This means that any .NET features that rely on the JIT compiler are not possible, mainly System.Reflection.Emit, Expression Trees, and runtime assembly loading. These features are used a lot in some advanced libraries, and it is not possible to make them work with AOT. Period. There will always be a market for .NET libraries "fixed" to work with Unity.
     
  6. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    @ZimM Support nuget does not really require full netstandard. Just compatible is enough. And most library in nuget does not really rely on dynamic codegen that much. The point is unity must use it and be nice with it

    If you are concern about AOT then CoreRT is there and it can still use nuget
     
  7. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    Sure, NuGet support will still be useful. I'm just saying it will never be 100% compatible.
    CoreRT has the same limitations.
     
  8. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    We should be fine with the same limitation as coreRT just to have nuget
     
  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    Actually, we are going to support .NET Standard 2.0 completely. It does not require JIT compiler support, so we can make it work on all platforms.

    Also, we're not moving toward IL2CPP only. It is the future of AOT support in Unity, but we're are absolutely keeping full Mono JIT support on platforms where JIT is allowed.

    Our goal is to make sure that any .NET library which can compile with .NET Standard 2.0 will work with any platform Unity supports.
     
    ModLunar, malyzeli and Thaina like this.
  10. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    We don't have plans to support Nuget at the moment. It is more of a resource issue than anything else - there are just other higher priority items.

    I think it would be possible to support Nuget entirely via editor scripts though, so maybe someone can put together an asset store package that does this.
     
  11. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    But what about features that do require JIT? Obviously, you can't, for example, load an assembly dynamically in runtime when AOT is used.
     
    Last edited: Nov 13, 2017
  12. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    @ZimM I think the point here is, in the future we should not use AoT as much as we do now

    Even WebAssembly which don't support JiT directly could support it by workaround on JS side (load another module dynamically by js). More and more platform does not require AoT. In mobile only iOS is need AoT nowaday and that was because iOS is naive senseless prison. Even then we could write webassembly game to run in mobile in the future. ServiceWorker in iOS also already in the plan

    AoT is just workaround in the meantime. Future game should always support JIT. So we could hope for deprecation of AoT and ready to do dynamic codegen in unity
     
  13. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,334
    All consoles are as of now AoT only. That attitude seems to be ingrained enough that I doubt we'll see any change on that front any time soon.
     
  14. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    I don't believe that anything in .NET Standard 2.0 requires JIT. For example, the APIs in System.Reflection.Emit are not in .NET Standard 2.0, so we don't need to support runtime code generation. Therefore, .NET Standard 2.0 will work on platforms like iOS and consoles.

    Also, it is possible to load an assembly via something like Assembly.Load if that assembly what AOT compiled. At the moment, IL2CPP requires that all assemblies be present together during a single AOT step, but that is a restriction we can lift if necessary. We do it now for optimization of cross-assembly method calls, but it is not a requirement. I can imagine in the future we may allow separate AOT compilation of assemblies, then allow those native shared objects to be loaded at runtime on platforms that allow this.
     
    Thaina likes this.
  15. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    Sound very suspicious that we could really do dynamic codegen if we managed put AoT compiler to compile the dynamic code again
     
  16. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    We're not planning for dynamic code generation. On platforms like iOS, that is not possible due to the restrictions on loading executable memory. But the Mono AOT engine already generates separate AOT'd binaries for each .NET assembly. IL2CPP could do that same (although it is not in our plans now). That's all I am saying.
     
  17. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    Actually, System.Reflection.Emit is available. Also, expression trees. So, nothing changes here, a library that uses these features would need to be adapted for AOT platforms.
     
  18. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    The actual complete netstandard was already implemented Expression Tree Interpreter that would be likely to work on AoT platform
    https://stackoverflow.com/a/47476886/1513380

    Only limitation here is Reflection.Emit that very rarely used in common library and it not much problem to just throw exception in iOS. Also we could just do interpreter wrapper on Reflection.Emit if needed

    And seriously, iOS would not accept the app that do many runtime codegen anyway
     
  19. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    Well, the parts of System.Reflection.Emit which are implemented don't provide any APIs for dynamic code generation, I think. They seems to be only enums and such: https://docs.microsoft.com/en-us/dotnet/api/system.reflection.emit?view=netstandard-2.0
     
  20. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
  21. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    That is correct. But I don't think these are a part of .NET Standard 2.0. So if you need to write portable code for Unity (maybe a logging library, for example), I would not recommend using this package.
     
  22. DenverCoulson

    DenverCoulson

    Joined:
    Aug 8, 2012
    Posts:
    9
    @JoshPeterson Is there any change on the roadmap for separate binaries for each .NET assembly or even for native libraries? This kills the use of many open source projects due to licensing restrictions.
     
  23. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    @DenverCoulson

    We don't have plans to do that now, although I wouldn't rule it out entirely.
     
  24. DenverCoulson

    DenverCoulson

    Joined:
    Aug 8, 2012
    Posts:
    9
    Got it, understood. Thanks @JoshPeterson.
     
  25. BullDoze

    BullDoze

    Joined:
    Jun 13, 2018
    Posts:
    14
    Can you elaborate a bit on that? I was not aware of this issue until you mentioned it. In what way is the licensing violation introduced?
     
  26. mrwellman_work

    mrwellman_work

    Joined:
    Jan 11, 2021
    Posts:
    4
    @JoshPeterson are there any updates on the topic.

    As of now I found three ways of importing non is really satisfying (Source https://forum.unity.com/threads/nuget-packages.1188286/#post-7602937)
    1. Download nuget package from nuget.org. unpack it as zip and copy dll from lib/netstandard2.0 or lib/netstandard2.1(for unity 2021.2) to you project. same should be done for each dependency of package.
      1. For downloading you can also use this process https://jonfoust.medium.com/using-gcp-nuget-packages-with-unity-8dbd29c42cc4, disclaimer I haven not tired it. It also explaines a bit more about the importance of the link.xml file.
    2. Use UnityNuGet scoped registry which have some nuget packages converted to upm. https://github.com/xoofx/UnityNuGet
    3. Use NuGetForUnity https://github.com/GlitchEnzo/NuGetForUnity. But it does not work for all packages e.g.: import of System.Collections.Immutable https://github.com/GlitchEnzo/NuGetForUnity/issues/401#issuecomment-1025355247.
    To me the most promising is 3. NuGetForUnity but it still needs a lot of work.

    Colliding dependencies are also a problem. E.g. we are using Microsoft AppCenterSDK and now I'm trying to add the currently best implementation of a Net GraphQL Client https://chillicream.com/docs/strawberryshake. Both depend on System.Runtime.CompilerServices.Unsafe.

    Also a big praise adding NET Standard 2.1 to Unity 2021 was what made it possibel to use StrawberryShake in the first place!
     
    Last edited: Jul 21, 2022
  27. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    You can find more details about our plan for NuGet Support in this blog post and the video of our GDC talk this year: https://blog.unity.com/technology/unity-and-net-whats-next

    I don't have an ETA for the release of this yet though, but we are actively working on it.
     
    mrwellman_work likes this.
  28. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    For such times. I feel like ETA is not the important information for us developer in any ecosystem. It is extremely unreliable and never really trustable

    But I think maybe the better indicator than ETA is transparency. Such as explanation of what already done and what is the roadblock so we can estimate it as "near future" or "far future" which is even more important than ETA
     
  29. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    The team is actively working now on moving the script compilation pipeline in Unity to use msbuild. That is a prerequisite for NuGet support. So I would estimate "far future" now, but I don't know how to define "far". :)
     
    bdovaz likes this.
  30. mrwellmann

    mrwellmann

    Joined:
    Nov 27, 2015
    Posts:
    37
    How could I missed this on GDC^^
    Even if it is "far future" you made may day:) and by watching the video I feel you are passionate about this topic :)

    Lately only controversial news like Ironsource and some comments from Riccitiello bubbled up to me. So this gives me a lot of faith in the Game Engine I spend most of my work live and also some private time with.

    ╱╱┏╮
    ╱╱┃┃
    ▉━╯┗━╮
    ▉┈┈┈┈┃
    ▉╮┈┈┈┃
    ╱╰━━━╯

    (MarkusFiredrill is my company account)
     
    bdovaz and JoshPeterson like this.
  31. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    That's the point though. Near and Far is subjective to each person circumstance so it's fine to know just that. Thanks for your information

    Also now I wish I could see new msbuild system came out to see how we would do things without asmdef
     
  32. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,163
    Hello, are there any progress in this field?

    I have also seen some people mention something interesting which would could made nuget package be more align with unity https://forum.unity.com/threads/unity-future-net-development-status.1092205/page-26#post-8539679
     
  33. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
  34. Nit_Ram

    Nit_Ram

    Joined:
    May 8, 2016
    Posts:
    27
    Any news on this info, now almost two years later?
     
  35. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    I don't have an ETA for this yet, but you can follow the uber-thread about .NET Modernization in Unity here: https://forum.unity.com/threads/unity-future-net-development-status.1092205/

    There is a lot of other discussion there as well. It is the best place to learn about what we are doing to support modern .NET in Unity.