Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Introducing OpenUPM - UPM Registry with Automatic Build Pipelines

Discussion in 'Package Manager' started by Favo-Yang, Dec 23, 2019.

  1. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
    Hi Folks,

    My Secret Santa Gift for the 2019:

    It's been a while since the first leak on the unity forum. For the past few months, I've been working on this hobby project. I'm glad that the OpenUPM service reaches beta stage!

    screencapture-openupm-2019-12-23-01_52_18.png
    OpenUPM is a service for hosting and building open source unity package manager (upm) packages. It's composed of two parts: a managed upm package registry with an automatic build pipelines. The intention is to create an universal platform to discover, share and distribute open source upm packages, and a community along with it.

    The major difference between OpenUPM and other 3rd-party upm registry is that the service does not require project owner to submit package updates regularly. Instead, it maintains a curated list of upm repository and enables any GitHub users (called packager hunter) to discover and submit upm repositories to the list. The automatic build pipelines monitor the changes and publish new releases later then. The unique approach helps grow the platform with the community.

    To learn more
    - Read full announcement at https://www.patreon.com/posts/openupm-beta-is-32544176
    - Read openupm documentation at https://openupm.com/docs/
    - Try it out!

    Happy Holiday and Happy Coding
     
  2. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    273
    Hey, amazing job you did there. Congratulations.

    I looked around for a while but I couldn't see an option for uploading a package from the command line. Is it possible?
     
  3. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
    @starikcetin Thanks for your awesome-upm project, really helpful.

    While there's no way to publish (upload) package via the command line (push mode). The service uses GitHub as a database for the curated package list (CRUD, PR as review), and the automatic build pipeline does the package publishing job for you (pull mode).

    I may create an `openupm publish/submit` command, but that will be a shortcut to fill the package form.

    There maybe one exception that if a package need very custom build process, which has to relay on push mode. But I'm looking for an example repository to figure it out.
     
  4. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    273
    Got it. Will be adding my packages when I get the chance :)
     
  5. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  6. AskCarol

    AskCarol

    Unity Technologies

    Joined:
    Nov 12, 2019
    Posts:
    214
    This is great! Congrats on this wonderful project.
    We shared it with the team :)
     
    Favo-Yang likes this.
  7. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  8. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  9. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  10. TheDudeFromCI

    TheDudeFromCI

    Joined:
    Oct 21, 2014
    Posts:
    15
    The project is growing pretty quickly! Trying it out; submitting and downloading packages feels really natural. This project is going to take off, for sure.
     
    bdovaz and Favo-Yang like this.
  11. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    785
  12. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    785
    @Favo-Yang There's one thing that worries me now that I see all this starting to take off.

    I see that there are many packages published in openupm that use third party libraries (usually nuget) and have the problem that when two packages include different versions they conflict.

    Real examples with official Unity packages:

    https://forum.unity.com/threads/ple...system-runtime-compilerservices-unsafe.732056

    Example of a package published in openupm:

    https://github.com/Halodi/halodi-unity-package-registry-manager/tree/master/ThirdParty

    https://openupm.com/packages/com.halodi.halodi-unity-package-registry-manager/

    For this there is a very good solution created by a Unity developer (@xoofx):

    https://github.com/xoofx/UnityNuGet

    In case @xoofx doesn't want to mantain new packages addition via PR:

    https://github.com/xoofx/UnityNuGet/blob/master/registry.json

    Would there be any way to publish the project from this repository on a server and make it publicly accessible? This way nuget dependencies could be referenced:

     
    YurySedyakin and Favo-Yang like this.
  13. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
    @bdovaz,

    Thanks for bringing this into my attention. Nesting DLLs directly into the UPM package is a hacky practice to make a single package just work. But get troubled when two packages included the same DLL. To resolve the conflict, you have to eject (embed) one package manually to the Packages, then remove the DLL. The ideal solution is to have a group of NuGet packages converts to UPM format to allow other packages to reference.

    xoofx's UnityNuGet is such a project with limited resources and an automatic approach. These packages are using org.nuget as scope.

    https://github.com/worldreaver is also something similar and published to OpenUPM. e.g. https://openupm.com/packages/com.worldreaver.systemmemory/. These packages are using com.worldreaver as scope.

    There're other cases like https://openupm.com/packages/jillejr.newtonsoft.json-for-unity/ does for Newtonsoft.Json, by adding more features like custom builds targeting different platforms and link.xml for code stripping.

    Notice that though these packages are using their own package namespace, the DLL filenames are the same. So you can only install one of them.

    What I may come with is a solution to make it easier to publish NuGet package to OpenUPM.
    - It can be something like xoofx does, using an automatic approach. I need more research to see if it can be achieved for adding customization automatically like the newtonsoft.json example above.
    - It can be something like worldreaver does use one GitHub repository for one NuGet package, but with a more "official" namespace. This will be less work for OpenUPM to verify those packages and more flexible to add customizations. And we may need to develop a GitHub Action to help track NuGet changes and create PR automatically.

    I'm not sure which way to go, probably need some trial-and-error process. When these packages are ready, we can start to promote it to the community, but promoting a good practice may take a long time.
     
  14. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  15. JesperSmith

    JesperSmith

    Joined:
    Mar 23, 2018
    Posts:
    17
    To avoid conflicting DLL's I
    - Prefix the name with a unique hash (package name resulted in very long path names)
    - Disable "Auto Reference"
    - In the packages asmdef, enable "Override references" and reference the assemblies explicitly.

    However, I have only tested with the same dependency in multiple packages. Yesterday, I made a mistake and forgot to update the assembly references and the package still worked after publishing, which is unexpected.

    If conflicts still show up, I might have to switch to including the source for third party dependencies instead of the dll and make them unique to the package with an assembly definition.
     
    Last edited: May 30, 2020
  16. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  17. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  18. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    401
  19. CDF

    CDF

    Joined:
    Sep 14, 2013
    Posts:
    911
    Thank you for taking the time to make openUPM.
    It's so good!
     
    Favo-Yang likes this.
unityunity