Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Unity Using GitHub Packages Registry with Unity Package Manager

Discussion in 'Package Manager' started by okcompute_unity, Apr 5, 2020.

  1. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    736
    Hi,

    Thanks to forum user @JakHussain, there is now a known method to use GitHub Packages Registry with Unity Package Manager. See the original post where @JakHussain shared his clever finding.

    WARNING: Using GitHub Packages registries with Unity Package Manager comes with a couple of limitations (Goto the end of the post for the list). Please be aware that we are posting this only to share community knowledge. Unity doesn't officially support GitHub Packages Registry. Use it at your own risk.

    There are a couple of steps needed to make it work.

    1 - You need to configure the
    publishConfig
    setting as described in GitHub Packages Registry documentation in a special way. You need to suffix the pakcage scope:

    Code (JavaScript):
    1.   "publishConfig": { "registry": "https://npm.pkg.github.com/@USER" },
    2.  
    Where USER is the Github organization or user (i.e.the scope). Basically, we are cheating on how NPM works and _inject_ the scope in the registry URL instead of prefixing the package name. This is one half of the trick. Unity Package Manager doesn't accept the
    @
    and
    /
    characters in package names. By moving the scope reference in the registry URL we don't need these characters in the package name anymore!

    Add the
    publisConfig
    configuration in your package manifest (package.json)

    2 - Set the name of your package without an NPM scope

    This is the other part of the magic. As stated in step #1, make sure you don't prefix you package name with an NPM scope (@User)

    Example:
    Code (JavaScript):
    1.   "name": "com.my-company.my-package",
    Not:
    Code (JavaScript):
    1.   "name": "@USER/com.my-company.my-package",
    WARNING: If you have previously published a package with an NPM scope in the name, it won't work. Your package should not have an MPM scope for all existing versions. It won't work otherwise.

    3 - Publish your package

    For that step, you can refer to GitHub documentation.

    4- Configure the scope registry in your Unity project manifest

    Code (CSharp):
    1.   "scopedRegistries": [
    2.     {
    3.       "name": "Company Packages",
    4.       "url": "https://npm.pkg.github.com/@USER",
    5.       "scopes": [
    6.         "com.my-company"
    7.       ]
    8.     }
    9.   ],
    10.  
    The URL must match the one configured in your publishConfig above in step #1.

    5 - Setup Npm Auth configuration

    WARNING: GitHub Packages registry always requires authentication. For private and public packages.

    For this one, I'll refer you to another post I've made explaining how.

    6 - Add your package in your project manifest

    GitHub Package Registry packages won't show up in the Package Manager UI All section. You have to install the package manually in your project manifest.

    Known limitations
    * The main limitation is discoverability. Even if your Github Packages registry-based scoped registry is correctly configured, packages won't show up in the Package Manager UI's All section. GitHub doesn't support any of the NPM search API endpoints (/-/all or /-/v1/search). You will have to resort to GitHub UI to search packages and manually add them to your project manifest.
    * Even if the GitHub Registry package is public, your users will need to provide authentication.

    As you can see, using GitHub Packages Registry with Unity Package Manager is not trivial. But, if you really need it, you can make it work!

    Hope this post will be useful to some of you. Again, thank you to @JakHussain for discovering how it can be done!

    Regards,

    Pascal and the Unity Package Manager team.
     
    _slash_, NagaChiang, hyagogow and 5 others like this.
  2. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    528
    This is sad. We should have option to use

    ```
    "publishConfig": { "registry": "https://npm.pkg.github.com" },

    // with

    "name": "USER/com.my-company.my-package", // without @
    ```

    Instead. This would be easier to orchestrate cross domain dependency. I may publish project that utilize project of other person or other organization easily. It would critically expand the usage of package manager over everything in one go with just one line of registry
     
  3. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    @Thaina that's down to how githubs package registry works rather than how unitys package manager works unfortunately. So far my experience with GitHub Packages has been pretty underwhelming since they really don't make it easy for you to use. Fingers crossed for the future though!
     
    hyagogow, Thaina and okcompute_unity like this.
  4. RobertAcksel

    RobertAcksel

    Joined:
    Sep 20, 2018
    Posts:
    3
    Why this? This excludes GitLab for use since GitLab requires scopes to be present and rejects packages without scopes.
    Seems that your suggested cheat does not work with Gitlab. And cheating should never be necessary (strictly against cheating, makes life harder for all of us).

    'Packages must be scoped in the root namespace of the project.'
    from https://docs.gitlab.com/ee/user/packages/npm_registry/#package-naming-convention
     
    Last edited: May 6, 2020
    watsonsong likes this.
  5. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    I would chalk this up to githubs package registry forcing our hand in cheating in this way. As far as I can tell, unitys package manager isn't the bottle neck in how packages are read and pulled in.
     
  6. JesperSmith

    JesperSmith

    Joined:
    Mar 23, 2018
    Posts:
    17
    Thanks for this guide. I tested it and it seems to work. We have the need to share with outside resources and this will help, although a pain to setup.

    We are using some custom tooling to manage/publish NPM packages, and this could be added to it.

    Is there a future where you think packages with the naming scheme @Scope/package-name will be supported? It seems that this is enforced by multiple NPM repository hosts. An option could be to ignore @Scope/ in the name except when pulling from the NPM repo, so the full package name will become "@vendor/tld.vendor.package-name"
     
    watsonsong and Favo-Yang like this.
  7. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    736
    Hi @JesperSmith,

    As you pointed out, the Gitthub Packages Registry is not easy to use with the Package Manager. Since it lacks discoverability API, we will not put more effort to try to be compatible with this registry. Maybe you can reach out to GitHub and ask them to support UPM registries format (including documentation, adapted API, etc.)?

    Regards,

    Pascal
     
    vzlomvl and wcoastsands like this.
  8. JesperSmith

    JesperSmith

    Joined:
    Mar 23, 2018
    Posts:
    17
    Fair enough. I'll see if we can make some extra tooling on our side to make it work well enough for our purposes.
     
  9. RobertAcksel

    RobertAcksel

    Joined:
    Sep 20, 2018
    Posts:
    3
    Unity would not need to support Github natively. We would even be capable of developing an extension on our own to discover packages on any registry which makes this somehow possible via an API. I don't belive GitHub will go implementing UPM registries format.

    Instead we are asking to support the NPM Scoping syntax which we will not be able to support via an extension on our own (except of hacking around it which would end up in writing our own package manager)
    This is because NPM registries like the one of GitLab (not GitHub...) requires a scope for packages and this is why unity editor can not import those packages from GitLab.

    (Maybe there is a better thread for this or there should be one)
     
    Last edited: May 14, 2020
  10. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    452
    I think the issue been discussed a few times. Supporting /-/all/ discovery endpoint is a better-to-have thing. What really blocks this is the scopeRegistry syntax rejects "@" letter. Unity decides to use dot notation as the scope, but other NPM based registry defaults to @Username syntax. It didn't sound like a tech limitation but a design decision that whether @Username conflicts with dot notation. Maybe there're other reasons behind, but for me, below are all valid scopes.
    Sure, there're other registries that won't force you to use @Username as scope for the private package. But it's kinda convention here for NPM, GitHub and GitLab. They're all big players on the market.
     
  11. sbinter_levelex

    sbinter_levelex

    Joined:
    Jul 5, 2018
    Posts:
    9
    Using 2019.3.7f1 we have been unable to replicate this behavior. UPM fails to pull in the manually specified package due to an inability to find it. Using npm install with the same token pointed towards the same URL works perfectly.

    This forum thread is only a little less than 2 months old, so it seems a bit strange that the behavior would already have diverged.
     
  12. TactileGames

    TactileGames

    Joined:
    Oct 6, 2014
    Posts:
    7
    This workaround seems to have been broken by a recent change by Github:

    https://github.community/t/unable-t...t-name-to-publish-url-as-of-07-10-2020/136643

    They no longer allow publishing of packages without the scope in the name.

    @okcompute_unity - do you know of any way that we can work around this and continue to use the Github Package Registry with Unity 2019.4?

    Alternatively, can you provide any suggestions for an alternative private package manager, that does not require us to host our own package manager (i.e. we are not really interested in self hosting verdaccio)?

    Best regards,
    Morten
     
    tactileunity50 likes this.
  13. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    736

    Hi @TactileGames,

    I'm sorry you were affected by this change from Github. Some discussions are happening right now between Unity and Github. I cannot promise anything but fingers crossed!

    In the meantime, a self-hosting solution that I can recommend is JFrog's Artifactory (https://jfrog.com/artifactory/). Easy setup and they offer a free tier that can be interesting for a small team/studio.

    Regards,

    Pascal
     
    JakHussain likes this.
  14. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    736
    I was told this morning by Github that they reverted the change they made:
    https://github.community/t/unable-t...sh-url-as-of-07-10-2020/136643/11?u=jcansdale

    So the method described in this post should work again. Note that I was advised this is temporary but it should work for a while. The next major version of their system may not support this anymore.

    Reminder: Unity doesn't officially support the Github Packages registry. Github did not implement all the npm API UPM relies on for the *scoped registries* feature.

    Regards,

    Pascal
     
  15. ddsim

    ddsim

    Joined:
    Oct 6, 2016
    Posts:
    5
    @okcompute_unity
    Excellent that it works again.

    What is the reason that the Unity Package Manager doesn't allow package names beginning with an `@` scope?
     
  16. bsymon

    bsymon

    Joined:
    Dec 11, 2016
    Posts:
    18
    @okcompute_unity Thank you for the guide ! It works, I succeed to publish my package on GitHub and download it on a project with the scoped registry.

    There is just one issue on the Package Manager. As you said, GitHub doesn't support for searching, but the Package Manager will try anyway and it "crash" it a bit.

    fail.png

    I'm on the "All packages" section, but once it failed to search on GitHub, it won't load any others packages ... It could be nice to just display a warning or fail silently.

    For information, i'm on 2019.4.5f1
     
  17. ImmotionPeter

    ImmotionPeter

    Joined:
    Sep 15, 2020
    Posts:
    10
    I've gotten as far as steps 4/5/6 - not sure which bit is failing. Package is hosted on a private GitHub organization repo.

    Unity gives me.
    I've got the .upmconfig.toml setup in Users/MyLogin/ with
    The project manifest.json is like so:
     
  18. bigcheese_

    bigcheese_

    Joined:
    Sep 6, 2011
    Posts:
    30
    Has anyone tried to have package dependencies and if you need to do the following with the scope @Owner/com.test.dependency ?
     
  19. frank-ijsfontein

    frank-ijsfontein

    Joined:
    Sep 11, 2015
    Posts:
    11
    Will the Unity Package Manager support npm’s scope notation in future versions? We use GitLab, and it can host npm packages, but only when using exactly that ("@Scope/package.name") naming convention. So after getting my package published on our GitLab server, I found out that all my work was for nothing because the UPM is not fully npm compatible.
    Should have read this thread first I guess. But seeing that GitHub also tried moving into the scoped naming requirement, are you going to fix this?
     
  20. UnityMaru

    UnityMaru

    Community Engagement Manager Unity Technologies

    Joined:
    Mar 16, 2016
    Posts:
    1,013
    In our view, this is not something that needs to be fixed, but a new feature request that we don't plan to implement at this time. We don't plan to support npm's scope notation in the package names, a package with the name `@foo/bar` is not valid.
     
  21. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    411
    I think the problem is not the scope notation supporting, but the request to support gitlab or github registry.
     
    Thaina and prisonerjohn like this.
  22. prisonerjohn

    prisonerjohn

    Joined:
    Jul 31, 2013
    Posts:
    24
    Agreed. We want to use Gitlab / Github repos as we don't need to run or host separate servers for it. That seems like a reasonable request, especially as these are the two largest code hosting services used today. If they use `@foo/bar` for package names then maybe Unity should look into supporting that nomenclature.
     
  23. bsymon

    bsymon

    Joined:
    Dec 11, 2016
    Posts:
    18
    I agree, should be supported. I use GitHub for personnal stuff and at work we use a self-hosted GitLab. Would save us tons of time.
     
    Enderlook likes this.
  24. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    411
    If there is no official solution, can add an extension point that let's the community support the gitlab?
     
  25. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    411
  26. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    615
  27. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    411
    I see an article in GitLab about `Store all of your packages in one GitLab project`:
    https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html

    I think it worth to try it in the new GitLab version.
     
  28. frank-ijsfontein

    frank-ijsfontein

    Joined:
    Sep 11, 2015
    Posts:
    11
    Has anyone tried this already on GitLab version 13.9? We're still on 13.4 and would like to get someone to update our server, but only if that actually helps solve this issue. From the docs it's not clear if this is actually implemented yet.
     
  29. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    411
    It delay to 13.10 (2021-03-22)。I am waiting it release and test directly on the gitlab.com.
     
    frank-ijsfontein likes this.
  30. IroxGames

    IroxGames

    Joined:
    Feb 8, 2020
    Posts:
    5
    HOW TO GITLAB PACKAGES

    TO PUBLISH A PACKAGE

    Create a .npmrc file in your package folder with this content:

    registry=https://gitlab.com/api/v4/packages/npm/
    //gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/:_authToken=YOURDEPLOYTOKEN


    In your package.json add this:

    "publishConfig": {
    "registry": "https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/"
    }


    you can find the project id beneath your project name at the gitlab project page

    TO DOWNLOAD THE PACKAGE

    Create a File at Users/Your Username/.upmconfig.toml
    [npmAuth."https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/"]
    _authToken = "YOURAUTHTOKEN"
    alwaysAuth = true


    Unity package manifest
    "scopedRegistries": [
    {
    "name": "PACKAGE NAME",
    "url": "https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/",
    "scopes": [
    "com.YOURCOMPANYNAME"
    ]
    }
    ],


    Sry for the caps sections, seems like it's not possible to format this bold.
    insert your data in the caps lock fields

    Not supported right now
    Discovery feature in the Unity Package Manager
     
    frank-ijsfontein and watsonsong like this.
unityunity