Official Using GitHub Packages Registry with Unity Package Manager

This is sad. We should have option to use

```
"publishConfig": { "registry": "https://npm.pkg.github.com" },

// with

"name": "USER/com.my-company.my-package", // without @
```

Instead. This would be easier to orchestrate cross domain dependency. I may publish project that utilize project of other person or other organization easily. It would critically expand the usage of package manager over everything in one go with just one line of registry

 

@Thaina that's down to how githubs package registry works rather than how unitys package manager works unfortunately. So far my experience with GitHub Packages has been pretty underwhelming since they really don't make it easy for you to use. Fingers crossed for the future though!

 

Why this? This excludes GitLab for use since GitLab requires scopes to be present and rejects packages without scopes.
Seems that your suggested cheat does not work with Gitlab. And cheating should never be necessary (strictly against cheating, makes life harder for all of us).

'Packages must be scoped in the root namespace of the project.'
from https://docs.gitlab.com/ee/user/packages/npm_registry/#package-naming-convention

 

I would chalk this up to githubs package registry forcing our hand in cheating in this way. As far as I can tell, unitys package manager isn't the bottle neck in how packages are read and pulled in.

 

Thanks for this guide. I tested it and it seems to work. We have the need to share with outside resources and this will help, although a pain to setup.

We are using some custom tooling to manage/publish NPM packages, and this could be added to it.

Is there a future where you think packages with the naming scheme @Scope/package-name will be supported? It seems that this is enforced by multiple NPM repository hosts. An option could be to ignore @Scope/ in the name except when pulling from the NPM repo, so the full package name will become "@vendor/tld.vendor.package-name"

 

Fair enough. I'll see if we can make some extra tooling on our side to make it work well enough for our purposes.

 

Unity would not need to support Github natively. We would even be capable of developing an extension on our own to discover packages on any registry which makes this somehow possible via an API. I don't belive GitHub will go implementing UPM registries format.

Instead we are asking to support the NPM Scoping syntax which we will not be able to support via an extension on our own (except of hacking around it which would end up in writing our own package manager)
This is because NPM registries like the one of GitLab (not GitHub...) requires a scope for packages and this is why unity editor can not import those packages from GitLab.

(Maybe there is a better thread for this or there should be one)

 

I think the issue been discussed a few times. Supporting /-/all/ discovery endpoint is a better-to-have thing. What really blocks this is the scopeRegistry syntax rejects "@" letter. Unity decides to use dot notation as the scope, but other NPM based registry defaults to @Username syntax. It didn't sound like a tech limitation but a design decision that whether @Username conflicts with dot notation. Maybe there're other reasons behind, but for me, below are all valid scopes.

• com.company
  
• @Username
• @Username/com.company

Sure, there're other registries that won't force you to use @Username as scope for the private package. But it's kinda convention here for NPM, GitHub and GitLab. They're all big players on the market.

 

Using 2019.3.7f1 we have been unable to replicate this behavior. UPM fails to pull in the manually specified package due to an inability to find it. Using npm install with the same token pointed towards the same URL works perfectly.

This forum thread is only a little less than 2 months old, so it seems a bit strange that the behavior would already have diverged.

 

This workaround seems to have been broken by a recent change by Github:

https://github.community/t/unable-t...t-name-to-publish-url-as-of-07-10-2020/136643

They no longer allow publishing of packages without the scope in the name.

@okcompute_unity - do you know of any way that we can work around this and continue to use the Github Package Registry with Unity 2019.4?

Alternatively, can you provide any suggestions for an alternative private package manager, that does not require us to host our own package manager (i.e. we are not really interested in self hosting verdaccio)?

Best regards,
Morten

 

@okcompute_unity
Excellent that it works again.

What is the reason that the Unity Package Manager doesn't allow package names beginning with an `@` scope?

 

@okcompute_unity Thank you for the guide ! It works, I succeed to publish my package on GitHub and download it on a project with the scoped registry.

There is just one issue on the Package Manager. As you said, GitHub doesn't support for searching, but the Package Manager will try anyway and it "crash" it a bit.



I'm on the "All packages" section, but once it failed to search on GitHub, it won't load any others packages ... It could be nice to just display a warning or fail silently.

For information, i'm on 2019.4.5f1

 

I've gotten as far as steps 4/5/6 - not sure which bit is failing. Package is hosted on a private GitHub organization repo.

Unity gives me.

I've got the .upmconfig.toml setup in Users/MyLogin/ with

The project manifest.json is like so:

 

Has anyone tried to have package dependencies and if you need to do the following with the scope @Owner/com.test.dependency ?

 

Will the Unity Package Manager support npm’s scope notation in future versions? We use GitLab, and it can host npm packages, but only when using exactly that ("@Scope/package.name") naming convention. So after getting my package published on our GitLab server, I found out that all my work was for nothing because the UPM is not fully npm compatible.
Should have read this thread first I guess. But seeing that GitHub also tried moving into the scoped naming requirement, are you going to fix this?

 

In our view, this is not something that needs to be fixed, but a new feature request that we don't plan to implement at this time. We don't plan to support npm's scope notation in the package names, a package with the name `@foo/bar` is not valid.

 

I think the problem is not the scope notation supporting, but the request to support gitlab or github registry.

 

Agreed. We want to use Gitlab / Github repos as we don't need to run or host separate servers for it. That seems like a reasonable request, especially as these are the two largest code hosting services used today. If they use `@foo/bar` for package names then maybe Unity should look into supporting that nomenclature.

 

I agree, should be supported. I use GitHub for personnal stuff and at work we use a self-hosted GitLab. Would save us tons of time.

 

If there is no official solution, can add an extension point that let's the community support the gitlab?

 

It seems the GitLab willing to resolve this problem in 13.9:
https://about.gitlab.com/upcoming-releases/
https://gitlab.com/gitlab-org/gitlab/-/issues/33685

 

Apparently with the caveat that it only applies to the project-level npm endpoint. Meaning you'll have to add a different scoped registry in Unity for every GitLab project you want to pull a package from, the instance-level npm endpoint will still require scopes as before.

 

I see an article in GitLab about `Store all of your packages in one GitLab project`:
https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html

I think it worth to try it in the new GitLab version.

 

Has anyone tried this already on GitLab version 13.9? We're still on 13.4 and would like to get someone to update our server, but only if that actually helps solve this issue. From the docs it's not clear if this is actually implemented yet.

 

It delay to 13.10 (2021-03-22)。I am waiting it release and test directly on the gitlab.com.

 

HOW TO GITLAB PACKAGES

TO PUBLISH A PACKAGE
Create a .npmrc file in your package folder with this content:

registry=https://gitlab.com/api/v4/packages/npm/

//gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/:_authToken=YOURDEPLOYTOKEN

In your package.json add this:

"publishConfig": {

    "registry": "https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/"

}

you can find the project id beneath your project name at the gitlab project page

TO DOWNLOAD THE PACKAGE

Create a File at Users/Your Username/.upmconfig.toml

[npmAuth."https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/"]

_authToken = "YOURAUTHTOKEN"

alwaysAuth = true

Unity package manifest

"scopedRegistries": [

    {

      "name": "PACKAGE NAME",

      "url": "https://gitlab.com/api/v4/projects/YOURPROJECTID/packages/npm/",

      "scopes": [

        "com.YOURCOMPANYNAME"

      ]

    }

  ],

Sry for the caps sections, seems like it's not possible to format this bold.
insert your data in the caps lock fields

Not supported right now
Discovery feature in the Unity Package Manager

 

I can confirm that this solution works with Gitlab!
However it would be great to also support discovery, since the Package Manager UI won't list all the available packages from the Gitlab registry.

As some people noted before, it would be really amazing if Unity cared to integrate correctly with either GitHub and/or Gitlab registries. These are two very popular code platforms that offer amazing integration and automation with the rest of the repository, making it as easy as merging to a master branch to automatically create a new release version with automatic description as shown here: https://docs.gitlab.com/ee/ci/examples/semantic-release.html

The correct way to do it right now to have full Unity support requires us to host a npm server ourselves, with all the infrastructure and maintenance overhead that implies for small or lone developers. Maybe the Unity team is not aware of how important this can be to small developers' workflows.

 

Another issue I'm noticing with this approach, is that for some reason the Package Manager UI is not showing me the package names or descriptions, even though they are configured in the package.jsons.

Edit:

I just gave a try to AWS CodeArtifact (https://aws.amazon.com/codeartifact/), which allows to host a private npm registry for free if using < 2GB/month bandwidth.
It's working better than the GItLab private repository. It shows me correctly the package title, description and author in the Package Manager UI. The discoverability doesn't work though, so it only lists installed packages from the registry.

 

Still having an issue with GitHub, their hotfix with allowing not using scopes if the name contains dots, seems to be removed.

Trying to publish a package with the name “com.myCompany.core”.
Getting error “405 Method Not Allowed - PUT https://npm.pkg.github.com/myName/com.myCompany.core”

If I change a name of a package to “@myName/com.myCompany.core” - it’s publishing like a charm. But Unity 2021 (as all the previous) can’t work with scope.

Is there any workaround to finally proceed with the GitHub registry?

 

@korsour we're using GitHub packages, I wrote a blog on my personal account on how I got them working https://forum.unity.com/threads/npm-registry-authentication.836308/page-2#post-6564022

Our packages use com.company.packagename - I think maybe you need "directory" : "com.company.packagename" in your package.json? Have a read of the blog post, see if there's some relevant info for you there.

 

Been following that, thanks for posting it! I'm getting the 405 error though following the example:

I'm able to publish the package to GitHub, so I know the bearer token is functioning. Furthermore when copying the get request in Unity Editor and trying it in Postman you get the same result, but making a get request for the package directly:

You get the correct response, returning the package.json file for it.

Thoughts?

 

The last section of the blog is about those errors, I ignore them. With those errors there, I have our github hosted packages in the package manager and refreshing for updates, which can be imported without issue.

To get them to show the first time I have to:
- add the scoped registery
- manually add the package in the manifest.json

 

I'm also a bit flabbergasted that the unity package manager does not support @Scope notation which has been the defacto standard way to annotate npm packages for quite a while now. It took me quite some time to realize that this was actually the issue as there is also no error presented by the package manager when you add an @Scope.

It would be really interesting to know what exactly the blockers are for this, as its being called by Unity Devs, "new feature". It appears to me that package resolution containing an @xyz in the beginning would be just as easy to regex/search for than reverse dns notation.

We are also using gitlab internally and not being able to use the instance level endpoint for packages is an issue as we now have to add a ton of scoped registries instead.

Edit: Just to clarify, this is not even about search functionality which as I understand needs a /all endpoint which Gitlab does not provide, but about basic package resolution when they are manually added to manifest.json or via the package manager add by name feature

Edit 2: My guess regarding the @ is that its already being used as denoting the package version in the PackageCache folder, but in that case you could just default to the last @ in order to fix that issue

 

Damn
Middle of 2022
*problem still exist*
Unity and Github: This is not our problem, we will not solve it

 

we were using this solution and it suddenly stoped working at the start of this week with no changes our side we are getting the:

 400 Bad Request - PUT https://npm.pkg.github.com/@MyOrg/com.myorg.mypackage - name in package.json "com.myorg.mypackage" does not match publish URL "@MyOrg/com.myorg.mypackage"

error again has anyone else noticed this

 

Yep. I've been getting it this week too. Was just about to post.

----
I recently started having issues publishing our packages, getting the error

Code (csharp):

1. npm ERR! 400 Bad Request - PUT https://npm.pkg.github.com/@myorg/com.bundle.package - name in package.json "com.bundle.package" does not match publish URL "@myorg/com.bundle.package"

I've figured out how to publish again:

the .npmrc file now has an additional line (the third one)

Code (csharp):

1. registry=https://npm.pkg.github.com/@myorg
2. //npm.pkg.github.com/:_authToken=<token>
3. @myorg:registry=https://npm.pkg.github.com/

And in each package.json I've had to change from

Code (csharp):

1. "name": "com.bundle.package"

to:

Code (csharp):

1. "name": "@myorg/com.bundle.package"

HOWEVER, this has stopped the package manager being able to see the new versions. I'm assuming this is because the name of the package is now "@myorg/com.bundle.package" and adding the @myorg to the packages in manifest.json doesn't work.

I'm stuck, I don't think there's a way around this now and npm packages on a private github can't be used anymore?

 

A solution that bypasses the package manager somewhat is to just have a folder next to your project that has a real package.json that references these as npm packages and then just reference the upm packages as local packages from the node_modules folder.

It seems that you cannot, or rather you cannot _anymore_ publish non-scoped packages to github.
https://github.com/actions/setup-node/issues/243#issuecomment-995742416

Which means that essentially you cannot use github packages for direct upm packages anymore as the unity package manager cannot work with scoped packages....

 

cheers thats going down as one of our backup plans.

 

whats odd is it didnt break on december last year when that issue is dated. it instead broke some time on the 1st of August this year

 

When you say the package manager cannot see new versions, does it also mean that if you manually set the new version in the manifest.json file, it does not work?
We have our own package manager UI for our custom scope anyways, so I do not care at all what unity's package manager can or cannot do, as long as it can actually download and import packages...

 

@okcompute_unity if the issue with Scoped package names really are the symbols `@` and `/` already meaning something to your parses, could you at least make use of some other symbols to signify scope? Like, wrapping the scope name in brackets or something: `<myscope>com.company.package`.

This way you greatly extend the usability of your package manager, and all of us that have built our infrastructure around Unity Package Manager and GitHub Packages, won't have to suddenly start hosting our own UPM repository.

Right now we are completely blocked from making code changes, and in my opinion, GitHub enforcing package scopes is not the problem here, it's Unity's lack of support for scoped packages.

 

We had the same problem that started last week, and our CI just started working again, I think Github may have rolled back the change again...

 

I can confirm GitHub rolled back this hard requirement indeed, and our CI started working again as well!
See you all again in 2 years when GitHub once more enforces this...

 

Has anyone got this working with LFS?

 

Wow Unity doesn't have enough time in 2 years to conditionally parse a name string? Don't they have an intern or two working there?

Like...

Code (CSharp):

1. if (packageName.StartsWith("@"))
2. {
3.   // treat as @scope syntax, terminated with a '/'
4. }
5. else
6. {
7.   // treat as .dot scope syntax like always
8. }
9.  

Also, who else thinks it's a little ironic and a bit of an oversight if Unity has "Scoped Registries" but doesn't support "Scoped Packages"? If Unity just supported scoped packages with `@Scope` notation, many many headaches would go away overnight.

 

Is there any update on this? This is a huge blocker for us. Using Verdaccio is an option, but adds lots of maintainability issue that could very easily be solved if we could use GitHub (or GitLab in our case) packages