Setup for scoped registries (private registries)

Hello @rizu,

Sure. Here it is!

Introduction
The concept of scopes is borrowed from npm (https://docs.npmjs.com/misc/scope), it allows a subset of packages to be hosted on a registry other than the default.

Configuration (manifest.json)
Project manifests (manifest.json) can now contain any number of scoped registry configurations in the scopedRegistries attribute. This new configuration does not supersede the registry attribute or the UPM_REGISTRY environment variable; those will still be used as the default when a package matches none of the configured scopes.

The scopes attribute of each configuration is an array of strings that can be mapped to a package name in two ways:

1. As a namespace. This type of configuration assumes that packages follow the Reverse Domain Name Notation (https://en.wikipedia.org/wiki/Reverse_domain_name_notation). In other words, a scope such as com.unity is the equivalent of com.unity.*
2. As an exact package name match.

When deciding in which registry a given package should be searched, Package Manager will settle on the scope that is the closest match to the requested package name.

The snippet below demonstrates how scope selection works:

Code (JavaScript):

1. {
2.   "scopedRegistries": [
3.     {
4.       "name": "Main",
5.       "url": "https://my.company.com/registry",
6.       "scopes": [
7.         "com.my-company", "com.my-company.tools.packageX"
8.       ]
9.     },
10.     {
11.       "name": "Tools",
12.       "url": "https://my.company.com/tools-registry",
13.       "scopes": [
14.         "com.my-company.tools"
15.       ]
16.     }
17.   ],
18.   "dependencies": {
19.     "com.unity.cinemachine": "1.0.0",
20.     "com.unity.2d.common": "1.0.0",
21.     "com.unity.2d.animation": "1.0.0",
22.     "com.my-company.package1": "1.0.0"
23.   }
24. }

With the above configuration:

1. com.my-company.packageA

   . will be fetched from Main because it is in the com.my-company.* namespace.

2. com.other-namespace.packageX

   will be fetched from Main because its name is an exact match. It also matches com.my-company.tools.* (Tools) but that is less specific.

3. com.my-company.tools.animation

   will be fetched from Tools because it is in the com.my-company.tools.* namespace. It also matches com.my-company.* (Main) but that is less specific.

4. com.other-company.packageA

   will be fetched from the default registry because it matches none of the configured scopes.

Why "scoped" registries?
It's to ensure determinism when deciding from which registry a package should be fetched. By using scopes, a package will always be mapped to one and only one registry, guaranteeing a consistent result regardless of network conditions.

Supported Registries type
Unity Package Manager supports npm protocol based registries. You can use any off the shelves npm registries server and it should work.

These are the registries that we know should work

• Verdaccio Probably the easiest server to configure

Search all package limitation

Some npm registry server does not support the

/all 

route to search all packages (e.g. Artifactory). Displaying scoped registry packages as part of the "All packages" list relies on scoped registries to support the old npm API protocol, which has an HTTP endpoint that returns the metadata of all published packages (eg: https://registry.my-company.com/-/all). When a registry does not support the old protocol, packages from that registry will simply not be displayed in the UI. However, this limitation does not apply to package resolution, so packages from scoped registries can still be manually added to the project manifest.

Limitations

• In the Package Manager window, the "All packages" tab will now display and allow you to install packages found in scoped registries. But, they will be mixed with the default Unity registry. There is no differentiation for now. In future releases of the UI, the source and the scope of each package will be made clearly visible.

Thanks for trying out this feature. Let us know if you have any issues. Look for a production quality release in 2019.1.

Regards,

Pascal

 

Hi @M_R ,

The Package Manager does not support authentication for now. Your repository needs to be public. One thing you can do is limit access to people inside your LAN.

If you have a use case that is common, we could look into adding this feature. Can you expand a little bit more on why you would need auth on a local/private registry?

Regards,

Pascal

 

Thank you @M_R for the feedback! Will transfer your request to our product manager to validate priority.

Regards,

Pascal

 

Hi @Adrian ,

I took note of your suggestion.

Thanks,

Pascal

 

@M_R I just wanted to give you a follow-up on this feature request. We analyzed it, and it will require a bit of work to provide a good user experience (persist credentials, env vars, login window, etc.). The complexity is a bit greater than I would have anticipated. I hope we can implement this feature sometime in 2019. We have a couple of higher priority issues to address in our backlog before attacking this one. As for your request to support Unity auth on scoped registries, it is not something we have in our roadmap. Sorry.

@Adrian Even though Package Manager is compatible with *npm* web API, it is not the same technology underneath (e.g. we don't share the same configuration scheme for scoped registries)

Regards,

Pascal

 

Hi @dzamani ,

Thank you for your suggestion. We have a well-defined plan to implement this feature. This is only a matter of prioritization. If you have an enterprise support account, you can contact your assigned account manager to express the value for your company to make this feature available sooner.

Regards,

Pascal

 

@nhold Can you expand a bit more on your setup? Do you have examples? Or a public repository that you can share?

Thank you!

Pascal

 

Interesting. We have no plan to support such a feature. I just don't want to give you false hope. But will see. If hosting out of the shelve npm registries turns out to be too hard/complex for people, we could consider this alternative approach.

Btw, were you aware that the Unity Package Manager supports Git URLs project dependencies? https://forum.unity.com/threads/git-support-on-package-manager.573673/

Thanks a lot for sharing. This gives me another perspective

Regards,

Pascal

 

Hi @dtaTrifork ,

The package manager conflict resolution algorithm will always choose the highest version number requested unless the package is a root dependency (i.e. you explicitly added the package to your project). We are working on documenting the conflict resolution algorithm. Coming soon.

The Package Manager UI does not show transitive dependencies. This is something we have on our roadmap.

Regards,

Pascal

 

Hi @giacomohomunculus,

I'm sorry. The answer is not yet. This feature was recently requested https://forum.unity.com/threads/https-communications-options.624787/#post-4184713. We set the priority for this feature very high.

Regards,

Pascal

 

Hi @astorms,

We will support authentication in the future although it will most likely not be based on npmrc. Unfortunately, I can't give you an ETA at this point.

Regards,
Benoit

 

Hi @N3uRo,

First, sorry for taking this long to give you an answer!

For your scenario, your Package B will now refer Package A version 2.0.0. The project manifest is the highest priority in the version resolution algorithm. This enables users to fix conflict issues.

And for your version range question, I just responded to the same question here: https://forum.unity.com/threads/semantic-versioning-like-1-0-x-is-not-working.649774/

Regards,

Pascal

 

Hi @MartijnGG ,

It depends a lot on your setup. What VCS are you using? Git? If yes, you could temporarily clone the repo under the

Packages/

folder. You could create an OS symlink in the

Packages/

folder to point to the package development folder. You could use the local package feature (`file:`) as you stated.

There is a couple of ways.

Pascal

 

@MartijnGG Yes, packages under the

Packages/

folder (we call this feature embedded packages) override any version defined in

manifest.json

. So, the local copy is used instead.

Unity developers have different workflows. It depends on the team. Some have lots of packages (e.g. DOTS team) needing to be worked at the same time. Some have just one package. From my understanding, the local package feature (

file:

) is extensively used when in development.

Regards,

Pascal

 

Hi @msandrof ,

You made me realized that our post is wrong. We use Artifactory internally but not for scopedRegistries. As you noticed, Artifactory doesn't support the

/all

path. I have updated the post. Really sorry about this!

That said, scopedRegistries should still work. The

/all

path is used for listing packages. It means that you won't see your company packages in the All tab. But, it does not prevent packages to be manually added to the project. Admitting though this not the optimal UX!

As far as what URL to setup, it depends a lot on your infrastructure. You can check the *Set Me Up* page in the Artifactory repo dashboard. Mine looks like this:

https://<hostname>/api/npm/<registry-name>

.

Pascal

 

@msandrof

This was not in our plan and have a lot of higher priority features to implement before. I'll bump the *user requested this feature* count by 1 though and let our PM know

Oh, I think I know what the issue is. We've noticed lately that *npm* CLI v 6.x.x and up now set the modification date of all files inside the tarball to

Oct, 26th, 1985

(yes, Marthy Macfly return date! ). This is a problem for the Unity Asset Database. Since the modification date is the same when you update a package, the asset database thinks the content has not changed and does not re-import. We made a fix on our end but it will only available starting in 2019.1. Can you confirm that you are using *npm CLI 6.x.x* to publish packages? If so, can you use *npm 5.x.x CLI* for now?

Regards,

Pascal

 

@jan-bajana Unity Package Manager does not support npm scope registries. Unity Package Manager is not npm. We are re-using the same backend technology because we would have written a similar backend API anyway. Bonus, there was also tons of OSS implementation and providers options available for users to host their own registries.

See our scoped registries implementation documentation on the first post of this thread.

Regards,

Pascal

 

This feature is now documented in Unity User Manual. Please look up Scoped package registries for more details.

Pascal

 

Hi @Fabien-LG ,

You were using an unsupported/undocumented feature and have been affected by an internal refactoring. We use to rely on *npm* library internally. This is not the case anymore. Your alternative should work and continue to work since it is part of the HTTP standard

Regards,

Pascal

 

I'm sorry. This is not something that we can manage to ship in 2019.x timeline. Expect this feature in 2020.x release cycle

This is also planned for next year. Sorry.

Thank you! Will do. Hope the introduced delays in providing the expected features won't change your view

Regards,

Pascal