Official Npm registry authentication

awesome, thanks! hoping 2019.3.4f1 will land today

 

Great to see this land!

Have you considered adding project-based authentication, like the undocumented "_auth" property in the project manifest allowed in earlier Unity versions?

Use case: When collaborating on a Unity project with others, who are not necessarily technically versed, I would like to make the process of checking out the project as frictionless as possible. Including a repository token directly in the project's manifest, people with access to the Git repository would simply be able to check out and use the project, without having to worry about logging in to the repository and managing separate login credentials.

I'm not too worried about sharing the token with everyone in the project, Git authentication already limits access and the token can be invalidated/updated when needed.

Having a GUI in Unity for logging into the registry and/or adding support for scoped registries from GitHub/GitLab would alleviate some of the friction but project-level authentication would still make the setup easier.

This could be achieved with a configuration in the project's manifest or by having the package manager check for a configuration file inside the project folder first.

 

Hello,

I tried this out, but I am getting the following error:

I am able to do npm install com.domain.package.

I created a .upmconfig.toml file and set it up like so:

Code (CSharp):

1. [npmAuth."https://nexus3.domain.com/repository/repo/"]
2. token = "SAME_TOKEN_THATS_IN_NPMRC="
3. email = "email@domain.com"
4. alwaysAuth = true

I couldn't find any useful logs. Any help would be amazing, thanks!

 

I had the same issue as @toeadd31 with a Nexus repository. I tried Verdaccio with authentication, and curiously the version 1.0.0 of my package was downloaded fine. But version 1.0.1 wouldn't download. I restarted Unity, but still had same issue. Log message said it "failed because it lacks valid authentication credentials".

I wonder if Unity caches some authentication information. I placed a wrong authToken in the .upmconfig.toml file, and I'd get the same behaviour (even after restarting Unity).

It would be great if we had extra logs/information on the error. I was using Unity 2019.3.4f1.

 

You need to delete slash at the end of the registry URL

Code (CSharp):

1. [npmAuth."https://nexus3.domain.com/repository/repo"]

 

Hello,

I have tested a few ways, I tested both with and both without a slash. I also tested the NpmToken. style token and the base64 usernameassword style token.

Nothing seems to work for me. I keep getting the same error.

I would assume I am just missing something? This is pointing to a nexus3 private repo with a hosted npm format.

Thanks again!
Todd

 

Has anyone gotten this to work with Azure DevOp's NPM artifact stream? Tried with a slash at the end and without as mentioned above, tried also the PAT token base64 encoded and not. Each way has resulted in the lacks valid authentication credentials message.

 

Ok, I got it working. Yeah, you need to make sure the URL is the same (even though the slash does not seem to make a difference). I guess my issue was a stupid mistake on my side (upmconfig had a 'https' URL, manifest had 'http' URL). Also make sure you are grabbing the correct token from the .npmrc file, because in my case it contained a couple auth tokens for the same repo (but slightly different urls).

 

What does the package manager error `unable to get local issuer certificate` mean in this situation? I haven't been able to get this to work, and I'm not sure if it's because something is configured incorrectly, or if it is because Unity is not finding the config file. Is there any way to verify where exactly Unity is looking?

Looks like the specific error in upm.log is

Code (CSharp):

1. {
2.   "message": "unable to get local issuer certificate",
3.   "stack": "Error: unable to get local issuer certificate\n    at TLSSocket.onConnectSecure (_tls_wrap.js:1285:34)\n    at TLSSocket.emit (events.js:196:13)\n    at TLSSocket.EventEmitter.emit (domain.js:494:23)\n    at TLSSocket._finishInit (_tls_wrap.js:758:8)\n    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:590:12)",
4.   "code": "UNABLE_TO_GET_ISSUER_CERT_LOCALLY"
5. }
6.  

 

Ah, yep that was my issue, thanks!

 

Also having trouble getting this to work using an Azure DevOps NPM Artifacts Feed.
Have tried all of the same things as @z000z but also getting the "lacks valid authentication credentials" message.
Would be fantastic to find a solution to this

 

Hi @okcompute_unity.

Yes, just as @z000z has mentioned, I have tried:

- With and without a trailing slash in the registry url (double-checking to make sure they were identical in the .upmconfig.toml file and in the manifest file)
- Using the token generated via logging into the registry
- Generating a PAT token manually on my Azure DevOps account, Base64 encoding it and adding it to `.upmconfig.toml` file with the "_auth" property name (also tried using the "token" property)
- Tried using the PAT token again but without encoding it in Base64
- Added the .upmconfig.toml file to the %USERPROFILE% directory AND the %ALLUSERSPROFILE%Unity/config/ServiceAccounts directory

Still, having tried everything above, I get the "lacks valid authentication credentials" message

Any help is greatly appreciated!

 

Also tried the _auth suggestion, but still ran into the credentials problem. Using a Mac so have the file in the ~/ folder, but otherwise same as @Darzer

 

As an update to this, I have installed Verdaccio locally and set my Azure DevOps NPM Artifact feed as an Uplink.
In the config file where that Uplink is defined, I added my bearer token (the one generated previously) inline.
I then setup a proxy for package requests starting with my feed namespace (like com.XYZ.*) to that Uplinked feed.

Having done the above, I simply changed my manifest.json file to point to Verdaccio on localhost and it managed to resolve my package!

Considering that Verdaccio is essentially "wrapping" around my hosted NPM feed as a proxy, I am surprised that it works, whereas attempting to go directly to my hosted feed fails. And seeing as the bearer token I'm using is the very same one I've defined in my .upmconfig.toml file I am perplexed.

Perhaps my .upmconfig.toml file is not even being found by Unity? I say this because it doesn't seem to matter what the contents of it are, I always get the "lacks valid authentication credentials" message (even if I delete the contents of .upmconfig.toml completely).

Hopefully this helps you @Ok_Computer with any debugging you may be doing.
It would be great to be able to resolve packages directly through my Azure DevOps feed but this workaround is perfect for the time being

 

Ok, followed given instructions and have resolved _that_ issue, but now I'm getting the same "lacks valid authentication credentials" message that everybody else seems to be getting. Verified that the upmconfig.toml is indeed being read correctly, but seems something else is causing the problem. Tried both regular token and base64 encoded token, neither work.

 

This is fantastic work @okcompute_unity, thank you so much for your efforts!
I will give this a go later tonight and report back here with my (hopeful) success

 

@okcompute_unity Just tested this out and that fixed it for me, thanks so much!

 

@okcompute_unity ... I am trying to get the package manager working with nexus and I am having a lot of trouble getting it to authorize using my username and password. I have tried several things from this forum to no success. This is what is printed to my console:

An error occurred while resolving packages:
Project has invalid dependencies:
com.zack.zackpack: Request 'http://localhost:8081/repository/zack-npm/com.zack.zackpack' failed with status code [401]

A re-import of the project may be required to fix the issue or a manual modification of D:/Work/Unversioned/AsyncTest/Packages/manifest.json file.

When I view the nexus interface, I see the package just fine. I should also mention that if I create a brand new node project and run `npm install --save com.zack.zackpack --registry=http://localhost:8081/repository/zack-npm` it works perfectly.

Any help would be greatly appreciated, as my company has decided that this is the way we want to deliver our SDK to an internal partner

 

I appreciate the timely response! I send you a private conversation

 

To sum up the findings on my answer. Turns out that this behavior was added in 2019.3.4f1 of the editor. If it just won't work for you, this may be the reason why

 

I've sadly not been as fortunate as @z000z and still cannot get this to work using an Azure DevOps feed
I'm confident that I've followed your steps meticulously. Just to be sure, I did the entire process three times (each time with a newly-generated PAT).

The only part I'm not confident about is the value to use for "username" when encoding the PAT in the colon-separated format. I am using the exact value supplied by Azure DevOps in step#1 of the image you added @okcompute_unity. Is this the right username? I've even tried again but lower-casing the value.

I've stripped out the trailing slash on the npmAuth url (and also tried with and without the slash in the scopedRegistries url) just as you said but still no luck

Considering @z000z got it to work, it's highly likely that I've gotten something wrong but I just don't know what it could be.
Did you have to do anything other than what @okcompute_unity said @z000z?

Edit: For clarity, I have tried this in Unity 2019.3.4f1, 2019.3.5f1 and 2020.1.0b1.

 

@Darzer The only thing that I can think of that might be different is I added awhile back a .npmrc file to the project's packages folder. Probably not required though, it just has the registry and always-auth=true like the Microsoft documentation mentions for setting up a package.

Other than that I removed the slash at the end of url in the .upmconfig.toml file, the .npmrc file in the packages folder, and from the ScopedRegisteries entry in the manifest.json. And switched to using base64 encoded user:token for the _auth line in the .upmconfig.toml file.

 

Hi @Darzer
I am trying to connect my Verdaccio instance to Azure Devops as well.
Could you post your Verdaccio config? I keep getting 401's from Azure Devops.
For the token are you putting the PAT token in directly or Base64 encoding it?
Any help would be greatly appreciated.

Thanks

 

@Darzer Did you ever get this working?
I seem to be stuck at the same place you are.

 

@Pelsepils, I was able to get the Azure DevOps npm registry working with package manager by base64 encoding

Code (CSharp):

1. "email:pat"

where 'pat' is a personal access token generated with packing read/write. I didn't use my account password.

 

@okcompute_unity I am guessing writing files to the user home dir from a build hook during a Unity Cloud build is not permitted. Are there any other means of passing credentials in a cloud build scenario?

 

Thanks,
The correct base64 encoding is email : pat
I actually think I must have gotten it to work before, its just that my packages are not showing in the Package Manager.
Not even now they show up, but if I enter the package and version directly in the manifest.json in the dependencies section, they will automatically load. And whats more, they become visible in the Package Manager, and I'm able to update to newer, or older, versions of the package.

 

Yes, that's expected as it seems Unity requires the registry to support the /all path to query all packages. ADO doesn't seem to support that though so you need to explicitly add your package to the manifest.json.

On a good note it seems once your package has been added, Unity will at least detect newer versions.

 

okcompute_unity more general question, but is it safe to assume the ability to use private/local or even public custom npm registries will continue to work with Unity into the future, or at least 2020/2021 versions? i.e., will Unity continue to provide the ability to supply custom registry URLs in the manifest.json? (or even add some UI for this in the package manager itself? ) And the package manager will continue to use npm registry queries to discover and install packages?

We're considering hosting our own public npm registry to provide a plugin/SDK but want to make sure we're not tying ourselves to an implementation detail of UPM that may change in the not too distant future! Thanks

 

I've created a simple GUI to maintain the credentials and scoped registries from inside the Unity Editor UI. This makes support within my company much easier than setting up config files.

https://github.com/Halodi/halodi-unity-package-registry-manager

I open sourced it because it might be useful for others as well.

 

Very good work, thank you!

 

This is how I got it working, it's a combination of some of the above...

I suspect the .npmrc file is not required to get the Package Manager working as the way we were trying it (vsts-npm-auth) would not give me an "authToken", it gave me a "_password" instead which was different.

--

1. Ensure there is a ".upmconfig.toml" file, for me I had it in my root user directory, it should match this:

Code (CSharp):

1. [npmAuth."https://your_registry_url_with_no_trailing_slash"]
2. _auth = "this_will_be_replaced"
3. email = "your_email_address_that_matches_azure_login"
4. alwaysAuth = true

2. Copy the Personal Access Token from the dev.azure.com login for your group (you may need to click Add or Regenerate if one already exists)

3. In command line, run the following command:

Code (CSharp):

1. node -e "require('readline').createInterface({input:process.stdin,output:process.stdout,historySize:0}).question('PAT> ',p => {b64=Buffer.from(p.trim()).toString('base64');console.log(b64);process.exit();})"

4. When prompted to enter your PAT, I entered

email:pat

obviously with the email and PAT replaced (email should probably match your dev.azure.com account and also the email specified in the .upmconfig.toml file)

5. Command line will give you a new Base 64 encoded string, copy that and paste into "_auth " in the upmconfig.toml file


I also removed the trailing slash inside manifest.json in Packages folder of the Unity project, unsure if required though.

 

I separated my big project into smaller chunks, each of them on gitlab and on a Verdaccio instance both running on my server, and I am trying since a week to get gitlab CI to test the repositories using a shell runner on my own computer.

The runner inserts the package dependencies into a copy of a default manifest.json to make unity import the package's dependencies on top of the default ones before starting the tests, I created the required .upmconfig.toml file, added my authentication token and I can see and import my packages inside Unity without problem, the runner executes itself properly...

But whenever the runner tries to import any of my scoped registry' package, the editor ran by the runner logs: `[...] failed because it lacks valid authentication credentials`.

Maybe I'm doing something wrong but is there any way to let unity know about the token other than via the .toml file? via command line argument maybe?
Is such feature planned for the Package Manager?

 

Use openupm-cli login command may simplify the process before the official login UI happens in Unity Hub. See more in How to Authenticate with a UPM Scoped Registry using CLI

 

Thanks for the heads up!

I will take a closer look to that, I have seen actually people implementing very good UI plugins already to make it simpler in Unity to manage registries, but my problem is that on my Unity editor there is no problem, the authentication does occur normally and I see all my packages...

It's only when the test runner starts its batchmode instance of Unity through powershell that the authentication fails.
And its only the case for the runner, because I tried to execute the very same script myself in a powershell I opened myself in the very same directory, and it worked.

Now I think the problem must come from the runner or its powershell and not from Unity :|

 

Thanks a lot for releasing this outside of the regular release cycle. We are using google artefact repository in our org. Unfortunately google cloud only provides 60min valid oauth tokens via a shell command, so we have to refresh those constantly while unity is running.

I'm using the following bash code to run it on UCB:

Code (CSharp):

1. #!/bin/bash
2.  
3. set -ex
4.  
5. CI_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
6.  
7. # create temporary copy of npm rc
8. cp $CI_DIR/npmrc $CI_DIR/npmrc.tmp
9.  
10. # let google cloud fill in password
11. export GOOGLE_APPLICATION_CREDENTIALS=$CI_DIR/gcp-sa-ucb.json
12. npx google-artifactregistry-auth $CI_DIR/npmrc.tmp
13.  
14. # extract password and base64 decode
15. ENCODED_TOKEN=$(grep -o '_password.*' "$CI_DIR/npmrc.tmp" | sed -e 's/.*_password="\(.*\)"/\1/')
16. DECODED_TOKEN=$(echo "$ENCODED_TOKEN" | base64 -D)
17.  
18. # override upmconfig
19. sed "s/\$TOKEN/$DECODED_TOKEN/g" $CI_DIR/upmconfig.toml >~/.upmconfig.toml
20.  
21. # cleanup
22. rm $CI_DIR/npmrc.tmp

 

After many attempts, i add my private verdaccio package server (with authentication) to Unity Cloud Build.

Step by step for those who want to do:

1) Create the

scripts/pre_build.sh

file in you root folder with contents:

Code (csharp):

1. echo "$UPM_TOM_CONTENT" > $HOME/.upmconfig.toml

Notes

HOME

- User path in cloud build. (built in)

UPM_TOM_CONTENT

- Content of your .upmconfig.toml (added by you)

2) Go to Advanced Options in you Cloud Build target settings and choice Advanced Options to insert

scripts/pre_build.sh

in Pre-Build Script Path field, next Click in Save Button.

3) Go back to your target settings, and open Environment Variables to add

UPM_TOM_CONTENT

's:



VERY IMPORTANT: You need to replace the break spaces by

\n

, I had syntax problems saving raw .toml data

Bad Sample

Code (csharp):

1.  
2. [npmAuth."<REGISTRY URL>"]
3. token = "<AUTH TOKEN  _authToken in .npmrc)>"
4. alwaysAuth = true
5.  

Good Sample

Code (csharp):

1.  
2. [npmAuth."<REGISTRY URL>"<AUTH TOKEN  _authToken in .npmrc)>"\nalwaysAuth = true
3.  

If all goes well, when you make a new build, your private packages will be found

 

Hi, we're currently facing a problem that we can use the Azure DevOps npm registry, but it is seemingly impossible to look up existing packages in it. This might be a problem on Microsofts side (as it seems like the API does, in fact, not support listing packages) or another authentication problem.

Can you list the registry? If so, how?

 

Same here also willing to use Azure Devops Artifact Feed only because the scopedRegistry isn't listing packages it isnt very useful.
Manually adding the packages to the manifest defeats the whole purpose of the package manager? If I do add it manually the package is found which is great but users need to be able to see the packages.
Any prospects on when package listing (the new way?) will be supported?

 

I'm facing an interesting issue:

I have a verdaccio server running with several packages without authentication, but limited by IP address.
Now I want to add authentication to the system, so what I did was to deploy a new verdaccio server with the exact same configurations as the previous one, but this time using authentication.

Then I copied all packages from the old server to the new server.
After following some tips from here I managed to list all packages in UPM with authentication working and all.

All was fine until I noticed that Samples from some packages where missing the ~ character.
This didn't happen with the previous server, and if I download the zipfiles of the packages from the new server, they contain the ~ character.

This makes me think that somehow, UPM messes up the ~ character when unzipping the package that was downloaded from a private registry under authentication. Any clues?

I also posted here: https://forum.unity.com/threads/empty-samples.998878/
I believe we are facing the same issue