Search Unity

npmjs as a scoped registry

Discussion in 'Package Manager' started by starikcetin, Sep 19, 2019.

  1. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    Is it possible to use https://www.npmjs.com/ as a scoped registry? If so, where can I find some information on the setup required?
     
    ModLunar likes this.
  2. samuelb_unity

    samuelb_unity

    Unity Technologies

    Joined:
    Nov 13, 2017
    Posts:
    151
    ModLunar and starikcetin like this.
  3. samuelb_unity

    samuelb_unity

    Unity Technologies

    Joined:
    Nov 13, 2017
    Posts:
    151
    Hi @starikcetin,

    I reached out to my team about whether we have a stance on using npmjs to host Unity Package Manager (UPM) packages. While it is technically feasible (I can add a scoped registry with
    "url": "http://registry.npmjs.com/"
    to my Unity project manifest and add packages to my project) you might find this goes against npmjs's policies on acceptable use:

    https://www.npmjs.com/policies/open-source-terms#acceptable-use

    For example
    Currently a UPM package is functionally compatible with the npm command-line client but it may not always be the case as we continue to add our own features. And the fact that a UPM package does not work in a regular Javascript project may go against npmjs's policy of what a "package" can be. So for now our advice would be: do so at your own risk!
     
    Last edited: Sep 20, 2019
    ModLunar, alfish and starikcetin like this.
  4. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    I appreciate you taking your time to investigate this. Thank you very much.
     
    ModLunar likes this.
  5. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    I sent a support ticket to npmjs guys asking about the acceptable use policy regarding UPM, I will let you guys know about their response.
     
    ModLunar and samuelb_unity like this.
  6. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    1,066
    Also note that npm's scoping and Unity's scoping are fundamentally incompatible. npm uses the «@scope/name» format while unity uses the reverse DNS notation, e.g. «com.unity.render-pipelines.high-definition».
    • Unity does not allow the @ or / characters in package names, you therefore cannot use npm scopes in Unity
    • npmjs.com doesn't allow dots in package names, you therefore cannot use Unity scopes on npmjs.com
    • npmjs.com also requires scopes for any non-public package
    This is unfortunately also the reason why the GitHub Package Registry is incompatible with Unity.
     
  7. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    I got a response:
    It seems it is not against npmjs policies to use it for UPM.
     
    Last edited: Oct 1, 2019
  8. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    Hi @Adrian,

    We haven't seen that package names cannot contain dots on npm registries. Perhaps this is a policy specific to the npmjs.org registry, but the rule that they describe in the documentation is that a package cannot *start* with a dot, which is something we also enforce, therefore it should be possible to publish to npmjs.org a Unity package. However, it is true that these packages cannot be made private on that registry because npm scopes are needed for that, which Unity doesn't support - and, quite accurately, this is the reason why GitHub Package Registry cannot be used with Unity packages at this time.
     
    ModLunar, alfish and starikcetin like this.
  9. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    I had published an upm package on npm, and it can be consumed by the upm system with no issue.

    Code (JavaScript):
    1. "scopedRegistries": [                    
    2.   {                                      
    3.     "name": "npmjs",                      
    4.     "url": "https://registry.npmjs.org/",
    5.     "scopes": [                          
    6.       "com.littlebigfun"                  
    7.     ]                                    
    8.   }                                      
    9. ],                                          
    @maximeb_unity may I ask for what reason Unity is not playing well with @Scope semantic? I understand the reverse dns notation is already a decision, but it doesn't seems conflict with "@" keyword. In npm, @Username/pkg-xyz and pkg-xyz are two different packages and have no fallback logic to each other.

    Code (JavaScript):
    1. com.littlebigfun.addressable-import
    2. @favoyang/com.littlebigfun.addressable-import
    For scoped registry, we could add "@favoyang/com.littlebigfun" as scopes to tell the system where to locate the package. Or even better add "@favoyang" as scopes could also work.

    Code (JavaScript):
    1. "scopedRegistries": [                      
    2.   {                                        
    3.     "name": "npmjs",                      
    4.     "url": "https://registry.github.com/favoyang",  
    5.     "scopes": [                            
    6.       "@favoyang/com.littlebigfun"                  
    7.     ]                                      
    8.   }                                        
    9. ],    
     
    starikcetin likes this.
  10. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    1,066
    Oh, good to hear it actually works. I was going by the npmjs.com documentation on creating a package.json file, which states regarding the name:
     
  11. ModLunar

    ModLunar

    Joined:
    Oct 16, 2016
    Posts:
    374
    This is a bit late, but I think this might help someone:

    Over a weekend recently, I went into into extensive depth into setting up a custom scoped registry for Unity packages of my own.

    I learned why https://npmjs.com (https://registry.npmjs.org) is NOT supported for hosting Unity packages that feed in the Unity Package Manager: It seems to just be because of different naming conventions.

    Unity packages by convention should use reverse domain notation (com.unity.mathematics for example, where the scope would be com.unity).
    However, for NPM packages, they must use an @ symbol to designate the scope, followed by a / symbol (which would be @com.unity/mathematics for example).
    Because NPM and Unity Package Manager make these assumptions that MUST be true in order for it to work... they don't work together.

    However, there's a free open source NPM package registry that DOES support Unity's naming conventions, called Verdaccio!
    I got it running with Docker Compose (thanks to their Getting Started pages (see ~7min into the video for a docker-compose.yml example)) and I deployed it on a very small AWS EC2 instance (about $6 a month) and now can put all my packages in a place that can make my own code sharing way easier.
    I love using it with Docker cause you can deploy the same thing locally if you want (being that Docker makes it easy to run the same thing predictably locally, on the cloud, etc.)

    ---

    I'm not sure if there's a workaround I didn't know about with the naming conventions to get NPMJS to work with Unity in any kind of fashion, but I opted for a more complete solution rather than a workaround! But please do correct me if I missed something along my brief research :)
     
  12. alfish

    alfish

    Joined:
    Apr 5, 2017
    Posts:
    26
    For any of you that are unaware, you can publish your open-source Unity packages on upm-packages.dev, which uses Verdaccio. That's where I published com.unity_x.modules.sceneref (a package that allows referencing scene assets at runtime). It works like npmjs (you can use npm publish after setting up that registry).
    I know that there was another site too, but it only works with GitHub repos and I forgot the name.
     
    ModLunar likes this.
  13. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    There is also OpenUPM developed by @Favo-Yang which is a godsent.
     
    ModLunar likes this.