Search Unity

Scoped registry server

Discussion in 'Package Manager' started by iPickle, Jun 27, 2019.

  1. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    31
    Hi
    We would like to create a scoped registry that links the packages from git urls (not host the files on the npm server)

    We cant seem to do this with traditional npm servers (PLEASE let me know if im wrong, we want the registry to act as an "index" to git repositories, and not manually add packages with git url)

    We dont mind implement it our selfs, but we dont want to write the whole npm protocol, so if we could find out what unity does under the hood to fetch the packages, we could adjust

    Thanks
     
  2. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    573
    Hi @iPickle,

    I'm not sure how could achieve what you want to do. Maybe it would be easier for you if you wait for us implementing Git packages dependencies support for transitive dependencies. This way, you won't even need a scoped registry. Everything would be hosted on your git repository. Albeit being a bit slow with our current implementation (i.e. cloning all git repos).

    Regards,

    Pascal
     
  3. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    31
    Hi!
    Thanks for the reply,

    I don't mind waiting, but will it be able to search out packages in the git?
    I would still like to show the list in the package manager, and not manually adding each git repo
     
  4. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    347
    The usual workflow is to have hook in your git repository that builds and uploads a new version to your npm server whenever you add a new tag or push to a branch.

    I don't think searching for git dependencies will be supported, since that would require supporting different git hosts and some way to distinguish between unity package repositories and others.
     
  5. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    31
    @Adrian Thats also something I was looking into as a backup plan, But I also couldn't find something useful, the only solution for this that i found is to setup my own server that receives the hook, download the repo, and then publish it

    Any info that i missed?
     
  6. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    347
    @iPickle That depends on your git server setup.

    If you have GitLab, which you can host by yourself, then you can use Gitlab Runners and gitlab-ci.yml. For Github you can use Github Actions (e.g. https://github.com/actions/npm). If you have only a simple git server, you can try using an external continuous integration service, though I don't know which work with private git servers.
     
  7. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    31
    @Adrian
    Thanks for the info, we have bitbucket server, I couldnt find anything that works

    But, (also for @okcompute_unity )
    I was able to sniff out whats going on inside the api of some npm registries, and it seems pretty easy to implement, so this is how far I got:

    I am able to see my package in the package manager,
    I am able to see all the package info on the package manager
    I am able to get a new version update to the package manager

    I am NOT able to download the package, here is why
    The specific part that unity use to download the file looks like this:


    "dist":
    {
    "integrity": "sha512-uHh/u6FhaOdM59l+DpVaHJSer/thBbO0D7UeoORB/ZJ/o+BeoWCt8Fhgdql5aPUBsd187smwTs/4pwqw1zJxBQ==",
    "shasum": "d377860aa78b9b570a85b8bef871267916d7b8ae",
    "tarball": "http://{Registry URL}/{Package ID}/-/{Package ID}-{Version}.tgz"
    }


    And what it does is, download the tarball file from the url, and then run some checksum checks on it before adding it to the project

    all that i need is to know if I can have something there that triggers git importing instead of tar file extraction

    (I am using 2019.3 so I can be up to date with the package manager version, if it matters)

    I would love to get some help from unity on that,
    So even an email address of someone that I can contact about it will be great

    Thanks!
     
  8. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    347
  9. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    573
    @iPickle , the Package Manager team's product manager is @DanielTG. You can reach out to him through a forum conversation.

    Regards,

    Pascal
     
  10. kork

    kork

    Joined:
    Jul 14, 2009
    Posts:
    265
    Support for transitive Git dependencies would be really helpful. Publishing components on GitHub is a lot quicker and easier than setting up a custom NPM registry.
     
  11. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    31
    @okcompute_unity thanks, I will
    I ended up implementing my own registry, so I'll have more control over packages distribution and usage
    I'm having other problems at the moment, but that's for a different post :)