Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Custom Package with Git Dependencies

Discussion in 'Package Manager' started by OloinMilsom, Feb 12, 2019.

  1. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,226
    Now this is stale again. Are there any progress at all? Could you just make works about this feature be more transparent?
     
  2. dpeter99

    dpeter99

    Joined:
    Sep 29, 2013
    Posts:
    7
    This is the second quite big road block in 2 days. And as far as I can see now i have to set up an NPM server and probably pay for hosting just so my Untility package can be downloaded, when somebody uses the main package.
    I get that unity is focusing on bug fixes... but in my eye this is a big bug.
     
  3. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    72
    GH and GL both have package registries so you can serve NPM packages from them. I have used GL for standalone packages in the manifest but not as dependencies yet.
     
  4. PerunCreative_JVasica

    PerunCreative_JVasica

    Joined:
    Aug 28, 2020
    Posts:
    47
  5. Vadimskyi

    Vadimskyi

    Joined:
    Jan 13, 2020
    Posts:
    13
    This issue is a big deal to package creators.
    I'm hesitating on using third-party dependency resolvers, since they rely on Unity's internal api which can change any moment.
     
  6. JeromeGodboutAmotus

    JeromeGodboutAmotus

    Joined:
    Apr 24, 2020
    Posts:
    40
    Why is this still a thing in 2021? a package manager that cannot resolve dependencies? can we git ssh key per domain too please? what a pain for such a basic features.
     
  7. noio

    noio

    Joined:
    Dec 17, 2013
    Posts:
    244
    +1

    The prospect of pulling out some commonly used utilities into packages seemed attractive! Just uploading each of them to a separate git repo and have the dependency of one utility on the other resolved automatically.

    For me that's the main use case..

    The only other simple solution is to just create one monolithic "utilities" package? Makes it a lot less attractive to go down the upm road. I guess I'll just keep copying files from one project to the next. :(
     
  8. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    341
    Any news on this one?
     
  9. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    Since it looks like we are not gonna have UPM handling git dependencies, the better option at the moment is using Scoped Registries, where you would create NPM packages out of your Unity Package and upload it to a NPM service and add the registry URL to your list of Scoped Registries on the Unity Package Manager Settings.

    I did test some services and the better option for having a private Registry was JFrog in my opinion.

    I'm writing a document with my Pros and Cons when using them (Verdaccio, JFrog, Gitlab, AWS CodeArtifact, NPM). And when I finish it I will share it here.
     
    nilsdr likes this.
  10. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    341
    There is still the problem of not being able to declare the scope of dependencies in the
    package.json
    itself.
     
  11. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    There is no problem at all when using Scoped Registries.
    You can declare all your dependencies in the package.json exactly how Unity does with their packages. If they are found in any registry added to your PackageManager settings (Unity registry is done automatically), all of them will be resolved automatically.

    That is how I do it:

    package.json
    Code (json):
    1. {
    2.   "name": "com.mycompanyname.mypackagename",
    3.   "version": "1.0.12",
    4.   "displayName": "My Package Name",
    5.   "description": "My package description",
    6.   "keywords": [
    7.     "mypackagename"
    8.   ],
    9.   "author": {
    10.     "name": "Packages with the same author are grouped on UPM window"
    11.   },
    12.   "dependencies": {
    13.     "com.unity.localization": "1.0.1",
    14.     "com.unity.addressables": "1.19.6",
    15.     "com.mycompanyname.myutilities": "1.0.7",
    16.     "com.othercompanyname.singleton": "1.0.1"
    17.   }
    18. }
    Packages/manifest.json
    Code (json):
    1. {
    2.   "scopedRegistries": [
    3.     {
    4.       "name": "My JFrog Registry",
    5.       "url": "https://mycompany.jfrog.io/artifactory/api/npm/unity-npm-local",
    6.       "scopes": [
    7.         "com.mycompanyname",
    8.         "com.othercompanyname"
    9.       ]
    10.     }
    11.   ]
    12. }
     
  12. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    375
    The problem is that if your scoped registry package depends on a second scoped registry, both scopes will have to be defined at the project level.

    Would love to be proven wrong.
     
  13. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    Are you talking about the
    "name": "@ scope/packagename"
    issue?
    If it is not about that specific issue (which I still don't fully understand), I believe it is correct to add every scoped registry since Unity needs to know from where it should download the dependencies.

    btw, I'm not an expert at NPM scoped registries, I have started looking into it this year just to make it work on Unity and it really works perfectly with JFrog for my needs.
     
  14. Michael-Ryan

    Michael-Ryan

    Joined:
    Apr 10, 2009
    Posts:
    184
    @sandolkakos Are you still working on your pros/cons document, and is JFrog still your preferred solution?

    I just started working on migrating some code to UPM packages stored in private GitHub repositories, and I would like to establish a private scoped registry that will serve packages from those private repositories.

    I have minimal experience working with NPM scoped registries and no experience setting them up. Any information you share would be greatly appreciated.
     
    sandolkakos likes this.
  15. Michael-Ryan

    Michael-Ryan

    Joined:
    Apr 10, 2009
    Posts:
    184
    I've read in a few posts that seem to suggest JFrog (like the GitHub Packages registry) doesn't allow searching the registry for available packages, but that it does resolve dependencies fine. Is that the case? It would be great to have the Unity Package Manager UI show all packages available on the private registry.
     
  16. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    1,100
    Can't speak for JFrog Artifactory but I've been running a Verdaccio instance and that has worked well. Package info, listing and searching works on the Unity side and it's relatively straightforward to install (I have it running in a docker container). I'm using Git hooks to automatically publish packages whenever I push a new tag.
    Verdaccio does seem to be geared more towards single-instance smaller setups and Artifactory more towards being part of a bigger infrastructure.
     
    Michael-Ryan likes this.
  17. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    @Michael-Ryan
    I was formatting the text and also trying to find a place to post it. So I used Gatsby + GitHub Pages.

    That is a compilation of my findings regarding the use of the Unity Package Manager + package dependencies:
    https://sandolkakos.github.io/my-blog/upm-studies/
     
    Last edited: Feb 5, 2022
  18. ekyah411

    ekyah411

    Joined:
    Aug 12, 2019
    Posts:
    6
    @sandolkakos thanks for the summary, it is very useful!
    I managed to get public npmjs packages to be visible in UPM though.
    For me, the problem was that one of the scope names was too long and it was not discoverable (e.g. com.supersuperlongname.name1.name2). After shortening this name it worked o_O
     
    sandolkakos likes this.
  19. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    Hey @ekyah411, nice to know that you like my post.

    Hmm, I haven't thought about long names. Could you try my packages with the settings you use:
    com.sandolkakosdigital.myunitypackage01
    com.sandolkakosdigital.myunitypackage02

    they are hosted here:
    https://www.npmjs.com/settings/sandolkakos/packages
     
  20. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    Wowww, I can confirm it.
    I just created another package with a shorter company name "com.sandolkakos.myunitypackage03" and it is now showing on the UPM window.
    Thanks for the hint @ekyah411, I will include it in my blog post.

     
    Molder and ekyah411 like this.
  21. Krahgnatokz

    Krahgnatokz

    Joined:
    Mar 18, 2014
    Posts:
    8
    Bump...?
    I'm trying to create incremental packages that I can reuse in my projects. Is there some easy and obvious solution that I'm missing instead of making a package for each piece of functionality?
     
  22. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    You can create packages with the structure you decide. But I strongly recommend following that layout:
    - https://docs.unity3d.com/Manual/cus-layout.html

    We have a lot of ways of creating/importing a Package using the Unity Package Manager, but maybe as a starting point for working with packages, you could do one of those (sorted by difficulty):

    1. Create a new folder inside your `UnityProject/Packages` with a package.json
    - you will see the folder inside your Unity Editor, as a child of the Packages folder.
    - do what you want inside it
    - commit it normally with your game code (same project and not shared)

    2. Create a git repository with a package.json at the root of the repo
    - clone it to a specific folder of your PC
    - import that folder using the Unity Package Manager > Add package from disk...
    - do what you want inside it
    - commit and push the content to your specific remote repo
    - In the project you just want to reuse:
    - import the git package using the Unity Package Manager > Add package from git URL...

    3. When you start to understand more about how the packages are handled, you will for sure want to learn how to use the Scoped Registries to store your packages.
     
  23. bartix96pl

    bartix96pl

    Joined:
    May 31, 2019
    Posts:
    7
    How about your ETA? Permission granted to get this feature in Unity 2022 ;)
     
  24. joshrs926

    joshrs926

    Joined:
    Jan 31, 2021
    Posts:
    116
    Any news on resolving git dependencies per package? I’d love to be able to write packages of code for my projects that depend on each other and then install one and the package manager automatically install the other packages from github that the installed package depends on.
     
  25. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    It is totally possible. For that you need to store your packages in an NPM service and make use of the Unity Scoped Registries to add the URL where your packages are stored, so that the Unity Package Manager will be able to download them and resolve dependencies as well.

    I wrote a blog text about how I have learned thst and maybe it can give you an idea about how it works:
     
  26. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    341
    They are Git URLs, which means he doesn't have a registry, and he is using Git URLs instead. Not a registry, but Git URLs. I repeat: Git repos as Git URLs, not an NPM-like registry.

    Unity doesn't have native support for this, and currently, the only way to make it work is by using something like this: https://github.com/mob-sakai/GitDependencyResolverForUnity

    I don't know why you are doing it, but please stop dodging the issue.
     
    Quasar47, Thaina and ImpossibleRobert like this.
  27. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    ohhh, totally my bad. I was reading that on my cell phone and totally missed the "git" word in joshrs926 question. Okay, git dependencies are not possible, sorry for my misleading.

    I'm just sharing a knowledge that I didn't have before and was bringing so much pain to my brain every time I wanted to keep on the same track with git URLs.

    Should I wait for Unity to redesign their whole UPM system (if they would do that someday) or should I try anything else to have a good workflow in my projects?
    Well, if I have the Scoped Registries to help me with what I need, I will of course make use of it instead of trying to use half of the functionalities I would have by using git URLs.

    I believe there are more people in the same situation I was in before, so talking about it really helps those guys. But if one does not want it, that is fine. There are lots of people that would like to develop games in Unity using Python as the programming language, but since Unity uses C#, what can those guys do about it?
     
  28. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,226
    Not in this thread because this thread from start to now only related to
    Custom Package with Git Dependencies
     
    HunterProduction likes this.
  29. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    1,100
    My point from more than two years ago still stands – don't plan on this feature arriving soon, find a good workflow with what's available now. As @sandolkakos has outlined.
     
  30. szsahajsz

    szsahajsz

    Joined:
    Aug 17, 2018
    Posts:
    13
    we still hope, this + custom dependency for specific samples within the package.
     
  31. Salveiro

    Salveiro

    Joined:
    Jul 24, 2018
    Posts:
    5
    2023 is about to be started, yet no bugfix would be more important for me than custom package git dependancies in UPM. Is there gonna ever again be any update on this?

    Even deny this feature development so I know I should abandon my hopes?
     
  32. JaamySk

    JaamySk

    Joined:
    Jan 8, 2022
    Posts:
    1
    Unity Are you there? Are you listening? For whom are you creating your product? Isn't that supposed to be us? Please add this feature. The Scoped registries are not equal to what we are asking here. Assume I'm developing two different projects on git, one depending on the other one, for each small change in the first repo, you want me to make a tar file and deploy it to a npm scoped registry? while that could be as easy as committing and pushing?
    It's about FOUR years from the day you have confirmed that what we are asking does make sense. Are you there? If yeah for who?
     
  33. Romaleks360

    Romaleks360

    Joined:
    Sep 9, 2017
    Posts:
    73
    BUMP! Unity pls fix this already :(
     
  34. Sluggy

    Sluggy

    Joined:
    Nov 27, 2012
    Posts:
    1,017
    When I recently came back to Unity and GamdeDev in general I saw the potential to Custom Packages but was also quickly disappointed by how they couldn't easily resolve dependencies. I ended up making a very simple tool to work around the issue. Essentially you just clone all of your packages to a single local folder and then you can point this tool to it and use that to quickly install which packages you need into a project either via file copy or symlinks depending on needs. It's been quite handy for myself lately even in its limited capacity. Perhaps someone else will find use for it. Here is a link to the source code.

    https://github.com/Slugronaut/UnityPackageInstaller

    It was a rush job so don't expect anything fancy. It's also Windows only as I wrote it using WPF. It currently can resolve simple dependencies between custom packages that are found within the folder but doesn't pull anything from the internet. Perhaps some day I'll consider adding that functionality but atm I don't have time. Maybe it will help someone in the meantime.
     
    Thaina and sandolkakos like this.
  35. Eclextic

    Eclextic

    Joined:
    Sep 28, 2020
    Posts:
    142
  36. Eclextic

    Eclextic

    Joined:
    Sep 28, 2020
    Posts:
    142
    So for anyone wondering, I do have a hacky way of making dependencies work...
    Code (CSharp):
    1.  
    2. public class Setup
    3.     {
    4.         [InitializeOnLoadMethod]
    5.         public static async void InstallDependencies()
    6.         {
    7.             var value = Client.List(false, true);
    8.  
    9.             while (!value.IsCompleted)
    10.                 await Task.Delay(100);
    11.  
    12.             foreach (var item in value.Result)
    13.             {
    14.                 if (item.name == "com.username.package")
    15.                     return;
    16.             }
    17.  
    18.             Debug.LogWarning("The dependency 'com.username.package' is not installed! Installing...");
    19.             Client.Add("https://github.com/gitusername/gitpackage.git");
    20.         }
    21.  
    It literally checks every time whenever there was a compilation triggered and searches for if the required dependency is in the project. If not it installs it forcibly...

    Just put it inside the Editor assembly of your package that requires the dependencies!
     
  37. matheus_inmotionvr

    matheus_inmotionvr

    Joined:
    Oct 3, 2018
    Posts:
    63
    Four years later, and we're still asking for this.
     
    Eclextic likes this.
  38. Kazko

    Kazko

    Joined:
    Apr 2, 2014
    Posts:
    82
  39. matheus_inmotionvr

    matheus_inmotionvr

    Joined:
    Oct 3, 2018
    Posts:
    63
    Thank you for the suggestion, but it seems like this approach involves .unitypackage files, which is exactly what I'm trying to shy away from. I would like to use UPM, and for it to solve some (private) git dependencies in packages.
     
    Eclextic likes this.
  40. sliptrixx

    sliptrixx

    Joined:
    Mar 24, 2016
    Posts:
    12
    It's been so long and I finally gave in and made my own tool to handle the git dependencies. I wish Unity would soon release native support for this and make this tool redundant.

    Here's a link to the repository: https://github.com/hibzzgames/Hibzz.DependencyResolver

    Basically, package developers can define dependencies under a new field in the package.json file called git-dependencies and list the dependencies. Like this:

    Code (JSON):
    1. "git-dependencies": [
    2.     "https://github.com/username/dependency1.git",
    3.     "https://github.com/username/dependency2.git"
    4.   ]
    And the tool will automatically download the dependencies (after prompting the user for security reasons). Let me know if you guys find it useful.
     
    starikcetin, Thaina, Quasar47 and 4 others like this.
  41. sandolkakos

    sandolkakos

    Joined:
    Jun 3, 2009
    Posts:
    285
    It looks like a nice approach, I've been working on a similar solution 2 years ago.
    I'm posting more to give you a suggestion... for people that use to execute the builds in Batch Mode, you will probably need to put a check like `if (Application.batchMode)` to bypass the confirmation dialog.
     
  42. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,226
    In the meantime I would like to present my own solution

    https://gist.github.com/Thaina/eec5752b25f7bfd3737f7dd9ed2fa53c

    This is a script to just curate all installed package, get all dependencies to see if any was missing, then present the wizard windows to let user reorder what and which should be resolved first

    Require package com.unity.editorcoroutines. And have not tested for anything more than 1 dependency as of yet

    upload_2023-11-12_21-23-37.png
     
    Last edited: Nov 14, 2023
  43. daxiongmao

    daxiongmao

    Joined:
    Feb 2, 2016
    Posts:
    413
    Found this thread and was hoping there might be some good news as the end of it. Seeing how unity has not replied in quite a while. This is probably dead.

    They complain about trying to solve every possibly use case. We don't need that. Just tell us how it has to be and let us be able to have a git package, get a dependency from another git package. By tag or commit, and if it doesn't follow their guidelines then it wont work.

    I just setup like 10 git repos for packages hoping I could have one include the other but no. Each package is a single thing, and would have a tag for every version if any.
    Sure it might not work for everyone but it seems like a simple and common use case.
    Just make that one work at least.

    But now I am going to go back to having my own package server and hopefully that will work.
    But that makes it harder to share since, git did the hosting for me. Now i have to get access for the packager server to have access to others.
     
  44. luvcraft

    luvcraft

    Joined:
    Aug 22, 2011
    Posts:
    79
    Until dependency URLs for packages are implemented, I've found a workaround that doesn't involve a custom package manager:

    provided your base package is coming from git, you can setup your dependencies as git submodules of your base package.

    for example, if you have a package A that depends on package B, you can go into terminal for package A and type
    git add submodule [path to repo for package B]
    , and that will add B as a submodule for A, and fortunately Unity Package Manager resolves submodules when it imports packages from git.
     
    sandolkakos likes this.
  45. Eclextic

    Eclextic

    Joined:
    Sep 28, 2020
    Posts:
    142
    Wait UPM resolves git submodules?! That thought never crossed me... Why would UPM allow submodules, but not actual git repos???

    You are a genius, if this actually works!
     
    luvcraft likes this.
  46. msfredb7

    msfredb7

    Joined:
    Nov 1, 2012
    Posts:
    174
    Maybe I'm ignorant, but aren't submodules supposed to be located inside an existing repository folder? And that would make a package look like this:
    Code (CSharp):
    1. package.json
    2. Runtime/..
    3. Editor/..
    4. MyDependencies/Submodule1/..
    5. MyDependencies/Submodule2/..
    6. MyDependencies/Submodule3/..
    So if two different packages use the same dependency submodule, they would duplicate the source?
     
  47. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,226
    Yes, I think so, and that's the reason I would not use it
     
    sandolkakos likes this.
  48. luvcraft

    luvcraft

    Joined:
    Aug 22, 2011
    Posts:
    79
    yeah, that's a downside of this method, but if only one of your packages uses that submodule then things are happy.
     
  49. EnduvoJD

    EnduvoJD

    Joined:
    Aug 12, 2016
    Posts:
    54
    It would probably be good to mention the minimum Unity version supported. Unity 2019.4 doesn't have package manager events.
     
  50. HunterProduction

    HunterProduction

    Joined:
    Sep 11, 2020
    Posts:
    23
    @maximeb_unity Do we have news about official support of this feature?
     
    sandolkakos likes this.