Search Unity

Feedback Some feedback on Package Manager git support

Discussion in 'Package Manager' started by TheValar, Sep 11, 2019.

  1. TheValar

    TheValar

    Joined:
    Nov 12, 2012
    Posts:
    760
    Spent some time this week experimenting with making my own packages as git repos hosted on gitlab. Overall it is promising but there were a few pain areas that I just wanted to report my feedback on.

    Project Structure:
    currently your package folder has to be the root of your repository which makes it kind of awkward to develop. Ideally you will be developing your package inside of a Unity project so assuming you want that project to be in git as well the only solution is workarounds involving sub modules or sub trees. It works fine but I feel like there could be a more convenient solution

    Versioning Support:
    currently there is no official support for package versions or updating your packages. This is a pretty important feature. You can work around it by manually editing the json but I think this needs to be supported in the package manager ui

    Dependencies:
    this is by far the most glaring deficiency. there really needs to be a way to make git hosted packages dependent upon other git hosted packages.

    I really like the idea of the package system and think this could be the future of unity assets, particularly open source ones distributed outside of the asset store so I hope you continue to work on custom package support.
     
    Lobolopez and starikcetin like this.
  2. Lobolopez

    Lobolopez

    Joined:
    Feb 26, 2018
    Posts:
    25
    Totally agree on those points (particularly hooking a package to a Git subdir would be excellent for a lot of reasons)
    Recursive as it might sound.. maybe if those things get to a testable stage it would be possible to release a packaged version with updates to the package manager without having to jump Unity versions....?
     
  3. samuelb_unity

    samuelb_unity

    Unity Technologies

    Joined:
    Nov 13, 2017
    Posts:
    151
    Hi @TheValar,

    Thank you for the great feedback! We take this all onboard and are currently working on the issues you raised. Can't go into much more detail than that for now, sorry!
     
    Last edited: Sep 17, 2019
    Lobolopez and TheValar like this.
  4. Lobolopez

    Lobolopez

    Joined:
    Feb 26, 2018
    Posts:
    25
    I am wondering if you had any updates of those questions and also, if not, where would it be the right place to check that some of those questions have been addressed? Is there a good way to track down package-manager-related development?
    I am thinking, specially, on the "project structure" part of the conversion, being able to hook to a subdir of a repo (rather than the root) would be extremely helpful for us. I can see how a lone dev might not have that much of an issue, but when several people develop a single package and they need a reference project for testing and the like it does get complicated (and there are no good solutions for that: two repos? messy, single repo + split? breaks down, soft links? need to be replicated by hand (particularly tricky in multiple dev platform setups) etc, etc)
    So, assuming that one feature is still on the way, how could I check that it's made it without bugging you guys? ;D
     
  5. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    On the release note of the new 2020 alpha, there is this line at the very end :

    Package Manager: Added support for a Git package to be located in a repository sub-folder.

    Any doc on that ?
     
  6. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    Hi @Jichaels,

    The documentation is in the work. Here is the draft that I sent to the technical writer:

    **********************************************
    Path query parameter (Subfolder)
    You can specify a repository subfolder for a Git package through the
    path
    query parameter. The Package manager will only register the package located in the specified repository subfolder and disregard the rest of the repository.

    This feature was introduced in Unity 2020.1.a21

    Special considerations:

    • path
      must be a relative path to the root of the repository. An absolute path won't work. (ex:
      path=c:\my\repo\subfolder
      is ❌️,
      path=/subfolder
      is ✅️)
    • .. and . indirection notation is supported but will block at the repository root (i.e. /../../.. will resolve to / )
    • path
      query parameter must be placed before the revision anchor. The reverse order will fail.
    • A package manifest (package.json) is expected in the specified path.
    Examples:

    Path query parameter

    https://github.com/user/repo.git?path=/example/folder


    Revision anchor and path query parameter
    https://github.com/user/repo.git?path=/example/folder#v1.2.3


    Two packages from the same repository

    https://github.com/user/repo.git?path=/packageA
    https://github.com/user/repo.git?path=/packageB


    **********************************************

    I hope it helps! And please, let us know if there are any issues.

    Regards,

    Pascal
     
  7. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    Super cool, thanks !

    Does that allow us to have dependencies of other subfolder as package dependencies ?

    Like in the package.json of
    https://github.com/user/repo.git?path=/packageA


    can I have
    https://github.com/user/repo.git?path=/packageB


    And more importantly, is the "multiple version" (like all the unity package, having the versions available and being able to hit "update" if there is a newer version according to SemVer) support coming ? Is there an ETA on that ?

    Thanks a lot !
     
  8. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    Not yet. But something we plan to support at some point. No ETA. Sorry.

    This is not something we are planning to implement.

    Regards,

    Pascal
     
    Thaina likes this.
  9. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    Ok cool, because currently I have to manually add packages in order, or I get errors, which is pretty bad for users.


    Really ? Isn't it already implemented ? Because it's how it works with all unity packages, and it would be really cool to have for us aswell :(
    Maybe not the "select your version from all available", but at least being able to upgrade with a button on the package manager. Having to delete the lock and all that is extremely not user-friendly, and a normal thing to have for managing packages :/
     
  10. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    Oh, yes! That we definitely want to support. My answer was about implementing a Git client in the Package Manager UI to enable listing branches, tags, and parse package versions, etc.. This we don't have plans to do.
     
  11. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    Ok fair enough ! Having dependencies and upgrade is fine to me !
     
  12. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    Ok, so I've been trying things today :

    - On the input field "Add from git URL...", I cannot type "#" (but I can paste it).

    - I made a github action file that will split a repo containing multiple packages to branches, that you can get with https://github.com/YourUserName/MainRepoName.git#PackageName and will also rename "Samples" to "Samples~" and "Documentation" to "Documentation~" so you work on your project with everything, but it's fine in the git packages
    Your main repo need to be structured like this :

    Code (CSharp):
    1. MainRepo
    2.   - Package1
    3.   - Package2
    4.   - ...
    Here is the github action :
    Code (Bash):
    1. name: Create package branches
    2. on:
    3.   push:
    4.     branches:
    5.     - master
    6.  
    7. jobs:
    8.   rename_samples:
    9.     name: Create branches
    10.     runs-on: ubuntu-latest
    11.     steps:
    12.       - uses: actions/checkout@v2
    13.       - run: |
    14.           for i in `ls -d */`; do
    15.             git checkout master
    16.             branch=${i::-1}
    17.             if [ `git branch --list $branch` ]; then
    18.               git branch -d $branch
    19.             fi;
    20.             git subtree split -P $branch -b $branch
    21.             git checkout $branch
    22.             git config --global user.name 'github-bot'
    23.             git config --global user.email 'github-bot@users.noreply.github.com'
    24.             if [[ -d "Samples" ]]; then
    25.               git mv Samples Samples~
    26.               rm -f Samples.meta
    27.               git commit -am "rename: Samples => Samples~"
    28.               git push -f -u origin $branch
    29.             else
    30.               echo "${i}Samples does not exist"
    31.             fi
    32.             if [[ -d "Documentation" ]]; then
    33.               git mv Documentation Documentation~
    34.               rm -f Documentation.meta
    35.               git commit -am "rename: Documentation => Documentation~"
    36.               git push -f -u origin $branch
    37.             else
    38.               echo "${i}Documentation does not exist"
    39.             fi
    40.           done;
    It's probably not perfect, but it's fine for what I wanted ! :) (and way easier to work with than working in the Packages/ folder of unity
     
  13. Lobolopez

    Lobolopez

    Joined:
    Feb 26, 2018
    Posts:
    25
    First, that's fantastic!
    Second... is there a way to get that through updating the package manager itself? I could use this yesterday but won't be able to migrate our Unity to 2020 in, well, quite a while. It would be incredibly useful if stuff like this could be cherry picked!
     
  14. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    We are looking into backporting this feature in 19.3 (finger crossed)
     
    Lobolopez, Thaina and xucian like this.
  15. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    That is a bug. I will report to the team but it would be even better if you could report an official bug too.

    Thank you!
     
  16. xucian

    xucian

    Joined:
    Mar 7, 2016
    Posts:
    846
    +1
    It'll be of great help, as most people will probably use 2019 for 1-2 more years
     
  17. ethan_jl_unity

    ethan_jl_unity

    Unity Technologies

    Joined:
    Sep 27, 2018
    Posts:
    104
    Which Unity version are you using? I just tired locally on Windows and Mac and everything seems to be working fine.
     
  18. Jichaels

    Jichaels

    Joined:
    Dec 27, 2018
    Posts:
    237
    2020.1.0a21.2690, using an AZERTY keyboard, Win10
     
  19. Vusa360

    Vusa360

    Joined:
    Oct 4, 2017
    Posts:
    12
    Is this something that is likely to happen?
     
  20. supita_unity

    supita_unity

    Unity Technologies

    Joined:
    Dec 13, 2019
    Posts:
    226
    Yes! :) The packport is already done, the support for Git packages on repository sub-folders will be in the upcoming Unity 2019.3.4f1
     
    Lobolopez and xucian like this.
  21. Vusa360

    Vusa360

    Joined:
    Oct 4, 2017
    Posts:
    12
    That's great :) Can't wait for that!
     
  22. xucian

    xucian

    Joined:
    Mar 7, 2016
    Posts:
    846
    Can someone on the team reply to this thread when 2019.3.4f1 is out? :)
    I'd like to switch to it right away
     
  23. jasonatkaruna

    jasonatkaruna

    Joined:
    Feb 26, 2019
    Posts:
    64
    It's out now, using the feature today
     
    supita_unity and xucian like this.
  24. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    340
    That is amazing! Thank you! <3
     
  25. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Is it possible to backported this to 2018.4 ?
     
  26. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    756
    I'm sorry, backporting this feature in 2018.4 is not planned at this time. If you have an enterprise account, you can reach out to your account manager to request this feature to be backported.

    Regards,

    Pascal
     
  27. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    That's AWESOME!
    Does that supports symlink?
    Let's say that I have a file in the root of the repo, and I use symlink for reference in some sub folders that I use as package folders. Would that work?
     
  28. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,364
    feedback: too complicated. it should be as simple as : paste the root url of the project and PM auto search the tree for all package.json then lists them
     
  29. nex_derek

    nex_derek

    Joined:
    Apr 19, 2021
    Posts:
    1
    Re: Revision anchor and path query parameter

    Does it support private ssh access? I found it doesn’t seem working on my side :-/
     
  30. RoflcopterInk

    RoflcopterInk

    Joined:
    Jul 15, 2021
    Posts:
    7
    Same here, SSH access does not seem to allow the path parameter..
     
  31. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    It works with SSH URLs but not when they are in the SCP-like format, which, unfortunately, is the format that is given by most (all?) major Git repository hosting services like Github.com for SSH access.

    You can easily convert the SCP-like form to a standard SSH URL by doing two changes to the original string (both changes are required):
    - Replace the colon (
    :
    ) between the server name and repo path with a forward slash (
    /
    )
    - Prefix the result with
    ssh://


    For example: Git URL.png

    The resulting
    ssh://...
    URL should work with the optional path and revision features.
     
  32. RoflcopterInk

    RoflcopterInk

    Joined:
    Jul 15, 2021
    Posts:
    7
    Perfect. Thank you!
     
  33. moolman-ruan

    moolman-ruan

    Joined:
    Feb 28, 2023
    Posts:
    1
    maximeb_unity likes this.
  34. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    By the way, as of 2021.3.21f1 / 2022.2.11f1 / 2023.2.0a6, Git URL dependencies using the SCP syntax now recognize the path and revision extension as well. The change was also backported to 2020.3 and will be available in its next release.
     
    NotaNaN, Thaina and De-Panther like this.
  35. mrstruijk

    mrstruijk

    Joined:
    Jan 24, 2018
    Posts:
    52
    It took me a while to figure this out, so I leave it here in case it's useful for someone else.

    If you want to have a particular commit (e.g. if version 1.5.1 broke your will to live, and you need 1.5.0), you can add the GitHub SHA.

    Find the commit you want to use in the repo (example: the Multiplayer Samples) under "History", and then hit one of the icons on the right (saying "Copy the full SHA").

    Then in your Manifest, add the SHA at the end of the Git URL, after the '#' sign, like so:

    https://github.com/Unity-Technologies/com.unity.multiplayer.samples.coop.git?path=/Packages/com.unity.multiplayer.samples.coop#26aa8ac82a1b2f4d3800b2aa5759a5ad65f935da 


    So your complete line in the Manifest:

    "com.unity.multiplayer.samples.coop": "https://github.com/Unity-Technologies/com.unity.multiplayer.samples.coop.git?path=/Packages/com.unity.multiplayer.samples.coop#26aa8ac82a1b2f4d3800b2aa5759a5ad65f935da", 
     
  36. bz-chris

    bz-chris

    Joined:
    Sep 7, 2022
    Posts:
    3
    Does the path navigation work with SCP-like urls?

    git@mycompany.github.com:gitproject/com.mycompany.mypackage.git?path=/folder1/folder2

    The package manager is currently giving me the below error:

    [Package Manager Window] Cannot perform upm operation: Unable to add package [git@mycompany.github.com:gitproject/com.mycompany.mypackage.git?path=/folder1/folder2]:
    Version range 'mycompany.github.com:gitproject/com.mycompany.mypackage.git?path=/folder1/folder2' is invalid. [InvalidParameter].

    Im using the latest LTS Unity 2022.3.2f1

    Thanks in advance
     
    Last edited: Jun 19, 2023
  37. bz-chris

    bz-chris

    Joined:
    Sep 7, 2022
    Posts:
    3
    Ignore me I have managed to get it working in the latest LTS Unity 2022.3.2f1