Search Unity

  1. Check out the Unite LA keynote for updates on the Visual Effect Editor, the FPS Sample, ECS, Unity for Film and more! Watch it now!
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  5. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Git support on Package Manager

Discussion in 'Package Manager' started by rizu, Oct 24, 2018.

  1. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    2018.3.0b7 added following:
    but docs haven't been updated for this yet, would it be possible to get example what to manually put into manifest to ref specific git branch?
     
    dzamani likes this.
  2. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    SugoiDev and dzamani like this.
  3. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    207
    Hi @rizu,

    Glad you have found the pattern :). As stated in the release notes, this is an experimental feature. We are targeting production ready quality for 2019.1. Super glad that you are trying this out though. Please, let us know if you find any issues.

    And here is a minimalist documentation if you would like to expand your testing further and maybe for others who are reading this thread.

    Requirements

    To use Git packages in a project, Git must be installed on the user machine. The Git executable path should be listed in the PATH system environment variable.

    Project manifest lock attribute

    This feature introduces a new lock attribute in the project manifest. This attribute is populated by the Package Manager to render the package configuration resolution deterministic.

    Supported URL schemes/protocols

    The Package Manager supports the same URL schemes as Git does. In fact, the URL is passed to the Git executable (minus the revision part). If Git ever adds support for more protocol/schemes, they will automatically be supported by the Package Manager.

    List of supported schemes
    • git:
    • ssh:
    • https:
    • http:
    • file: (only URL-like path are supported for now. i.e. file://absolute/path/repo.git. System path will be supported later. This could allow using relative path for example file:../../repo.git)
    We also support SCP style pattern:

    git@github.com:user/repo.git


    Revisions

    You can select a specific revision to be cloned by the Package Manager. The revision is prefixed by the # character. The revision should be added after .git extension:

     https://repository/path.git#revision


    The revision can be any commit-ish - an expression that eventually resolves to a commit hash. The most common types of commit-ish are commit hashes, tags, and branches.

    Limitations and good to know:
    • The Package Manager UI was not updated to support this new package source (ETA 2019.1). I haven't seen any more issue though.
    • Big repositories can take a long time to clone. We don't cache the repository so if you change revision, the repository will be cloned again.
    • The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.
    • Since the Package Manager uses the Git executable installed on your machine. Any global or system configuration or environment variable will be picked up. These settings may affect the behaviour of the system.
    • Some package managers offer a shorthand version of their supported URL schemes. For example, npm support short URLs like these: <github user>/<repository>, gitlab:<gitlab user>/<repository>. We don't support these syntaxes for the moment.

    Hope this will help you test the feature.

    Regards,

    Pascal
     
  4. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    @okcompute_unity, thanks, that was very helpful. Any chance to get similar brief intro for scoped registries (private registries)? :D
     
  5. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    207
    Sure! Just start a thread for it :p
     
    rizu likes this.
  6. falkj17

    falkj17

    Joined:
    Sep 11, 2017
    Posts:
    8
    @okcompute_unity thanks for that information! That is really useful!!
    Is there any chance that the package manager will support finding packages in subdirectories of a git repository? Let's say I have one repository that contains multiple packages in various subfolders of that repo. I want to target a specific one, would that be possible?
     
  7. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    207
    @falkj17 Yes, this is something on our roadmap. But we suggest putting your package at the root of the repository if possible. This will be easier for your users to find the package. For instance, users will have to know and specify the name of your package if it is deep in the repo. While if the package is at the root, only the URL will be required, the name will be optional.
     
  8. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
  9. falkj17

    falkj17

    Joined:
    Sep 11, 2017
    Posts:
    8
    Great, thanks! Any idea on how long it will take to add that functionality? ;)

    I completely agree that for visibility it would be better to have it in the root, but we would like to group them by different themes and have all relevant packages in one repo, if possible. Having every single package in a different repo also makes it more difficult to develop multiple packages in parallel
     
  10. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    IMO for development alone it's better to keep the repos separated, but I could see use for being able to keep different packages in single repo as well, like if you keep the git as custom repository that just holds various open source assets as packages (I already have two separate repos for this purpose atm).

    Also if you have like say, custom SRP fork right now, the repo contains many packages already and it would be handy if people could just ref to them directly.

    Right now you can workaround this by just putting different packages into different branches/releases but it's kinda hacky IMO.
     
  11. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    38
    Hi, quick question about this:

    If there isn't any revision specified, is the package manager able to detect new version of the git package ?

    In other dependency manager similar to this, you can for example specify an exact version match and it will go fetch that one that's what specifying the revision looks like at the moment. But is there a way with git repos (I know I can do it with the scoped registries) to fetch and check if there is an update of the repo (based on the package.json) ?

    Thanks again for this feature!
     
  12. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    if you don't specify the revision, I think it will just use the default branch configured for the git repo, and to update it later on refer to:
    Package specific lock attribute is generated at the end of manifest once the package manager processes the git clone/update, it basically stores the hash from git repo and revision. If you clear it, it should update to latest.
     
  13. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    38
    That's what I was asking, dependency managers like Cocoapods will check if it's the latest commit if you did not specify a revision.

    So what I'm actually asking is : Is there any plan to handle this usecase where the package manager will check if he has the latest commit or not on a specific branch?
     
  14. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    270
    I tried this and it didn't work.
    I managed to make it work by specifying the url with the ssh scheme:
    ssh://git@bitbucket.org/myorg/my-repo.git#1.0.0

    git: scheme didn't work. also note the slash "/" between host and org instead of colon ":"
    http(s) scheme is not working as it complains about missing credentials (and I don't want to specify my password in the url (that will get committed)) (note: my repos are all private)
     
  15. Rotary-Heart

    Rotary-Heart

    Joined:
    Dec 18, 2012
    Posts:
    348
    Is there anyway to add a dependency between git packages? Current dependency seems to only accept name and version:
    Code (CSharp):
    1.     "dependencies": {
    2.         "packagename": "1.1.1-preview"
    3.     }
    If a git link is added it will throw an error since it doesn't recognize it as a version number:
    Code (CSharp):
    1.     "dependencies": {
    2.         "packagename": "https://gitprovider.com/MyPackage.git"
    3.     }
    Other than this, pretty amazing how easily is to setup the git links, great job!
     
  16. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    I renamed the thread for clarity as this has gone well past the syntax of the manifest entry :)

    I'd guess it's the same deal as with direct folder refs atm (no dependency support). I do admit it would be handy but how should that be setup so it would work both locally and from git repos?
     
    Rotary-Heart likes this.
  17. Rotary-Heart

    Rotary-Heart

    Joined:
    Dec 18, 2012
    Posts:
    348
    I see, I missed that dependencies didn't work with local folders either.

    I would guess that the same way you setup them on your project manifest. All it really needs is to point to the location where the other package.json is located and process it.
     
  18. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    32
    I think you're missing a major use case for this. There is a LOT of great stuff already on Github and it usually ends up cluttering up Asset folders because there's no other way to bring it in to a project.

    If you supported packages in subdirectories we could:

    1. Fork the repo we want to use
    2. Add a package.json to the correct directory.

    Most Unity repos are entire projects. Forking and adding a package.json to the root dir won't work in most cases. Forking and rearranging the directory structure makes it impossible to track upstream changes.

    I've experimented with submodules, sparse checkouts and similar to get this working but they are extremely fragile.
     
  19. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    505
    If you share whole project IMO best place for custom packages is just to place them into Packages folder for two reasons: you don't have to modify manifest at all, Unity picks them from there as packages automatically and it also keeps your Assets folder clean as they are not there bloating it.

    I'd love to be able to point into that git repos Packages/<packagename> path directly from the manifest as this would let people share whole projects and also at the same time let others pick the package only from the project.
     
  20. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    32
    Unless I'm misreading you - you're making a slightly orthogonal point (which I agree with)?

    Just checking I haven't misunderstood.
     
  21. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    270
    I also understand that you are talking 2 different thing (I agree on both):
    - @rizu says, if you own a repo with a Unity project, you should put your stuff inside Packages/<your.package.name>/ and leave Assets/ empty (or with samples only)
    - @andybak says, existing github repos in the wild currently have stuff in Assets/, and he needs to fork them, add a package.json in their "root" (and asmdefs probably) then point to it.

    given that (especially wrt. existing stuff), I vote for being able to point UPM to a subfolder of a git repo.

    ps: question for unity devs: what would be the best practice for developing a git-hosted package from now on (until we get the ability to push them privately to your registry/asset store)?
    (1) repo should contain everything needed to develop the package, without being a submodule
    (2) I want to be able to pull it from package manager, either by using a "git:" dependency pointing to a tag, or by releasing it to a registry (main one or scoped) and requesting with the version
     
  22. zeh

    zeh

    Joined:
    Feb 2, 2013
    Posts:
    28
    Another point to being able to point to a folder (say,
    https://YourGitHost.com/UserName/RepoName.git/folderName
    ): you don't want everything on the root of a public repository. That'd make things a bit messy. For public repositories, it's better to just have a
    README.md
    , a
    LICENSE.md
    , a
    CONTRIBUTE.md
    , and any other documentation on the root, and the actual package on a subfolder so people don't need to download useless files into a project.

    The Unity WakaTime plugin at https://github.com/vladfaust/unity-wakatime is currently following this syntax. But it means it can't be included in the manifest as a Git URL because the actual package is inside a subfolder.
     
  23. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    226
    Hi @okcompute_unity

    Is there plans to support not only git repo but mercurial either ?
     
  24. pedro_unity

    pedro_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    93
    Hello all,

    @Jes28 : There are, currently, no plans to directly support Mercurial at this point. I'll bubble up your request to the team nonetheless.

    @andybak : You're right, currently package.json needs to be at the root of the package to be consumed by UPM. I've taken note of your request and we will follow up internally.

    Cheers,
    Pedro
    Unity STE
     
  25. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    226
    Thanks for fast response :)

    May be UT can just open some api, so we can add mercurial support ourselves :)
     
  26. stopiccot_onthespot

    stopiccot_onthespot

    Joined:
    Oct 1, 2016
    Posts:
    18
    @andybak @zeh I agree, without possibility to reference a subfolder as a package git referencing is almost useless(
     
  27. pedro_unity

    pedro_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    93
    @Jes28 As your signature states, "Nothing's Impossible" :)

    Cheers,
    Pedro