Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Git support on Package Manager

Discussion in 'Package Manager' started by rizu, Oct 24, 2018.

  1. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    443
  2. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    I did not change the url of the git packages when I was offline.

    I'll send the report asap and here's the repro steps.

    1. Add the repo in the manifest.json as a dependency while connected to the internet.
    2. Save & close the project
    3. Disconnect from the internet
    4. Launch project
    5. See errors
     
  3. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    443
    Package Manager does not SCP pattern for the moment. It is in our plan though. For now, use the SSH scheme.

    Ex:
    "ssh://gitlab.company.com/group-name/package-name.git#branch-name"


    Regards,

    Pascal
     
  4. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    84
    I believe every git url has two forms. For example

    1. https://github.com/Unity-Technologies/ScriptableRenderPipeline.git
    2. git@github.com:Unity-Technologies/ScriptableRenderPipeline.git

    They use different authentication methods but as far as I know all git servers support both forms so try converting your url to the other form.
     
  5. pablothedolphin

    pablothedolphin

    Joined:
    Oct 20, 2016
    Posts:
    53
    Thanks for the suggestion guys. This seems to have progressed me but now I'm getting errors saying that permission has been denied. This is usually the same error I get if I ever forget to add my ssh key to the putty agent when using source tree. But my ssh key definitely is assigned to my pageant key list, the repo url is definitely correct, the repo exists and I'm the owner of both the group and the project.

    In short, I'm kind of at a loss at what to try next. Any help appreciated!
     
  6. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    443
    @pablothedolphin ,did you try to do

    Code (csharp):
    1. git clone ssh://gitlab.company.com/group-name/package-name.git#branch-name
    in the terminal? Just to make sure the issue is not related to Unity?
     
    pablothedolphin likes this.
  7. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    65
    Actually the second form is a variant form of SSH URLs. SSH is a different mechanism to establish the connection with the remote Git server and perform authentication. HTTPS and SSH URLs are not strictly interchangeable, since some configuration changes may be needed.

    A note about SSH URLs: the Package Manager supports them, but not in that variant form. It needs to be in the following form:
    Code (CSharp):
    1. ssh://git@github.com/Unity-Technologies/ScriptableRenderPipeline.git
    Notice that the colon (':' symbol) needs to be changed to a forward slash ('/' symbol).
     
    Last edited: Apr 9, 2019
  8. GarthSmith

    GarthSmith

    Joined:
    Apr 26, 2012
    Posts:
    1,239
    I know for us, we used SourceTree which loaded SSH keys from Pageant. SourceTree uses it's own internal git.

    The git installed to the system, aka the git used from PowerShell or the Terminal, uses a different SSH key. I had to convert my old key or create a new one and put it at C:\users\UserName\.ssh\id_rsa . This allowed Unity to connect to our git repos, since it didn't have access to SourceTree's internal git.
     
  9. pablothedolphin

    pablothedolphin

    Joined:
    Oct 20, 2016
    Posts:
    53
    Thanks @GarthSmith! I did exactly as you said and made sure to add the new public key I made using git gui to my gitlab profile and that the key itself had an empty password. But even when I try to clone the repository from the command line I'm still getting the same issue. It asks for a password and denies me permission even though my repo is public, I'm the owner and I've got my non source tree git all set up.

    It likely isn't a Unity issue since I can't clone form the terminal but now I'm even more confused as to what's wrong.
     
  10. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    One other thing I noticed as well is that the unity package manager recursively checks out submodules, but that breaks things for me cause I've added other packages I'm currently working on into my main project to easily update them.

    Is there a way to prevent this submodule checkout?
     
  11. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    65
    I'm not sure what you mean about this. The Package Manager only clones the repository, there are no submodule operations performed.

    Could you please provide a bit more detail on your setup? Ideally, I suggest Reporting a Bug (in Unity: Help > Report a Bug) with a small repro project where this issue happens, and post the case number here. Thanks!
     
  12. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    With Git, cloning does do submodule operations if given the --recursive argument is provided.

    I have a main project that has submodules and I've symbolically linked the folders into my project as a sibling of my proper upm package. Next, I perform a split tree git operation on the main package I want downloaded into user's machines which then gets pushed to a upm specific branch where I put my release tags.

    But it seems that the Unity Package Manager is cloning the branch recursively which is also pulling in the submodules in projects that are using the main module.

    I'm not sure it warrants a bug report as it may be by design, but I'd at least like to be able to have the option of adding a package without it recursively cloning the repository.
     
    Last edited: Apr 13, 2019
  13. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    I'm pretty confident that the package manager is pulling in the submodules as well.

    You can try this yourself by adding this to your project dependencies

    Code (JavaScript):
    1. {
    2.   "dependencies": {
    3.     "com.xrtk.core": "https://github.com/XRTK/XRTK-Core.git#0.1.1",
    4.     ...
    5.   },
    6. }
    I'm getting an error about the path being too long, but it's really just bc it's trying to pull in the submodule when it shouldn't.

    Case # 1146083
     
    Last edited: Apr 15, 2019
  14. andybak

    andybak

    Joined:
    Jan 14, 2017
    Posts:
    84
    If we're discussing the best defaults I can think of more situations where you'd want submodules than when you wouldn't.
     
  15. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    65
    I dug further in the implementation and you're absolutely right - the
    --recurse-submodules
    argument is passed to the git clone operation. And yes, it turns out that this is by design: the reason for that is that our implementation was initially derived from npm, and keeping the same behaviour would avoid unexpected differences between the Unity Package Manager and npm.

    We don't have any plans (yet) to change it at this time since it would be a breaking change, and this limitation can be worked around fairly easily by using submodules outside of packages only, or under folders that are ignored by Unity ('.'-prefixed or '~'-suffixed, for instance). Your workflow seems non-trivial, so maybe the following idea won't help, but you could consider using .gitignore/.npmignore as well to prevent the submodule files from being committed to the split tree repository, or included in the final package.
     
    Stephen-Hodgson likes this.
  16. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    The interesting thing is, that the submodules don't appear in the split tree at all but they're still pulled in my the upm.

    For my Continuous Integration I do need the submodules to be checked out so I can properly generate my docs.

    I'll give the .npmignore a try
     
  17. Hermesis

    Hermesis

    Joined:
    Apr 3, 2014
    Posts:
    2
  18. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    @maximeb_unity adding the .npmignore didn't help.

    But I did stumble upon an interesting fact. While observing the package download I noticed that it wasn't pulling the specific branch, but instead the master (default) branch first, then switching to the tag/branch specified by the # suffix.

    This could be part of the problem we're seeing. Instead if the upm was checking out that specific tag/branch directly, then the submodules won't show up (they're left overs from the branch switch).

    This explains why there's no submodule folder in my upm branch, but one appears in the package cache.

    I tested this out with a local clone as well just to make sure, and sure enough, the submodlues folder gets left behind if you check out the master (defualt) branch first vs checking out the tag/branch directly.

    One other option is to do a hard reset when checking out the specific branch/tag to ensure there's not any left over files, but ideally we should be checking out only the branch/tag specified.
     
    Last edited: Apr 16, 2019
  19. Stephen-Hodgson

    Stephen-Hodgson

    Joined:
    Jul 15, 2014
    Posts:
    18
    So one way I was able to work around this was by setting my repository's default branch to the upm split tree, but this is also not ideal.
     
    Last edited: Apr 16, 2019
  20. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    443
  21. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    397
    Since it doesn't support subfolders, yet, I tried to use a simple Unity project as a package (just a single script). However, when UPM tries to resolve the package, I'm getting these errors:

    Read only asset Packages/com.company.common/Packages has no meta file.
    Read only asset Packages/com.company.common/Assets has no meta file.
    Read only asset Packages/com.company.common/ProjectSettings has no meta file.
    Read only asset Packages/com.company.common/package.json has no meta file.


    The package also can't be seen under Packages. What did I miss?
     
  22. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    397
    I tried this setup but I couldn't reference the code in Source to the test Unity project. I already added an assembly definition file to the folder in Source. How did you do it?
     
  23. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    397
    I was able to make it work. The workflow that worked for me is to use git submodule and store in a separate folder outside of Assets. I then add as local package the Assets folder of the repository that was added as a submodule. This way, I could still maintain the Unity project of that external common code repository.
     
  24. Hermesis

    Hermesis

    Joined:
    Apr 3, 2014
    Posts:
    2
    every file must have a
    *.meta
    file