Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Multiple package sources

Discussion in 'Package Manager' started by Kruemelchen, Nov 6, 2019.

  1. Kruemelchen

    Kruemelchen

    Joined:
    Aug 26, 2019
    Posts:
    18
    Hello,

    I'd like to ask if there is any possibility to use multiple package sources at once? E.g. a git, local and a registry source simultaneously?

    In my use case I'm using local packages but they are not distributed in my main git repository and therefore unavailable for others like the build server for example.
    I decided to use local packages because my main project and the packages are heavily connected to each other and I'm in a early development stage where I have to make lot's of changes in the custom packages I wrote/ am writing. By using local packages I have the opportunity to edit the packages in my IDE side by side with the main assemblies.

    But there may also be another use case. Imagine your git or your registry server is down. All your builds will fail because of missing packages.

    Wouldn't I be great to have some sort of FALLBACK sources for the specified packages?
    Then I could stay with my local packages and the build server would receive the packages via git.
     
  2. felipemunoz_unity

    felipemunoz_unity

    Unity Technologies

    Joined:
    Apr 5, 2019
    Posts:
    10
    You can use multiple sources within the same project, depending on the unity version you are using:

    Embedded packages: You can have your package source in a folder under your project Packages folder. This is considered an "embedded" package, and at that point it becomes the first entry point to your code. If you have an embedded package that also exists in the registry, the embedded package takes precedence

    Hosted Registry (2018.3+): You can host packages in a registry and point your project to that specific registry by modifying the manifest.json file, adding a new key called "registry":"<url to your registry>"

    Scoped Registries (2019.1+): You can have scoped registries for some of your packages. This allows you to have packages sourced from different registries in your project (e.g. Fetch an official package from https://packages.unity.com while also fetching your package from your personal registries)

    Git Packages (Beta in 2018.3+, Relased in 2019.1): You can add a package dependency pointing directly to a git repository.

    Tarball Packages (2019.3+): You can add a package dependency pointing directly to a .tgz file.

    Now, there is no "fallback" mechanism for package resolution (and I think there are no plans to support this neither). So, if you have a package being fetched from a git repository and you lose internet connection, then you will have to manually switch the fetching mechanism to something else if you want to.

    That being said, if you ever lose connection then Unity should be using the cached version of the packages the project depends on. So you should be able to continue working even offline, but there might be errors if you update your project manifest while offline...

    Some extra info on how packages work: https://docs.unity3d.com/2019.1/Documentation/Manual/Packages.html
     
  3. Kruemelchen

    Kruemelchen

    Joined:
    Aug 26, 2019
    Posts:
    18
    @felipemunoz_unity thanks for the quick response.
    I already know how packages work. But I don't understand why a fallback mechanism would be a bad idea.
    With local packages I have great advantages like:
    * each package can be a git repository
    * editing packages directly in multiple different unity projects without having to push/pull, updating the package etc.
    Just look at this use case:
    I'm currently developing an inventory system for my game. I split the inventory code to a custom package so that I can re-use it in my other projects too.
    As I'm currently developing it's important for my to ensure a barrier-free work flow for accessing the package source code without having to re-build / re-packaging. Of course I could also switch to embedded packages but then I would have to check-in the package to the unity projects version control.
    The setup I would like to have is the following:
    * Packages are stored external on a disc. They are also a git repository
    * Configuring packages as local packages gives me the ability to edit them across all of my projects simultaneously. I could run the tests in a package host test project and can also make changes in the main unity project when using the inventory system in real conditions
    * Having the ability to specify multiple sources at once would lead the build server or team mates to use the package from the git repository rather than aborting because of not finding the package on their disk

    The workaround I'm now going to try:
    Using 2 unity projects
    * Package Host (packages are used locally): Runs tests, developing packages
    * Main project (packages via git url)

    Other approach:
    Using embedded packages and exporting them after their development succeeded. But let's be honest: they are never really finished. Especially in an early state of development

    But well.. it's just an idea. Local packages are great but they are actually really useless in a real world scenario...
     
  4. felipemunoz_unity

    felipemunoz_unity

    Unity Technologies

    Joined:
    Apr 5, 2019
    Posts:
    10
    The issue with fallback mechanisms is that it causes package fetching to be non deterministic, and the whole point of packages is to have a deterministic approach to acquire code so that it can be easily replicated in different machines... otherwise you could end up with one machine using a package from the registry, and another unknowingly using a local version of it that could be incompatible/outdated and cause issues later on when sharing the project because the dev didn't know that the API he used is deprecated/removed in the future.

    As far as I know, this is not a feature that is being planned for the future neither.
     
  5. Kruemelchen

    Kruemelchen

    Joined:
    Aug 26, 2019
    Posts:
    18
    @felipemunoz_unity
    I understand the problematic with a fallback mechanism. I now switched from local packages to git packages but I have to say that it's really a pain developing custom packages.
    Whenever I update the package's repository I have to edit my manifest MANUALLY and remove the lock. After that I also have to delete my cache MANUALLY to make unity work again. Not speaking of the circumstances that I have to use another unity project just for editing the packages via local reference!!
    Guys.. there must be another way...
    I understand that a package is a kind of closed construct which is being tested internally and developed isolated from other projects. BUT in reality you just CAN'T do it that way all the time. E.g. I developed a package for all of my unity extension methods. Whenever I want to have a new extension method I now have follow this work flow:
    • Switch to a "package host" unity project (all of my packages are referenced as local packages)
    • Edit the package
    • Git add, commit, push, etc.
    • Switch back to my main project (where I'm using my custom package via git url)
    • Delete the lock in the project's manifest.json
    • Delete the cache
    These are simply to much steps!
    First thing that may help: Update/Pull button in the package window for git packages
     
  6. Adrian

    Adrian

    Joined:
    Apr 5, 2008
    Posts:
    378
    Why not just edit your manifest.json, change the package to a local one and never commit that change? If you're worried about accidentally checking it in, you can add a pre-commit hook to prevent local package references from being checked in.

    The development flow then would be:
    • Turn the package into a local one
    • Work on the package, immediately test it and commit when convenient
    • Before bumping the package's version, reset the manifest.json, test the package through git and commit the updated lock
    • Repeat…