Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice

Any robust package & dependency management systems for Unity?

Discussion in 'Scripting' started by kennyd-itp, Oct 27, 2014.

  1. kennyd-itp

    kennyd-itp

    Joined:
    Jan 13, 2014
    Posts:
    30
    We are working on a few cross-platform Unity-based applications at the moment, and it's becoming important to share code between them in a reliable way. We have strong toolchains and best practices in place for iOS, Android and Web development at our company - I'm trying to bring our Unity pipeline up to the same level of software development quality... which is a challenge, to be honest.

    I know it's fairly easy to share code using things like git submodules, but that's cumbersome and bound to create issues sooner or later. I'm particularly worried about dependency hell and version conflicts - what if I split my code up into several modules, but the dependencies between them start to diverge? E.g. my "unity-mobile" module is dependent on version 1.2 of my "unity-tools" module, but some other module in the project was written expecting version 1.1? NPM elegantly solves these kinds of issues by containing dependencies in module-specific silo's, but I don't see a clear way to do that with C# code for Unity.

    Does anyone know of a robust solution for this kind of thing? I.e. something resembling NPM or CocoaPods but applicable to Unity. I'm developing on Mac OS X by the way, so no Windows/Visual Studio-only solutions please.

    Thanks!
     
    Last edited: Oct 27, 2014
    mthorade likes this.
  2. jimrogerz

    jimrogerz

    Joined:
    Aug 8, 2013
    Posts:
    6
    How about just using npm? Put Unity modules in git repos, use github references in package.json, and node scripts in the module repos that copy files around to the right place.

    I had the same thing in mind and was wondering if anyone has tried it.
     
  3. Somaweb

    Somaweb

    Joined:
    Feb 4, 2015
    Posts:
    14
    Hey kennyd-itp, struggling on the same topic 1 year later. Do you have any solution so far?
     
  4. kennyd-itp

    kennyd-itp

    Joined:
    Jan 13, 2014
    Posts:
    30
    Hey guys,

    I started work on my own npm-style package manager in NodeJS a while ago, with the intention of distributing it through npm itself when it's ready. It's set up to work very much like npm or bower; basically you add a .json file in your Unity project with information about the dependencies in your project, and the tool checks them out from git and adds them to your project. There is no central package repository, just git repo URI's and some metadata about versioning. Here's a screenshot:



    Still needs some work on error handling and has only been tested on Mac so far - though I'd definitely want it to support Windows as well at some point. It also only supports C# code assets, as I grab all the .cs files from the dependency and compile it into a .dll using Mono, which then goes into the local Unity project. @jimrogerz: that's why I'm not using straight npm, I like the idea of dropping properly versioned/namespaced assemblies into Unity better than a straight copy of the source code.

    Ideally I'd like to have recursive dependencies versioned at the .dll level like the way npm does it, though I'm not even sure yet if that kind of silo'ing is supported by Mono. Does anyone have experience with this? And there's a bunch of other useful features you could build in once the basics are solid.

    I'm thinking of putting this up on GitHub, if anyone here feels like contributing that'd be awesome.
     
  5. Somaweb

    Somaweb

    Joined:
    Feb 4, 2015
    Posts:
    14
    Last edited: Mar 26, 2015
  6. bilke

    bilke

    Joined:
    Jul 13, 2012
    Posts:
    54
    Hi kennyd-itp,

    I am also curious about testing your dependency manager! Any news on putting it on GitHub?

    Thanks!
     
  7. kennyd-itp

    kennyd-itp

    Joined:
    Jan 13, 2014
    Posts:
    30
    @bilke Yeah, this has gotten pushed back quite a bit. I hope to put it up sometime soon but we've just been swamped with projects lately. If I have anything useful to share I'll let you know!
     
  8. UCh

    UCh

    Joined:
    Nov 18, 2012
    Posts:
    29
  9. kennyd-itp

    kennyd-itp

    Joined:
    Jan 13, 2014
    Posts:
    30
    Thanks for the list! The gradle/Kaizen solution looks most like the kind of solution I would like to see, and it has the advantage of fitting fairly well with our existing build setup at our company (we also use Gradle for both iOS and Android projects). Has anyone here used Kaizen before, or would anyone from Unity care to chime in on this use of it?

    I'm going to make some time soon to look into this more — if we do end up working on our own CLI tool inhouse, we intend to develop it in the open on GitHub. I'll let you know.
     
  10. cyrfer

    cyrfer

    Joined:
    Nov 4, 2015
    Posts:
    3
    I would love to make some progress on this ASAP. What is the latest? The community awaits!
     
  11. UCh

    UCh

    Joined:
    Nov 18, 2012
    Posts:
    29
  12. Meta-tron

    Meta-tron

    Joined:
    Oct 22, 2015
    Posts:
    19
    This is actually a very important issue with Unity development. If not one of the biggest productivity problems in Unity. There is a very elegant solution to this, but first I want to describe the problem. I know its a long read but its worth it if you have are trying to manage more than one game in your game development life.

    If you have written more than one game, You might have just instinctively started an entirely new project folder. The moment you do, you have just branched off your code,art and packages! Not only have you branched off, your probably inclined to start branching off your source control. After all, it seems logical to make this separation right?

    But now you have forever condemned yourself to switching projects between your projects to make updates, reviewing your work requires switching and loading. Adding and removing packages and tools requires doing this in every project folder. Even if they are common tools you need always. You probably have been tempted to create a common library for all your projects, but from then on, you know you will forever be plagued by going through every project and reloading and updating any changes that are shared between projects. At the end of the day, all you have really done was copy your library/code multiple times into each project even if they are exactly the same! (The kiss of death for maintenance reasons)
    So eventually you will get tired of trying to update all your projects all the time, and mothball a few of your them for a few weeks or months, only to later blow off the dust and find that over time, some of the packages (even unity) has been updated. And your perfectly working archived game now looks like it has some pretty nasty to fix errors in it. This whack-a-mole situation never stops and gets worse as the months wear on, your code base gets bigger, your projects accumulate, your packages accumulate. Now you find yourself spending a whole day on an old game just trying to make it compile again! You get done and now it has a few quirks!

    You revisit this one project folder idea concept again, but of course its impossible because not only would you get a zillion code conflicts. Your combined art and code assets would bring Unity to it's knees if it attempted to make a pass through all this. And forget about doing searches or using intelisense through all this stuff. Visual studio is good, but it cant slog through 20 games worth of code without bring it also to its knees.
    So you shrug your shoulders and just forge ahead like you did.

    But what if Unity could just be told to temporarily ignore groups of project folders except the ones associated with your current game? What if Unity would just build a solution file for Visual Studio with just the files needed. And when you want to switch to a different game (or flavor). You just press a button and Unity simply re-instates some folders and ignores a new set. What if this was an integral part of Unity, such that there was a list of your game groups. And all you had to do was click on the new set. And Unity magically switches over? Without copying, moving or altering any of your files? Then if it would build a new solution file then Visual Studio would also switch over and all things Unity would live happily ever after!

    Then you could keep all your shared libraries in the same place, all your packages are left alone (including your custom changes). Updates could be made once knowing that this has been done instantly to all your previous work. Your source control maintenance would be simple, as it see's all your projects (even if Unity does not). If you wanted to do a Global search and replace (to all your projects at once) you could!!!! Such as renaming a namespace no problem. All without bogging down either Unity or Visual studios (or yourself). You could switch between all your games (without exiting) in seconds and verify that all is well after a major change. And if there is some major update to Unity (and associated packages) you can just cycle through your games and verify everything works.

    I have a tool in the asset store called ProjectMaster which solves the above problem. It will manage your projects asset folder by ghosting and unghosting sets of folders based on a selected list that you have picked from the project view.


    Each item in the list describes a set of folders in your project view that is needed for your game. By selecting a game, you force unity to ignore (or not ignore) code,art and packages. Unity will immediately re-scan reload a new project view and rebuild the solution file, and load the scene file. You will never have any code conflicts or bog down unity because unity sees only what you want it to see. Yet all the files are 100% there, your source control will see them, your favorite editor will see them (if you want it to).

    There is also a separate feature for dealing with asset store packages

    And is also part of a package I wrote called PackageMaster.
    You may on the fly turn on and off Packages. (As well as code you wrote). This works in harmony with ProjectMaster as it will use this also when selecting the game you want to work on.

    Here are the links to the packages.
    http://u3d.as/oHp ProjectMaster
    http://u3d.as/qmk PackageMaster

    And in my opinion this is an optimal solution for game development teams who work on several games at once.
    I also have an entire Source revision control system that is specifically designed for Unity game development

    http://u3d.as/sYo Chronos Time Machine
     
    Last edited: Jun 15, 2016
  13. mthorade

    mthorade

    Joined:
    Jan 26, 2017
    Posts:
    3