Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Excluding folders from build

Discussion in 'Editor & General Support' started by YondernautsGames, Jun 1, 2016.

  1. YondernautsGames

    YondernautsGames

    Joined:
    Nov 24, 2014
    Posts:
    354
    Hey all

    I just posted a suggestion for something I would find super useful and I'm sure many others would:
    https://feedback.unity3d.com/suggestions/allow-tagging-folders-for-exclusion-in-builds

    An example of where it would be useful is, I have a project that I want to release on the Amazon App Store and Google Play. If I add plugins to enable various store / platform specific features then the only control I have is to say these plugins are Android. Thing is, I don't want any trace of the Amazon plugins in the Google Play build and vice versa. Now imagine that I also want to release a paid version and a free ad supported version of the game. I want the paid version to have no on-line requirements, and no funky permissions due to ad libraries or social services. How to do that:
    1. Maintain separate scenes for each and trust that the build process will chop out anything that isn't referenced. I'm totally happy relying on this for assets like textures, audio, etc. Not so much for plugins. I've recently released a game that used a popular Android plugin for a few minor platform specific features. In the Google Play developer dashboard I tick that this game does not use adverts, and it immediately pops up a warning saying "well your APK contains ad libraries, so we ain't inclined to believe ya".
    2. Maintain multiple projects and use version control to sync up shared folders. This is how I handled my last one. I use externals to reference shared folders containing all my game's scripts and assets that aren't plugin based. It works, but it's super clunky. It also means it takes up way more space on disk and requires way more time transferring changes to & from VC. It falls apart at the slightest provocation (possibly because I use SVN for version control), and it's not hard to get my shared folders out of sync either through forgetfulness or some other user error.
    3. Write build scripts that copy/paste or rename folders so they are excluded. No. Just no.
    If there's some editor feature that I've missed that makes this super easy then please chime in. In that case I'll delete the suggestion and throw a party for whoever sets me straight.

    Anyhow, how I'd see it working:
    • You define tags somewhere in the project / build settings (probably Project Settings > Editor). Similar to how you define custom layers or object tags.
    • The inspector for a folder in the project view has a drop-down where you select which tags apply to each folder (defaults to all). Saves to meta as you might expect.
    • In the build settings you have a drop-down where you select which tags to build.
    • Additional member in BuildOptions for people that want to control the build pipeline through scripts.
    From the outside looking in, it seems like it would be a relatively simple feature to implement (I may be very wrong). It would make a huge difference to the neatness, flexibility and maintainability of my projects, though.

    Chris
     
  2. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    AFAIK, there is no built-in or a plugin that does this at the moment.

    We have also faced a similar requirement working on our previous project: our game had to be built in a few different variations. By variation, i mean different platforms, and different configurations per platform.

    This meant we had to configure assets, scenes, plugins, etc per every build variation.

    There are different solutions your can do to work around this issue, which will probably cover most of what you need. For example - for scripts, it's pretty easy to differentiate code per variation using scripting defines (e.g: #if AMAZON).

    What you'd like to do (include/exclude certain folders) is not possible with how Unity works. I've read about some solutions that suggest that if you move content into a particular folder (.tmp i think) it will be excluded but i never tried this.

    What we implemented was a system we called "Workspaces". It used symbolic links, which is supported both on Windows and OSX. Essentially, we created a bunch of folders outside of the Assets folder. Each folder was assigned a "tag" like you suggested. When setting your project to a certain "workspace", the system we built would create symbolic links inside Assets to all folders that belong to that tag, so the editor "thinks" these folders are part of the project and include them. At any point you can switch to a different workspace and it would reset the symbolic links and include another set of folders.

    It worked quite well though it did had a few shortcomings.

    Maybe this gives you an idea on how you can solve this issue yourself for now, until Unity will support this out of the box.
     
    Last edited: Oct 5, 2017
  3. YondernautsGames

    YondernautsGames

    Joined:
    Nov 24, 2014
    Posts:
    354
    Hence why I posted an idea. If it seems interesting and useful to you then give it a vote so that the Unity team can see it would make a difference.

    I'd be interested to hear what the shortcomings are? I tried similar with SVN externals, but it was far too unwieldy to really use in production. Switching between configs was a complicated error prone process that could take quite a bit of time. Plus when you actually move files / folders around switching back to the editor then means it has to reimport all scan through all those files and reimport a number of them.

    Thanks for the suggestions anyhow. I'll have a dig into symbolic links and .tmp folders and see what difference they make.
     
  4. andymads

    andymads

    Joined:
    Jun 16, 2011
    Posts:
    1,614
    See here about using the WebPlayerTemplates folder for scripts you don't want to compile. Probably not what you want but worth knowing.
     
  5. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    The shortcomings (as far as i can remember them) were partly related to our implementation (bugs, etc).
    Also, we had to decide whether we commit the currently active "workspace" to source control, or whether we should leave that as a setting that is relevant per developer.
    The solution itself worked - it did not involve any copying of folders; it only created a link to the actual folders. No copying/moving was involved.

    After modifying your current workspace, Unity would reimport all things that were changed, since as far as the editor is concerned, the contents of the projects got updated. This can be a lengthy process, all depending on the content you include per platform.

    For us, the game itself did not change apart from a view popups / images. You would generally want to keep most of your project common between all different flavours, and only change very small parts of it, in order to keep maintenance to a reasonable cost (of developer's efforts and time).
     
  6. EddieOffermann

    EddieOffermann

    Joined:
    Sep 13, 2015
    Posts:
    13
    I just upvoted the suggestion. I have a pretty good workaround but it's not ideal.

    Currently, I handle this through organizing my hierarchy so that conditional compilation is accomplished by disabling entire hierarchies from the top level. You can add/remove a tilde (~) from the end of the directory names to suppress visibility to Unity. Unity will entirely ignore all variety of assets below those directories, scripts, geometry or anything else.

    This isn't hard to make part of your build process if you have a little bit of automation. When you add the tilde, all scripts in the project are immediately recompiled. To remove it you'll need to go into your platform's file manager, though, unless you build a tool for it.

    I use it to simplify building for the Moverio glasses (on Android) and the HoloLens (on UWP).

    There really should be a tool inside Unity to do this though.
     
    DungDajHjep, ModLunar, IARI and 6 others like this.
  7. Beguiled

    Beguiled

    Joined:
    Mar 15, 2014
    Posts:
    14
    Thanks for the tips, EddieOffermann. It helped me get over my hurdle of deploying to multiple platforms. I agree that Unity should be doing this by default and invisible to us (aside from elements we add to our scenes).

    For those who want to know more about the tilde (~) method, you should review this document:
    https://docs.unity3d.com/Manual/SpecialFolders.html
     
    tarkhon and IARI like this.
  8. samizzo

    samizzo

    Joined:
    Sep 7, 2011
    Posts:
    487
    Thanks - that tilde trick is great to know!
     
    tarkhon likes this.
  9. sillyplayground

    sillyplayground

    Joined:
    Aug 27, 2014
    Posts:
    20
    How does the tilde trick work with revision control? Wouldn't each rename count as a change?
     
  10. samizzo

    samizzo

    Joined:
    Sep 7, 2011
    Posts:
    487
    I wanted to rename as part of a build process, so it wouldn't affect source control for me. I would rename, do the build, then rename back.
     
  11. greg-harding

    greg-harding

    Joined:
    Apr 11, 2013
    Posts:
    525
    We tested hiding folders again in 5.4 and 5.5 a while ago using the method of renaming with ~ but Unity's metadata didn't work properly and case-related errors appeared in the Editor (OSX) console about half the time making it unusable. We filed a bug which was accepted by QA but hasn't gone any further yet.
     
  12. Florian-Nouviale

    Florian-Nouviale

    Joined:
    Mar 14, 2013
    Posts:
    54
    I'm interested in this too.I would love for Unity to have an option on folders like it has on plugins, allowing to exclude/include them depending on the platform !
     
  13. eppz

    eppz

    Joined:
    Aug 2, 2014
    Posts:
    172
  14. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I actually tried using that but it didn't work out so well. I advise you to do the same.

    For example - our project took a very long amount of time to switch platforms due to many art assets. I've tried moving some of the unused stuff to a folder that starts with ~ to avoid having those assets imported, but Unity still processed them (at least, from what i can remember).

    We ended up not using this approach at all.
     
  15. eppz

    eppz

    Joined:
    Aug 2, 2014
    Posts:
    172
    I'm using it for plugin development just fine: native iOS / Android plugin projects can live within folders starting with ~.

    For asset management there should be different answers. Starting with "why you keep unused stuff around" at all? :D
     
  16. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Consider a case where the project contains multiple "variatons" or "flavors". For example : paid vs. free versions, different platforms (under Android there could also be Google play / Amazon versions), etc.

    Having a solution that lets you configure different "active" assets according to the active variation is a nice feature.
     
  17. eppz

    eppz

    Joined:
    Aug 2, 2014
    Posts:
    172
    You'll need an Editor Tool for this anyway.
    Moving files around or suffix folders with ~ makes not much difference.
     
  18. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    of course, but the underlying mechanism for this tool isn't the ~ prefix for folders. We eventually went with something else which i previously wrote about in this thread.
     
  19. eppz

    eppz

    Joined:
    Aug 2, 2014
    Posts:
    172
    Actually I did not read the entire thread, but the title.
     
  20. crdmrn

    crdmrn

    Joined:
    Dec 24, 2013
    Posts:
    152
    They have to END with "~" not start :D
    As the quoted page states:

    "Files and folders which end with ‘~’."

    Besides that, does anyone knows if those files get still included in the build (like the streaming assets) or if they get ignored for builds as well (which is what I'm looking for)?
     
  21. ZanLeO

    ZanLeO

    Joined:
    Jan 17, 2013
    Posts:
    1
    Checked in Unity 5.6.3f3
    file Assets/Resources/Textures/LoadingScreen/LoadingScreen~.png
    is included in build ((
     
  22. comworm

    comworm

    Joined:
    Aug 13, 2014
    Posts:
    11
    It's out dated, but I want to leave my case. According to Unity document, I could exclude my 'Resources' folder when building, by moving 'Resources' folder into 'Editor' folder. It solved my headache problem.
     
  23. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,307
    You probably should have - it's only 22 postings, not one of those 22 pages threads ;-)

    @EddieOffermann posted about the tilde(~)-solution here, and @Beguiled followed up with a link to the documentation.

    But I'll admit - when I first saw this thread I also missed that important piece of information and went right to the entry on Unity Feedback to support it ... but maybe this was a good thing because I eventually found a solution which works extremely well for me and is not only compatible with, but actually based on version control (so if you're not using version control, or, more specifically, if you're not using Git for version control, you're probably out of luck for this one):

    Git has kind of an esoteric feature called sparse-checkout. What this does is that it lets you define only specific folders to check out from a repository. Depending on how you have set up your project, you'd then either use this (tell it which folders you want), or use /* and then !/... for the folders you want to exclude.

    In my situation, I have one main development machine with the complete project, and one iMac build machine, and another PC for doing Android builds. But obviously, you could achieve the same on just a single machine by checking out the project folder three times, and then synchronizing the folders via git. I have been using this workflow for about two weeks now and am super-happy with it.

    Just be a little careful when you already have the project checked out, and it's a very large project (I'm not that proficient in using git-command line and caused a situation where I had to re-get the whole project which takes almost an hour, and I was kind of scared I messed up that project on that machine during that time ;-) ).

    I started out simply deleting the folders and ignoring the huge number of deleted files in version control. sparse-checkout then remedied that.

    So here's how I went about it:
    1. Obviously, you need to use Git for version control, with git-lfs for the large files.
    2. In a "safe clone" (that you feel comfortable messing up), delete the folders that you believe you don't need.
    3. Put the path of each folder into the sparse-checkout file, which is located in <projectFolder>\.git\info. It has no file-extension, the full name is really just sparse-checkout and it's a simple text-file. See below for an example.
    4. In a command line or terminal, go into your project folder and execute the command git config core.sparsecheckout true
    5. Then, do git reset, then do git checkout.
    6. Problem solved :)
    7. Copy your sparse-checkout into the root of the project - that way, you can version control it, and copy it to the "active location" in clones of the project where you need it. Just don't forget doing git config core.sparsecheckout true in those clones - ideally before checking everything out.
    8. You might even have different versions of sparse-checkout for different project configurations.
    9. You might need to cut some dependencies by using conditional compilation (I use this in a pretty large project with lots of dependencies). The only thing to be careful with here is making sure you don't accidentally revert such changes while experimenting (I had 20,000 files deleted at one point - very easy to miss real changes when you revert, then)
    Sparse checkout example:

    Code (csharp):
    1. # see also: https://stackoverflow.com/questions/51241736/how-can-i-hide-not-ignore-deleted-files-in-one-clone/51243634#51243634
    2. # put into \.git\info
    3.  
    4. # Only on build machines for iOS and Android (no need for CSCore and Morph3D there)
    5.  
    6. /*
    7. !/Assets/Morph3D/
    8. !/Assets/Morph3D.meta
    9. !/Assets/MORPH3D/
    10. !/Assets/MORPH3D.meta
    11. !/Assets/Plugins/3rdParty/CSCore/
    12. !/Assets/Plugins/3rdParty/CSCore.meta
    13.  
    What does this do? /* includes everything, then the !/Assets/... lines include what I need. As mentioned above, you could also just include the folders you need, whichever is more convenient in your case.
     
  24. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Even though this thread is around 2 years old, it's still relevant today. I actually had to revisit this recently in a project and i believe this can actually be solved in a nice way using editor scripting.

    I implemented a small system for supporting what @Survivalist-Games initially wanted. In case someone is still interested i can share some details. I think it can also make up for a nice asset store package for people to buy (or get for free, we'll see).
     
    Last edited: Jul 19, 2018
    jashan likes this.
  25. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,307
    Sounds cool, I'm definitely interested. One nice thing about physically excluding the folders (see my posting above) is that the "sparse projects" are much smaller and therefore much easier to work with, and I see anything that might break immediately during edit time (some things still only become obvious when doing an actual build but only comparatively few).

    But if you can handle @Survivalist-Games idea using editor scripting and make it open source, I'd be happy to add code that auto-generates the sparse-checkout file from that information (could also go the other way round and apply a tag by parsing a sparse-checkout file).
     
  26. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I am actually using "sparse checkout" but for totally different reasons. My main problems with this approach are that
    1. It is specific to a particular VCS system (git), although it may exist in other systems as well.
    2. You really have to re-structure your project to efficiently support this. this may not always be possible (for example - plugins that get installed under a specific folder, for example).

    Ideally, i would just want to "not checkout" the "Amazon" folder and be done with it. if i have to start picking folders that will not be checked out, it gets slightly more complicated.
     
  27. LouisDelassus

    LouisDelassus

    Joined:
    Jul 10, 2017
    Posts:
    3
    I'm interested. I did find a solution too but I don't think it is a really a good one.

    I have created an "_Assets" folder in which I have added a folder for each platform. Then I have created symbolic linked folders, for each of one of those, inside the "Asset" folder.

    To exclude folders, I did a little script using the CompilationPipeline event, and renamed the symbolic linked folders with the "~" character regarding of the value of EditorUserBuildSettings.activeBuildTarget.

    @liortal Can you share some details of your solution ?
     
  28. jeromeWork

    jeromeWork

    Joined:
    Sep 1, 2015
    Posts:
    429
    @liortal I'd love more details too, or would be interested in a paid asset. Would be very useful.
     
    liortal likes this.
  29. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    This is a very old thread. The system i described back then worked for us (in 2016), what more info would you like to get?

    It would be best if you described your exact requirements or what you're trying to achieve, then maybe i can help you out better in finding the exact solution.
     
  30. jeromeWork

    jeromeWork

    Joined:
    Sep 1, 2015
    Posts:
    429
    @liortal I guess we're just following up on your post from this July; "I implemented a small system for supporting what @Survivalist-Games initially wanted. In case someone is still interested i can share some details. I think it can also make up for a nice asset store package for people to buy (or get for free, we'll see)."

    I tried the tilde hack but it doesn't really work for me.Broke stuff as some files/assets were excluded while others were not. I found it easier to just cut and paste entire folders out of the Asset folder. Not ideal, so when you mentioned you had a solution that might even be turned into an asset, I got interested :)

    The reason I'm looking for something like this is that as I'm optimizing I'm finding a bunch of stuff from stray packages are making it into my build. Nested Resources folders, shaders, that kind of thing. I'd love to be able to disable things without having to uninstall, to try different variations and compare performance.
     
  31. StefanoCecere

    StefanoCecere

    Joined:
    Jun 10, 2011
    Posts:
    211
    i think a native solution to exclude some Resources folders at build time via script is very important.
    i subscribe to this thread just for updates
     
  32. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I think that some "workaround" can be developed that is fairly simple (until official Unity support rolls out, if ever).

    @StefanoCecere let me know if you're interested in hearing more about that (are you only after removing a set of pre-selected folders from the build?)
     
  33. snoche

    snoche

    Joined:
    Feb 22, 2010
    Posts:
    82
    +1 come one Unity, that should be very easy to implement and very useful for us!
     
    fritzs likes this.
  34. fritzs

    fritzs

    Joined:
    Jul 10, 2017
    Posts:
    4
    +1 we need
     
  35. karrlozrg

    karrlozrg

    Joined:
    Oct 4, 2017
    Posts:
    1
    +1 we also needed
     
  36. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    649
    +500 we need
     
  37. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    375
    yes please
     
  38. krishnanpc

    krishnanpc

    Joined:
    Oct 30, 2017
    Posts:
    19
  39. Tomi-Tukiainen

    Tomi-Tukiainen

    Joined:
    Jun 8, 2013
    Posts:
    28
    Hi, according to the documentation, adding a hidden flag for asset folder also prevents it from being included, just like adding ~ to the end of the name. Using hidden flags may be the more convenient option as the directory won't be moved back and forth in version control.

    That said, excluding assets from builds and then reverting them back can cause problems. As an example, I excluded UMA 2 Avatars from my project's server build and used conditional compilation (#if ... #endif) to remove my UMA-related classes from the server build. Later when I continued working with the client build again, I noticed that component references were broken from prefabs to the UMA-related components that I had not compiled for server build. I had to rebuild the entire Library folder to get them back (Unity 2019.1.8f1) -- took about 30 minutes.

    To keep this simple, I have found that it works best to have "the full master version" in version control. From that you can make platform specific project directories that exclude some parts. As an example, you could make a server build directory that excludes Audio, Animations and Textures with rsync as easily as this:

    mkdir ../[game]-server ; rsync -r -v --delete Assets ProjectSettings Packages ../[game]-server --exclude 'Assets/Audio' --exclude 'Assets/Animations' --exclude 'Assets/Textures'
     
    Last edited: Jul 5, 2019
  40. cdytoby

    cdytoby

    Joined:
    Nov 19, 2014
    Posts:
    181
    OK, I had a solution in Unity 2017 LTS but currently I'm moving our project into Unity 2018 and this solution doesn't work properly anymore. It's related to excluding folder, cloud build, and multi-platforms.

    What I did is write a few scripts about changing settings, changing icons, change compile symbols, and renaming folders that needs exclude. Those folders are simply renamed to "Folder~" in order to be excluded. And it works in cloud build. At the same time, compile symbols and potencial compiling errors MUST be handled correctly, in compile symbol define, and in scripts too. It worked on Cloud Build. And I need to make sure those folders are renamed back immediately after builds in order to prevent errors. All folders are in "Assets" folder.

    But I'm moving the project to Unity 2018 LTS, it seems the behaviour is changed, and I have trouble setting compile symbols during cloud build process. Even the compile symbol is set, the script still consider itself that the symbol is NOT SET. It works on local but not on cloud build.

    Well, still need this feature.
     
  41. fwalker

    fwalker

    Joined:
    Feb 5, 2013
    Posts:
    255
    One more from me ! Please !!!!
     
  42. shwa

    shwa

    Joined:
    Apr 9, 2012
    Posts:
    461
  43. Gasimo

    Gasimo

    Joined:
    Mar 3, 2015
    Posts:
    80
  44. SergioRubio

    SergioRubio

    Joined:
    Nov 7, 2019
    Posts:
    1
  45. Opeth001

    Opeth001

    Joined:
    Jan 28, 2017
    Posts:
    1,126
  46. MAWMatt

    MAWMatt

    Joined:
    Nov 10, 2016
    Posts:
    78
    This would be a great feature!

    In my case, I built support into my game to create and edit levels from the game itself. I have thumbnails I load up from the resources directory that I use during this in-game development process. I use a preprocessor define to know whether I need to include the various options for doing this.

    It'd be great if I could have a pre-build function that checks if that directive is defined, and if not, it marks my particular folder to be not included. For now, I don't see any way around deleting it before I make a real build (there's a lot of content in there not worth ignoring).

    Thanks, hope this gets into the engine soon!
    -Matt
     
  47. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,058
  48. AskCarol

    AskCarol

    Joined:
    Nov 12, 2019
    Posts:
    234
  49. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,058
    I don't use collaborate and the solution to this problem it shouldn't be tied to that service.
     
  50. Shadowing

    Shadowing

    Joined:
    Jan 29, 2015
    Posts:
    1,648
    +100 sucks to find out this doesn't exist :(
     
    ModLunar and VadimPyrozhok like this.