Search Unity

Feature Request Build Automation Feature Requests

Discussion in 'Unity Build Automation' started by wrossmck-unity, Nov 13, 2023.

  1. wrossmck-unity

    wrossmck-unity

    Unity Technologies

    Joined:
    Nov 9, 2021
    Posts:
    106
    Hi All,

    I wanted to start a new thread on Feature Requests now that we have launched Build Automation. So please use this thread to ask for specific features, improvements, and requests.

    The team and I will be looking at this thread (and the forum at large) actively but I will be locking the old feature request thread since that has been around for years and many of the requests in there have been resolved.
     
    wrossmck-unity, Nov 13, 2023
    #1

  2. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Not sure this would counted as feature. But I want the build system changed to be git on virtual disk

    I have tried to inquire the problem in this post > https://forum.unity.com/threads/zip...that-take-up-build-time.1447591/#post-9081787

    To be summarized I just want the system to rework for these problems
    1 - The need to zipping and loading Library folder. For the large project this process waste too much time
    2 - The system also need to pull whole project from repo for each run, which also waste too much time and resource. Especially when internet to our repo server are not great
    3 - Rerunning just need to do this whole process again

    All these burn too much build minute over the build process itself

    The solution I can think of is, if we use git repo as project source, unity should store both library folder and project repo together like the actual project in our machine. And put those all in virtual disk that can be mounted/unmounted on the server

    Also project should be stored as git repo (just shallow clone would be enough?). So it could be pulled to update more efficiently

    Since the library folder most likely larger than project itself so it has not much point to store library folder but not the project

    With this system I propose, the rerunning also would be a lot faster

    ps. I don't think my request in the old thread has been resolved as of yet, hasn't it?
     
    Thaina, Nov 14, 2023
    #2

  3. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    303
    An improvement of build speeds are the number one requested feature from our users. We are actively looking at ways to solve this problem. The solution you propose is one of many that we are investigating. We are analyzing timing data to determine which portions of the build process are the slowest. From this data we are determining what solutions we can implement that would improve the speed of these various processes. For example, if the majority of time is spent importing assets and building the player, that would be an area we might target improvements on before we hit on things like caching.

    To confirm, we are looking at longer term persistent "block" storage instead of using Library/Workspace (object storage) caching on a per target basis. We don't have a timeline on when this would be implemented, or if it provides the speed benefit we want.
     
    Benjamin-Gooding, Nov 15, 2023
    #3
    RyotaMurohoshi and Thaina like this.

  4. mike6502

    mike6502

    Joined:
    Oct 5, 2016
    Posts:
    49
    I'd love to have the ability to test post-build deployment scripts without having to kick a full build of the Unity project.
     
    mike6502, Nov 17, 2023
    #4
    IOU_RAY, RyotaMurohoshi and crawfis like this.

  5. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,168
    Thaina, Nov 17, 2023
    #5

  6. wrossmck-unity

    wrossmck-unity

    Unity Technologies

    Joined:
    Nov 9, 2021
    Posts:
    106
    @Thania we are investigating adding logic to those steps such as retries on failure.

    @mike6502 I'm not sure we will be able to make a way to test post-build deployment scripts without having to launch a new build, but if we get the step logic separated out into discrete steps which can be re-ran independently that might help you with the problem you're having. I'll think on it some more and see what we can do to help. Are you specifically interested in testing the code that you supply? can this be done locally? or are you more interested in testing if the code you supply working in the build environment? I do plan on being way more transparent about the execution environment and what's installed/available on the build machines etc, so again that might help you out.
     
    wrossmck-unity, Nov 23, 2023
    #6
    Thaina likes this.

  7. mike6502

    mike6502

    Joined:
    Oct 5, 2016
    Posts:
    49
    Thanks @wrossmck-unity, yes, I'm interested in validating and fixing my own deployment scripts in the build environment. I definitely test scripts locally, using as best-as-I-can simulation of the build environment. But this does not guarantee that everything will work in the real pipeline. For example, I'm on a Mac and cannot test cygwin. Another example: I had a script fail on a windows builder because it couldn't find npm. Another: I had a typo in an environment variable, and there was no way to know until the build finished and I could inspect the log files. etc.
    Most of the time the fix steps are 1) figure out why the server is different than local (paths, missing tool, etc) by checking build logs and forums/blogs, 2) make trivial changes to the deployment scripts, commit/push/merge and try again. When the billing was flat rate, this was inconvenient. Now with metered billing, this is costly.

    More transparency about the build machines and environment would be very helpful.

    I should also note that my deployments might be pretty common for mobile developers:
    * Deploy to Firebase App Distribution (firebase-tools via npm)
    * Deploy to Google Play Store (shell script with fastlane)
    * Deploy to App Store Connect (shell script with xcrun)
    You might want to consider offering built-in versions of these deployments, in which case mobile devs like me would not need to write or test our own scripts.
     
    mike6502, Nov 24, 2023
    #7
    Thaina likes this.

  8. wrossmck-unity

    wrossmck-unity

    Unity Technologies

    Joined:
    Nov 9, 2021
    Posts:
    106
    @mike6502 Appreciate the response and the details! I think I understand what might help here. We have an idea to run arbitrary scripts in the same execution environment that builds run in, which should enable much faster turn around that waiting for a build to complete first. might not give you 100% confidence but should decrease the iteration time (and cost!) significantly. Something similar to this is being worked on today and might be public facing around summer (sadly cannot promise this though!).

    Totally agree on adding dedicated support for common deployment targets such as the ones you listed and steam, itch.io, etc. It's definitely on the list of things we want to invest in, although not actively being worked on today. We're currently doing our initial planning for next year, so I'll bring this to the team and see what we can do.
     
    wrossmck-unity, Nov 24, 2023
    #8
    Thaina and mike6502 like this.

  9. Myron

    Myron

    Joined:
    Oct 21, 2012
    Posts:
    8
    Configure a maximum number of builds to keep for each Build target configuration.
    For example I want to keep the 50 last releases but only the 5 last nightly and commit builds. Currently we don't use our nightly or commit builds because we don't want to have a lot of builds to clean manually and we want to keep our storage cost as small as possible.

    Allow to use CLI tools from Steam and the Meta Quest store so we can use them in a post-build script to publish automatically to a branch any successful build
    To do a proper continuous integration you need to be able to deploy your build somewhere automatically and now publishing still requires us to download the apk and to use the command line on our computer to upload it
     
    Myron, Dec 1, 2023
    #9

  10. wrossmck-unity

    wrossmck-unity

    Unity Technologies

    Joined:
    Nov 9, 2021
    Posts:
    106
    @Myron yes absolutely agree on the build retention settings to help manage storage. it's on the list. We want to do number of builds per config as well as an overall sum of GB (maybe useful per project/org) too. For now this is only achievable via the API, only works based on time/date instead of number of builds, and is not branch/config aware - so it applies to all builds across the whole project. we will improve this later.

    Code (JavaScript):
    1. Do a POST request to /api/v1/orgs/{ORG_ID}/projects/{PROJECT_ID}/artifacts/delete/?allWithExclusions=true
    2. with a body of:
    3. {
    4.     "excludedBuilds": [],
    5.     "filters": {},
    6.     "deleteOnOrBeforeDate": "2018-10-01T18:51:33.412Z"
    7. }

    Also completely agree on the stores deployment topic. I'll add the meta quest store to our list for integrations
     
    wrossmck-unity, Dec 1, 2023
    #10

  11. crawfis

    crawfis

    Joined:
    Jan 30, 2014
    Posts:
    114
    I need a way to sign a Windows build. Not sure that exists currently. Documentation exists for macOS in the manual, but not Windows.
     
    Last edited: Dec 2, 2023
    crawfis, Dec 1, 2023
    #11
    LilGames likes this.

  12. mark-007

    mark-007

    Joined:
    Mar 4, 2015
    Posts:
    27
    Would really love to see a supported (automated) way of pushing builds to pretty common external locations such as S3 post build
     
    mark-007, Dec 14, 2023
    #12

  13. troylightfox

    troylightfox

    Joined:
    Feb 1, 2023
    Posts:
    30
    I'd like to see a unique build machine/container ID of some sort in a build overview. We sometimes get 2-3 spurious failures in a row(i.e. failed to download a stock Unity package, locked file when building addressables, etc.) and it'd be useful to know if it's the same build machine/container each time. Would also be useful for build timing to see whether consecutive non-clean runs are actually running in the same environment or not.
     
    troylightfox, Dec 22, 2023
    #13

  14. RyotaMurohoshi

    RyotaMurohoshi

    Joined:
    Jun 20, 2013
    Posts:
    24
    Would you support officially UnityCloudBuild Manifest.json file at "Assets/UnityCloud/Resources"?

    With old Unity Cloud Build, JSON file named "UnityCloudBuild.json.txt" is put on "Assets/UnityCloud/Resources". I used it. I displayed some this file content on my mobile game to show the build version and build number for debug.

    This is Japanese document: https://docs.unity3d.com/ja/2021.3/Manual/UnityCloudBuildManifest.html
    This is Korean document: https://docs.unity3d.com/kr/2021.3/Manual/UnityCloudBuildManifest.html
    This is Chinese document: https://docs.unity3d.com/cn/2021.3/Manual/UnityCloudBuildManifest.html
    English document may be deleted.

    With new Unity Build Automation, there are no documentation about "UnityCloudBuild.json.txt".

    By the way, I tried to load "Assets/UnityCloud/Resources/UnityCloudBuild.json.txt" with Unity Build Automation, I could get some target information. I think Unity Build Automation seems support "Assets/UnityCloud/Resources/UnityCloudBuild.json.txt". But I think this is not officially.

    Would you support officially UnityCloudBuild Manifest.json file at "Assets/UnityCloud/Resources"? And would you document it like old Unity Cloud Build?
     
    RyotaMurohoshi, Feb 11, 2024
    #14

  15. mark-007

    mark-007

    Joined:
    Mar 4, 2015
    Posts:
    27
    In our application we need to know the build number and branch (so we can determine release track for our app, cos we have built an auto updater system) so we hacked it to save UCB_BUILD_NUMBER (and other build time env vars to a file in streaming assets which we then load and query at runtime to know the exact build we're running.. but that feels like a hack. I'd wondered if if would be better for unity to automatically save build time ENV vars eg things like like "SCM_BRANCH", "UCB_BUILD_NUMBER", "BUILD_TARGET", "PROJECT_ID", "NUMBER_OF_PROCESSORS" to the build automatically somehow so that they can be queried as env vars at runtime?
     
    mark-007, Feb 20, 2024
    #15