Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Feedback Zip cache library take too long and that take up build time

Discussion in 'Unity Build Automation' started by Thaina, Jun 12, 2023.

  1. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,049
    Not only it waste my build minute quota, it also unnecessary delay my time to get the build result

    upload_2023-6-12_16-35-28.png

    upload_2023-6-12_16-35-35.png

    Two image above is difference build but take almost the same zipping time

    Not using cache also not an option. Reimport whole project making build process a lot longer. So are there any way to make cloud build not doing the zip process?

    I wish it should cache library as is instead of zipping, or at least use faster zip algorithm
     
  2. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    240
    You can disable compression at the end by making an API call to update the target config.

    This may result in reduced time spent zipping the Library at the end of a build and potentially faster performance when restoring the cache. Updating the cache compression isn't available in the UI but can be configured via the Cloud Build API.To enable no compression make a call to the /orgs/{orgid}/projects/{projectid}/buildtargets/{buildtargetid} endpoint with the following JSON payload:

    Code (Boo):
    1. {
    2.   "settings": {
    3.     "advanced": {
    4.       "cache": {
    5.         "compressionMethod": "none"
    6.       }
    7.     }
    8.   }
    9. }
     
    Thaina likes this.
  3. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,049
    Thank you very much. I would try that

    Hope this config will be available in the UI soon
     
  4. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,049
    Want to add that this API require PUT request with `Content-Type: application/json`
     
  5. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    240
    I can't go into details right now but we have a ton of things planned for the second half of this year around caching that should give everyone more control over their cache, including compression levels
     
    Thaina likes this.
  6. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,049
    I have set compressionMethod as this (this screenshot are from GET api)

    upload_2023-6-14_16-47-40.png

    But there still zipping time at the end of building

    upload_2023-6-14_16-48-34.png

    Also, I have use my local git server for cloud build. But it seem there are no caching for project and need to clone whole project, I have expect that unity would cache git and only pull changes for each success build. Are there any plan for that or I need to move to unity SCM?

    upload_2023-6-14_16-52-57.png
     
  7. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    240
    The zip at the end is creating a zero compression archive of the files for storage.

    The cache is purely the Library folder. This is the same for all source control systems although I do recommend looking at UVCS as we are working to provide very tight integration with it.

    As I mentioned in my last post we are working on some enhancements but I cannot go into details right now.
     
  8. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,049
    Thank you very much for your explanation

    Sadly this mean unity cloud build are still not advantageous for large project. There are inevitably zipping and loading time for library cache which is even larger than project itself (On the contrary, caching whole project along with library would be beneficial if it would make pulling process)

    I wish the system would changed to be some kind of mounting virtual disk, and so it would have both project and library ready for pull the changed and incrementally build project
     
  9. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    240
    I agree that there are limitations with our current implementation. Investigating additional means of attachable storage, etc. are on our roadmap. We want to provide a fast and reliable service that is catered towards the needs of large project that are unique to RT3D.
     
    Thaina likes this.