Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Unite 2023 Registrations are now LIVE!

Full project sync every time?

Discussion in 'Unity Build Automation' started by golonko, Nov 9, 2014.

  1. golonko

    golonko

    Joined:
    Nov 6, 2012
    Posts:
    13
    Hi,

    It seems to me like every time a build is made, UCB does a force sync on the whole repository.
    My assets folder is currently at around 1,5GB (and i suspect it is on a small side), so the sync itself is taking between 30 minutes and an hour. Even when the only change is a single source file.
    This means that the "Cache Library" setting doesn't make any big difference.

    Can anyone explain why the full sync every time?

    I've also noticed UCB tends to create new perforce workspaces even while working on the same platform (so i have UCB-default-ios--xxx10, UCB-default-ios--xxx14 etc...
    I can understand separate workspaces for different platforms as UCB can run the builds simultaneously (although it would be nicer if you simply made local copies of the repo), but why for the same platform?
    We're currently using a free version of perforce with 20 workspaces limit and as it is we'll be unable to use UCB long-term due to this limitation.
    Multiple workspaces would also explain full repo syncing, but it happens even when UCB decides to reuse an old workspace.
     
    golonko, Nov 9, 2014
    #1

  2. hypeNate

    hypeNate

    Unity Technologies

    Joined:
    Apr 4, 2014
    Posts:
    272
    Hi @golonko - Yes, since Unity Cloud Build does not cache your repo data we have to do a full sync for every build. The speed of this sync is dependent on the speed and accessibility of your repo host. We’re working on some experimental features that might assist customers who are experiencing long sync times, so please file a support ticket with us if you think you might need this feature.

    Regarding the workspaces: yes, we are aware of this issue and are working on a fix that will improve how Unity Cloud Build handles and works with workspaces in Perforce.

    It never hurts to file a support ticket about these kind of issues, since it gives us a way to track problems and reach out to you when improvements are made. Thanks for trying the service while it’s in beta!
     
    hypeNate, Nov 13, 2014
    #2

  3. fischer_ag

    fischer_ag

    Joined:
    Nov 19, 2014
    Posts:
    23
    Our build time is >30min. It's really too long! E.g. the same builds in TeamCity take 3-7 minutes!
     
    fischer_ag, Nov 19, 2014
    #3

  4. hypeNate

    hypeNate

    Unity Technologies

    Joined:
    Apr 4, 2014
    Posts:
    272
    Hi @fischer_ag - are you hosting your own source control server inside a local network? Or is it a large provider like GitHub / BitBucket, etc?

    If you can describe your situation we might have some options that would help. Thanks!
    (If you'd rather not go into details publicly you can also file a support ticket at https://build.cloud.unity3d.com/support/ )
     
    hypeNate, Nov 19, 2014
    #4

  5. StarManta

    StarManta

    Joined:
    Oct 23, 2006
    Posts:
    8,738
    I'm experiencing the same. Our projects are hosted on BitBucket.
     
    StarManta, Nov 19, 2014
    #5

  6. deekpyro

    deekpyro

    Joined:
    Oct 16, 2012
    Posts:
    71
    So the consensus is that BitBucket is slow for pulls?

    EDIT: I did a test where I took a project that takes 20-30 minutes to build on iOS (with caching) from BitBucket and moved it over to Unfuddle. I'm getting the same 20-30 iOS build times with that service as well :/
     
    Last edited: Dec 3, 2014
    deekpyro, Dec 2, 2014
    #6

  7. deekpyro

    deekpyro

    Joined:
    Oct 16, 2012
    Posts:
    71
    I don't understand.

    If build times are expected to take 30+ minutes (for projects larger than 1GB, which are nearly all of mine) then why not make Unity Cloud Build an application users can buy (or package it in with the team license) and run it as a service on their own hardware? Then projects don't have to be downloaded every commit and developers that rely on Unity for their livelihood can have the best tool for continuous integration without compromises.
     
    deekpyro, Dec 5, 2014
    #7
    grrava likes this.

  8. uniMaxi

    uniMaxi

    Joined:
    Mar 10, 2013
    Posts:
    15
    For various reasons, i'd rather have my entire project fully in the cloud, with building, deployment on Web etc.

    Right now, i'm using Dropbox and a 4th party tool (synkee) to automate my ftp deployments. This involves
    1) downloading the entirety of new build
    2) uploading the entirety of new build to Dropbox
    3) have synkee catch change in Dropbox and pass it to ftp

    Atm, my builds are ~less than 20Mb in size, so this transferring around is a non-issue. But it might become an issue as this gets larger.

    Don't see an easy solution as long as the output of Unity Cloud Build is a single file, though :(
    Push-to-FTP feature would go a long way in helping out, though
     
    uniMaxi, Dec 5, 2014
    #8

  9. hypeNate

    hypeNate

    Unity Technologies

    Joined:
    Apr 4, 2014
    Posts:
    272
    Hi @deekpyro - yeah, that sounds like a normal build time. If you build for multiple platforms at the same time on Unity Cloud Build, you'll see an overall increase in the total amount of time saved. Plus all the other benefits it provides: distribution, advanced settings, etc!

    However, we're always working on ways to speed up build times, and as the service matures I'm sure we will be able to offer improvements in that area.

    @uniMaxi - are you currently using Unity Cloud Build? If so, what source control setup are you working with? Can you explain what you mean when you say "Don't see an easy solution as long as the output of Unity Cloud Build is a single file, though"?
    EDIT- never mind, I just saw your other thread! I replied there.
     
    Last edited: Dec 5, 2014
    hypeNate, Dec 5, 2014
    #9

  10. v01pe_

    v01pe_

    Joined:
    Mar 25, 2015
    Posts:
    70
    I realize this is a very old post, but the behavior is still the same… the whole repository is pulled for every build, even if there are multiple platforms built on the same branch/commit. This seems like a huge waste of resources, also on Unity's end. Is there any way to build multiple platforms on the same commit w/o the full clone every time?

    One strange thing I see in the full log, even though it was built in the cloud ~4h ago:

    First time build. Skipping changelog.​

    We use a GIT repository hosted on our own server, we cloud build automatically on a branch that is only merged into if we want new builds. The repo size is about 1.5 GB.
     
    v01pe_, Aug 16, 2016
    #10