Search Unity

Project Size vs Repo Size

Discussion in 'Unity Build Automation' started by niX_BB, Mar 4, 2015.

  1. niX_BB

    niX_BB

    Joined:
    Mar 29, 2010
    Posts:
    26
    With the beta over the unity cloud build servers have started enforcing the size limit based on your subscription plan. I've run into a situation where the execution of the plans do not seem to match with the description of the plans.

    In my case I'm using bitbucket which reports my git repo as 1GB, correlating with the size of my local repo as reported by 'git count-objects - v'
    The project files in my working copy totals to 1.7GB - excluding the .git folder

    I'm on the Pro Plan which should accommodate a repo size of up to 2GB yet my builds fail with the following error message:
    ! postcheckoutstatus failed. Project is too large to build for this tier of service. (2734132126 bytes exceeds limit by 734.132126 bytes). See https://build.cloud.unity3d.com/plans/ to upgrade.​

    It seems like the size being tested is the total folder size (ie repo and working copy combined)

    The list at https://build.cloud.unity3d.com/plans/ specifically states that the limit is based on 'The size of your repository.'

    If the description is accurate the check should be evaluating the size of the repo itself without the working copy. Alternatively the description should at least be expanded to clarify that it's the combined size of the repo and the working copy.

    Hope this helps anyone else confused by this disparity.
     
    niX_BB, Mar 4, 2015
    #1

  2. super77gg

    super77gg

    Joined:
    Sep 25, 2014
    Posts:
    46
    We are experiencing the exact same problems 700 MB Repo but for some reason builds to larger than 2 GB hitting the 2 GB cap. Would love to see limit increased OR a fix to only evaluate repo size only before building.
     
    super77gg, Mar 4, 2015
    #2

  3. David-Berger

    David-Berger

    Unity Technologies

    Joined:
    Jul 16, 2014
    Posts:
    745
    Thank you for your feedback! The size we measure is a actual snapshot of the project so without the history e.g. within the .git folder when using Git. We currently monitor the repositories and see what goes wrong.
     
    David-Berger, Mar 5, 2015
    #3

  4. topherr

    topherr

    Joined:
    Jun 1, 2013
    Posts:
    25
    Our repo (including history) is 300mb and it's reporting "Project is too large to build for this tier of service.
    (2.08 GB exceeds limit by 1.08 GB)."

    This seems like quite a discrepancy. Has this been resolved?
     
    topherr, Apr 13, 2015
    #4

  5. David-Berger

    David-Berger

    Unity Technologies

    Joined:
    Jul 16, 2014
    Posts:
    745
    This seems wrong. Could you please post your project name so we can have a look?
     
    David-Berger, Apr 14, 2015
    #5

  6. topherr

    topherr

    Joined:
    Jun 1, 2013
    Posts:
    25
    Sure, it's "tms-app".
     
    topherr, Apr 14, 2015
    #6

  7. David-Berger

    David-Berger

    Unity Technologies

    Joined:
    Jul 16, 2014
    Posts:
    745
    Well, you seem to have disabled your ssh key - we can’t rebuild it.
     
    David-Berger, Apr 15, 2015
    #7
    shkar-noori likes this.

  8. topherr

    topherr

    Joined:
    Jun 1, 2013
    Posts:
    25
    I've not changed it, and I just double checked the key is still set for read-only access on the repo.

    When I perform a build, it doesn't give any issue fetching from the repo. I just get this log:

    Code (CSharp):
    1. 1: Running Prebuild steps
    2. 2: done.
    3. 3: bvr 0.2.42
    4. 4: bvr-api 0.0.40
    5. 5: Download of cache file workspace_latest finished, size is 2597685001 bytes
    6. 6: Workspace successfully restored from remote cache
    7. 7: Using strategy: Default
    8. 8: Last Built Revision: Revision 2a53f57da7dc1aad5a6e71fa53768c108f1573dd (origin/master)
    9. 9: Fetching changes from 1 remote Git repository
    10. 10: Fetching upstream changes from origin
    11. 11: Commencing build of Revision 0bb56a0dc4c0d5d7fadc75a4ef6b8f3f6fe9c60a (origin/master)
    12. 12: Checking out Revision 0bb56a0dc4c0d5d7fadc75a4ef6b8f3f6fe9c60a (origin/master)
    13. 13: BUILDJOB_CACHE=workspace
    14. 14: BUILD_PLATFORM=android
    15. 15: CLEANUP_WORKSPACE=true
    16. 16: SANDBOX_BUILD_USER=true
    17. 17: ! postcheckoutstatus failed. Project is too large to build for this tier of service. (2.14 GB exceeds limit by 1.14 GB). See https://build.cloud.unity3d.com/plans/ to upgrade.
     
    topherr, Apr 15, 2015
    #8

  9. David-Berger

    David-Berger

    Unity Technologies

    Joined:
    Jul 16, 2014
    Posts:
    745
    The path you entered is probably not correct. If you use subfolders in your repositories you need to add them in Project Subfolder field in Project Settings. As a test we set it to unity as unity is included in the checkout, but we have no idea how the structure of you repository looks like. You need to specify the direct path to the directory which includes the Assets and ProjectSettings folder! Please let me know if this helps! :)
     
    David-Berger, Apr 17, 2015
    #9

  10. topherr

    topherr

    Joined:
    Jun 1, 2013
    Posts:
    25
    Are you looking at the correct project? Our Project Subfolder is already set, and it's set to 'unity', and hasn't changed since I set it up weeks ago.

    The issue here is the repo is reporting a size in the error log, that it isn't. This project built fine and without changing settings, suddenly the size was too big despite the size being 300mb.
     
    topherr, Apr 18, 2015
    #10

  11. hypeNate

    hypeNate

    Unity Technologies

    Joined:
    Apr 4, 2014
    Posts:
    272
    Hi @topherr - yes, it looks like Cloud Build is seeing a much larger size than it did previously. I assume you haven't recently committed some large files? Maybe do a "get Info" or "properties" of that repo folder and let us know what the calculated size is?

    We will also look into the issue on our end.
     
    hypeNate, Apr 20, 2015
    #11

  12. michielbrinkers

    michielbrinkers

    Joined:
    Jan 17, 2013
    Posts:
    5
    I'm running into the same issue. The repos is about 1.3GB but the build system reports it as 2.6GB, which makes me guess the .svn folder is included in the count, which isn't what I would be expecting.
     
    michielbrinkers, Apr 21, 2015
    #12

  13. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    829
    So, to confirm, Unity Cloud Build is doing a shallow pull without any history past the current state and checking the size of that, not the total size of the repository with it's whole history in .git folders, right?
     
    bac9-flcl, Apr 24, 2015
    #13

  14. David-Berger

    David-Berger

    Unity Technologies

    Joined:
    Jul 16, 2014
    Posts:
    745
    Correct!
     
    David-Berger, Apr 30, 2015
    #14

  15. Twistplay

    Twistplay

    Joined:
    Dec 6, 2012
    Posts:
    36
    I'm currently seeing the message "3.92 GB exceeds limit by 1.92 GB", however our full checkout of our repository data is definitely only 550 MB (I just checked). Is there anything else it takes into account rather than just your checked out assets ?

    e.g.:
    - If building for 2 platforms, is the limit effectively halved ?
    - Does it take into account the size of the Library and Temp folders Unity generates whilst importing and building (i.e. those that are not in version control)?
     
    Twistplay, Sep 25, 2015
    #15

  16. patrickc

    patrickc

    Unity Technologies

    Joined:
    Jan 23, 2015
    Posts:
    67
    Twistplay,

    The size is calculated from the root of the source control, so if you're using Git or Mercurial, we're downloading (and calculating) the file size of everything downloaded, ignoring history files. If you have source assets in the same repo but in a parallel directory, we're still downloading them, so it still fits into the size calculation.

    If you have the Library or Temp folders in source control (which you shouldn't), they will be included in the size calculation.

    If you're using SVN or Perforce, you can configure your project such that Cloud Build only downloads the specific path that contains your Unity project. Note that the "subdirectory" field in Cloud Build is only used to mean "the subdirectory beyond the root that we download from".

    Let us know if you have questions or if the problem persists.

    Cheers,
    Patrick
     
    patrickc, Sep 30, 2015
    #16

  17. Twistplay

    Twistplay

    Joined:
    Dec 6, 2012
    Posts:
    36
    Thanks Patrick - I already accounted for every single folder in SVN, all the way to the root when I calculated the 550MB figure. Definitely no Library or Temp folders there.

    I left it until all builds had finished, built again and the problem disappeared by itself (SVN had definitely not shrunk in the intervening time!), so I suspect it may happen again at some point even though we are below the repo size limit.
     
    Twistplay, Sep 30, 2015
    #17

  18. redthrawn

    redthrawn

    Joined:
    May 8, 2013
    Posts:
    27
    @patrickc I'm having the same problem. My folder is set right, and the entire size of my SVN repo is well below the max for my plan, but it still stops the build saying that it is too large. The weird part is that it only does this for iOS, not Android, Mac, PC, or Linux.
     
    redthrawn, Oct 16, 2015
    #18

  19. Deozaan

    Deozaan

    Joined:
    Oct 27, 2010
    Posts:
    707
    This is still pretty confusing.

    BitBucket tells me that my repo is only 300MB, but our latest commit has Cloud Build saying my repo is 1.15GB which exceeds the limit by 148MB.
     
    Deozaan, Oct 26, 2015
    #19
    RaventurnStefan likes this.

  20. korporeal_games

    korporeal_games

    Joined:
    Oct 16, 2015
    Posts:
    2
    Hi all, I'm posting to this thread with the same problem that just popped up today.

    Our Bitbucket repo is only 283.2 MB.

    Our Droid and iOS builds failed:

    1: ! postcheckoutstatus failed. Project is too large to build for this tier of service. (1.09 GB exceeds limit by 93.72 MB). blah blah blah.

    On my local machine the entire folder (even with Library) is only 546.6MB

    I'll re-read this entire thread but I didn't really see any answers to the question how is (in our case) 1.09GB was calculated.

    I think a lot of us are scratching our heads on the maths.
     
    korporeal_games, Dec 12, 2015
    #20

  21. korporeal_games

    korporeal_games

    Joined:
    Oct 16, 2015
    Posts:
    2
    Also, we're Pro so I'm not sure why our account says "FREE" in the Cloud dashboard
     
    korporeal_games, Dec 12, 2015
    #21