Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Very large project in unity (Slow)

Discussion in 'Editor & General Support' started by kondrup, Feb 7, 2011.

  1. kondrup

    kondrup

    Joined:
    Apr 23, 2009
    Posts:
    205
    Hi,

    We have been working with unity for a long time now and we have found out that unity gets slower for each asset you import into your asset folder.

    At the moment our project is 40GB+ and consists of more than 1.000.000 files and 215.000 folders.

    The reason why we have so many files is because we are using SVN to version-control our files - and it adds an extra folder for each folder in our project, thus making the project very rich on folders :(

    We have had endless crashes and re-imports of the whole projects during the last 6 months (reimport of the entire project takes around 7-8 hours). Luckily we have now switched to 3.1 and that seems to help on the crashes. Unity 2,6 had an error where the memory did not empty properly and that is now fixed in 3.1.

    Unfortunately the project is still very slow. It takes 30min to start up and 15min to shut down. Refresh assets can take up to a minute even though you just changed a text-file.

    - Our plan in the future is to reduce the amount of folders but they will come again eventually when we get more assets.

    So, does anyone have some sort of solution to this slow nightmare?

    Auto-refresh in unity is of course turned Off ;)
     
  2. sebako

    sebako

    Joined:
    Jun 27, 2009
    Posts:
    301
    first I would suggest to go away from svn - it isn't made for binary's
    you should use git or unity asset server for that.

    Also you probably should reorganize your directory structure within the project ... 215.000 folders just seem like way to much.
     
  3. Don-Gray

    Don-Gray

    Joined:
    Mar 18, 2009
    Posts:
    2,276
    My project folder is 18.9 GB and takes 17 seconds to load my scene.
    24,173 files and 1,305 folders.
     
  4. OutCyder

    OutCyder

    Joined:
    Mar 19, 2009
    Posts:
    2,176
    I wonder what kind of project could involve 200000 folders and 1000000 assets... NASA?
     
  5. Don-Gray

    Don-Gray

    Joined:
    Mar 18, 2009
    Posts:
    2,276
    NASA the game...

    Just kidding :)
     
  6. kondrup

    kondrup

    Joined:
    Apr 23, 2009
    Posts:
    205
    zemog86 - Hmm you may be right about SVN, none the less it works pretty well - maybe a bit slow at times :)

    We don't know much about git but we will definitely check it out - thanks.

    Nasa the game :D

    It's our decorator application http://www.lovemyhome.dk/

    unfortunately we have set it up in a way that gives us too many folders.

    For each asset I think we have 10 folders :/ and when a client wants to display ALL 140 fabrics on their sofa model - and they have 300 sofas - then it can easily result in a lot of files - especially when it creates a Meta-file for each file !!
     
  7. OutCyder

    OutCyder

    Joined:
    Mar 19, 2009
    Posts:
    2,176
    Awwwww... nice project man, but maybe you should reorganize the whole thing before it gets to... 10000000 files.
     
  8. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,603
    shouldn't be needed but a 4-8 disk RAID or 1-2 disk SSD setup should be favorable performance wise as the many random accesses beyond any scale of pages at the time might very well be responsible for it.
     
  9. kondrup

    kondrup

    Joined:
    Apr 23, 2009
    Posts:
    205
    SSD is definitely something we have considered but they are pretty expensive and we don't know how much it will actually help :confused:

    We also think that unity should be better at handling a project like this. For example:

    If I create a new folder in unity it can easily take 2-5 minutes before it's there and then renaming it takes another 2-5 minutes. If I on the other hand created the folder in windows and refresh it's finished within 20-60seconds. - Same goes for moving files around in unity.

    - if windows can handle all the folders/files why can't unity? It's not like we are using all assets in a scene or something - it's just the project tab we are using.
     
  10. DjOscar527

    DjOscar527

    Joined:
    Nov 15, 2009
    Posts:
    800
    lol +1
     
unityunity