Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

[Suggestion] Optimize system resource usage

Discussion in 'Unity Collaborate' started by nrodemund, Jan 27, 2019.

  1. nrodemund

    nrodemund

    Joined:
    Sep 15, 2018
    Posts:
    16
    Unity Collab "checking for changes" needs an awful amount of time. On my Laptop (i7,24gb RAM, default SSD) it needs about 5-6 minutes, on my high-end pc (Threadripper 2950, 32gb RAM, Raid0 NVM) it still needs a minute. Thats not too bad, but still, my project isn't that big (20k files, 11gb).

    When monitoring system resources i noticed that Unity Collab only uses less than 1,5% of disk usage and only 4,5% of CPU.

    I assume that the long "checking for changes" occurs from hashing the assets folder and some other files.

    For benchmarking purposes I tried if it is possible to use much more system resources. Using multithreaded, queued and buffered FileStream and a faster hashing function (i.e. xxHash) I managed to incrase disk usage to over 53% (which may be max due to IOPS) and CPU usage to 63%. CPU usage increases to 96,5% when hashing a larger folder)

    The times are not directly compareable as the benchmarking tool just hashes each asset folder file, while UnityCollab seems to read less than the whole folder (time*read is lower than asset folder size), there may be some optimizations.

    upload_2019-1-27_12-0-38.png

    On my lower resources laptop this needs 10 seconds while unity collab needs up to 6 minutes.

    Whouldn't it be possible to optimize hashing / whatever unity collab is doing using full system resources?
     
    Last edited: Jan 27, 2019
  2. ryanc-unity

    ryanc-unity

    Unity Technologies

    Joined:
    Mar 23, 2016
    Posts:
    1,739
    Thank you for the awesome technical breakdown, @nrodemund! You're right that the hashing used for "checking for changes" is not as optimized as it should be. But since we removed the old snapshot system that depended on that check we'll soon be cutting out that check altogether in an upcoming version of 2018.3. The fix has already made it's way into trunk and is in the process of getting backported to 2018.3. We'll update everyone on the forum when this change makes it in.
     
  3. nrodemund

    nrodemund

    Joined:
    Sep 15, 2018
    Posts:
    16
    Thanks for your help! Looking forward for that!
     
  4. ryanc-unity

    ryanc-unity

    Unity Technologies

    Joined:
    Mar 23, 2016
    Posts:
    1,739
    The 2.0 package is now out, which will directly addres the "Checking for changes..." message! Check out the release notes / known issues post for more information. Thank you guys for your patience while we worked on this fix. :)
     
unityunity