Search Unity

Fix for AOT Prebuild's insane memory usage

Discussion in 'Visual Scripting' started by Rob-Fireproof, Aug 30, 2022.

  1. Rob-Fireproof

    Rob-Fireproof

    Joined:
    Dec 18, 2012
    Posts:
    56
    I've been looking into why the AOT Prebuild for visual scripting takes an insane amount of memory (up to 30GB for our project), and it turns out that it loads every asset in the database just to see if the main object in the asset implements IAotStubbable.

    There's a trivial fix to check the type of the asset *before* bothering to load it in. Just change the GetAllAssetsOfType function in AssetUtility.cs to this:

    Code (CSharp):
    1.        
    2.         public static IEnumerable<T> GetAllAssetsOfType<T>()
    3.         {
    4.             if (typeof(UnityObject).IsAssignableFrom(typeof(T)))
    5.             {
    6.                 return AssetDatabase.FindAssets($"t:{typeof(T).Name}")
    7.                     .Select(AssetDatabase.GUIDToAssetPath)
    8.                     .Select(AssetDatabase.LoadMainAssetAtPath)
    9.                     .OfType<T>();
    10.             }
    11.             else
    12.             {
    13.                 // GetAllAssetPaths is undocumented and sometimes returns
    14.                 // paths that are outside the assets folder, hence the where filter.
    15.                 var result = AssetDatabase.GetAllAssetPaths()
    16.                     .Where(p => p.StartsWith("Assets"))
    17.                     .Where(p => typeof(T).IsAssignableFrom(AssetDatabase.GetMainAssetTypeAtPath(p)))
    18.                     .Select(AssetDatabase.LoadMainAssetAtPath)
    19.                     .OfType<T>();
    20.  
    21.                 EditorUtility.UnloadUnusedAssetsImmediate();
    22.                 return result;
    23.             }
    24.         }
    This not only fixes the memory usage, but also speeds the process up massively.

    Incidentally, I think there's also a bug in AotPreBuilder.FindAllAssetStubs(). It loads every asset in the project that's of type GameObject (so, all prefabs) and checks if it has any IAotStubbable components, but it only checks the root objects. I'm pretty sure GetComponents<IAotStubbable> should actually be GetComponentsInChildren<IAotStubbable>

    Hope that helps someone! If anyone at Unity is still working on Bolt it'd be great if these fixes could be checked over, and integrated into the package.

    Thanks!
    Rob.
     
    torsknod, jendrix, PanthenEye and 3 others like this.
  2. eggsamurai

    eggsamurai

    Joined:
    Oct 10, 2015
    Posts:
    114
    OH MY GOD....I just need to bump this OR I have to buy 512GB RAM for build
     
    Last edited: Dec 28, 2022
  3. eggsamurai

    eggsamurai

    Joined:
    Oct 10, 2015
    Posts:
    114
    bump, I think unity should take a look...
    AOT Prebuild is broken without this mod.
     
  4. jendrix

    jendrix

    Joined:
    Aug 25, 2014
    Posts:
    5
    Thank you! Sometimes heros don´t wear capes!
     
    eggsamurai likes this.
  5. torsknod

    torsknod

    Joined:
    Mar 7, 2023
    Posts:
    3
    Did you open an issue for that?
     
  6. Rob-Fireproof

    Rob-Fireproof

    Joined:
    Dec 18, 2012
    Posts:
    56
  7. onimitch

    onimitch

    Joined:
    Mar 26, 2014
    Posts:
    7
    This really saved us, so thanks for reporting and sharing a fix Rob.
     
  8. Rob-Fireproof

    Rob-Fireproof

    Joined:
    Dec 18, 2012
    Posts:
    56
    You're welcome! Getting close to a year since I reported this, and the fix still isn't released. I'm starting to worry that Visual Scripting has been abandoned...
     
  9. PanthenEye

    PanthenEye

    Joined:
    Oct 14, 2013
    Posts:
    2,071
    See the stickied November 2022 update. They're working on a new major version, I'm fairly sure it will eliminate the need for AOT prebuild entirely since it'll run using codegen instead of reflection like it does now. Unfortunately, the current version is left as is in the meantime more or less. They'll fix critical bugs but I guess they don't consider this critical.
     
  10. PanthenEye

    PanthenEye

    Joined:
    Oct 14, 2013
    Posts:
    2,071
  11. iScriptz

    iScriptz

    Joined:
    Jul 4, 2018
    Posts:
    17
  12. timmehhhhhhh

    timmehhhhhhh

    Joined:
    Sep 10, 2013
    Posts:
    157
    We’ve updated to 1.9, but still seeing 40GB+ being used, with builds that often then fail partway through. Are there additional settings or strategies for managing this issue?