Fix for AOT Prebuild's insane memory usage

I've been looking into why the AOT Prebuild for visual scripting takes an insane amount of memory (up to 30GB for our project), and it turns out that it loads every asset in the database just to see if the main object in the asset implements IAotStubbable.

There's a trivial fix to check the type of the asset *before* bothering to load it in. Just change the GetAllAssetsOfType function in AssetUtility.cs to this:

Code (CSharp):

1.        
2.         public static IEnumerable<T> GetAllAssetsOfType<T>()
3.         {
4.             if (typeof(UnityObject).IsAssignableFrom(typeof(T)))
5.             {
6.                 return AssetDatabase.FindAssets($"t:{typeof(T).Name}")
7.                     .Select(AssetDatabase.GUIDToAssetPath)
8.                     .Select(AssetDatabase.LoadMainAssetAtPath)
9.                     .OfType<T>();
10.             }
11.             else
12.             {
13.                 // GetAllAssetPaths is undocumented and sometimes returns
14.                 // paths that are outside the assets folder, hence the where filter.
15.                 var result = AssetDatabase.GetAllAssetPaths()
16.                     .Where(p => p.StartsWith("Assets"))
17.                     .Where(p => typeof(T).IsAssignableFrom(AssetDatabase.GetMainAssetTypeAtPath(p)))
18.                     .Select(AssetDatabase.LoadMainAssetAtPath)
19.                     .OfType<T>();
20.  
21.                 EditorUtility.UnloadUnusedAssetsImmediate();
22.                 return result;
23.             }
24.         }

This not only fixes the memory usage, but also speeds the process up massively.

Incidentally, I think there's also a bug in AotPreBuilder.FindAllAssetStubs(). It loads every asset in the project that's of type GameObject (so, all prefabs) and checks if it has any IAotStubbable components, but it only checks the root objects. I'm pretty sure GetComponents<IAotStubbable> should actually be GetComponentsInChildren<IAotStubbable>

Hope that helps someone! If anyone at Unity is still working on Bolt it'd be great if these fixes could be checked over, and integrated into the package.

Thanks!
Rob.

 

OH MY GOD....I just need to bump this OR I have to buy 512GB RAM for build

 

bump, I think unity should take a look...
AOT Prebuild is broken without this mod.

 

Thank you! Sometimes heros don´t wear capes!

 

Did you open an issue for that?

 

Yep. https://issuetracker.unity3d.com/issues/large-amount-of-memory-used-in-aot-prebuild
(Looks like it's going to be in 1.81, whenever that's due)

 

This really saved us, so thanks for reporting and sharing a fix Rob.

 

You're welcome! Getting close to a year since I reported this, and the fix still isn't released. I'm starting to worry that Visual Scripting has been abandoned...

 

See the stickied November 2022 update. They're working on a new major version, I'm fairly sure it will eliminate the need for AOT prebuild entirely since it'll run using codegen instead of reflection like it does now. Unfortunately, the current version is left as is in the meantime more or less. They'll fix critical bugs but I guess they don't consider this critical.

 

Version 1.9.0 was silently released a week and a half ago: https://docs.unity3d.com/Packages/com.unity.visualscripting@1.9/changelog/CHANGELOG.html

This should be finally fixed.

 

I'm terrified to update ngl. Going to bite the bullet and try though.

 

We’ve updated to 1.9, but still seeing 40GB+ being used, with builds that often then fail partway through. Are there additional settings or strategies for managing this issue?