Feedback [IN-36148] Huge performance drop in BuildLayoutGenerationTask (1.20.3 -> 1.21.9)

After I upgraded our project to use Addressables 1.21.9 from 1.20.3, BuildLayoutGenerationTask became very slow. Before, it took only less than 6 seconds to finish, but after upgrading, the Editor stuck in BuildLayoutGenerationTask for over 10 minutes and I had to kill the Editor process.

After some investigation, I found it's due to the following code:

Code (CSharp):

1. public class BuildLayoutGenerationTask : IBuildTask
2. {
3.     // ...
4.  
5.     private LayoutLookupTables GenerateLookupTables(AddressableAssetsBuildContext aaContext)
6.     {
7.         // ...
8.  
9.         foreach (BuildLayout.File file in lookup.Files.Values)
10.         {
11.             // ...
12.  
13.             // Cache all object types from results to find type for when implicit asset
14.             Dictionary<ObjectIdentifier, Type[]> objectTypes = new Dictionary<ObjectIdentifier, Type[]>(1024);
15.             foreach (KeyValuePair<GUID, AssetResultData> assetResult in m_Results.AssetResults)
16.             {
17.                 foreach (var resultEntry in assetResult.Value.ObjectTypes)
18.                 {
19.                     if(!objectTypes.ContainsKey(resultEntry.Key))
20.                         objectTypes.Add(resultEntry.Key, resultEntry.Value);
21.                 }
22.             }
23.    
24.             // ...
25.         }
26.  
27.         // ...
28.     }
29. }

In our project, lookup.Files.Values has 26336 items, m_Results.AssetResults has 36242 items and objectTypes has 695450 items.

It seems the code of constructing objectTypes can be moved outside the loop:

Code (CSharp):

1. public class BuildLayoutGenerationTask : IBuildTask
2. {
3.     // ...
4.  
5.     private LayoutLookupTables GenerateLookupTables(AddressableAssetsBuildContext aaContext)
6.     {
7.         // ...
8.  
9.         // Cache all object types from results to find type for when implicit asset
10.         Dictionary<ObjectIdentifier, Type[]> objectTypes = new Dictionary<ObjectIdentifier, Type[]>(1024);
11.         foreach (KeyValuePair<GUID, AssetResultData> assetResult in m_Results.AssetResults)
12.         {
13.             foreach (var resultEntry in assetResult.Value.ObjectTypes)
14.             {
15.                 if(!objectTypes.ContainsKey(resultEntry.Key))
16.                     objectTypes.Add(resultEntry.Key, resultEntry.Value);
17.             }
18.         }
19.  
20.         foreach (BuildLayout.File file in lookup.Files.Values)
21.         {
22.             // ...
23.         }
24.  
25.         // ...
26.     }
27. }

With this modification, BuildLayoutGenerationTask took about 80 seconds to finish, which is still worse than the old version, but is usable in our project.

Hope Unity developers can take a look at this issue.

 

Here is the comparision between 1.20.3 and 1.21.9 (with my modification above):
1.20.3:

1.21.9:

 

@Alan-Liu, did you also notice that the BuildLayoutGenerationTask is requiring more RAM than before? In addition to it becoming slow, it also started to require RAM in the realm of nearly a hundred of gigabytes (for our project).

 

I didn't notice that before, but I did some tests today, and confirmed there is an unexpected large memory usage in BuildLayoutGenerationTask of Addressables 1.21.9.

Here is the data from my test in Windows Editor (Unity 2020.3.30 + Addressables 1.21.9):


1,2,3 refer to the locations in the code:

Code (CSharp):

1.  
2. public class BuildLayoutGenerationTask : IBuildTask
3. {
4.     // ...
5.  
6.     private BuildLayout CreateBuildLayout()
7.     {
8.         AddressableAssetsBuildContext aaContext = (AddressableAssetsBuildContext)m_AaBuildContext;
9.         LayoutLookupTables lookup = null;
10.         BuildLayout result = null;
11.         using (m_Log.ScopedStep(LogLevel.Info, "Generate Lookup tables"))
12.         {
13.             // ----------- 1 -----------
14.  
15.             lookup = GenerateLookupTables(aaContext);
16.  
17.             // ----------- 2 -----------
18.         }
19.         using (m_Log.ScopedStep(LogLevel.Info, "Generate Build Layout"))
20.             result = GenerateBuildLayout(aaContext, lookup);
21.  
22.         // ----------- 3 -----------
23.         return result;
24.     }
25.  
26.     // ...
27. }
28.  

Apparently, something in GenerateLookupTables caused the memory increased about 8GB.

Looking through the code, I think it's due to the following code:

Code (CSharp):

1. private static Dictionary<long, string> GetObjectsIdForAsset(string assetPath)
2. {
3.     UnityEngine.Object[] assetSubObjects = AssetDatabase.LoadAllAssetsAtPath(assetPath);
4.     Dictionary<long, string> localIdentifierToObjectName = new Dictionary<long, string>(assetSubObjects.Length);
5.     if (assetSubObjects != null)
6.     {
7.         foreach (var o in assetSubObjects)
8.         {
9.             if (o != null && AssetDatabase.TryGetGUIDAndLocalFileIdentifier(o, out _, out long localId))
10.                 localIdentifierToObjectName[localId] = o.name;
11.         }
12.     }
13.  
14.     return localIdentifierToObjectName;
15. }

As you can see, it calls AssetDatabase.LoadAllAssetsAtPath to load assets and during the execution of BuildLayoutGenerationTask, Unity dosen't have any chance to unload them, so the memory keeps increasing until all assets that should be packed in asset bundles are loaded.

A workaround is that unloading unused assets after BuildLayoutGenerationTask.GenerateLookupTables processes some asset bundles. Here is an example (Diff for BuildLayoutGenerationTask.cs):

Code (CSharp):

1. ===================================================================
2. @@ -241,6 +241,8 @@
3.              Dictionary<string, List<BuildLayout.DataFromOtherAsset>> guidToPulledInBuckets =
4.                  new Dictionary<string, List<BuildLayout.DataFromOtherAsset>>();
5.  
6. +           long memory = UnityEngine.Profiling.Profiler.usedHeapSizeLong;
7. +
8.              foreach (BuildLayout.File file in lookup.Files.Values)
9.              {
10.                  Dictionary<string, AssetBucket> buckets = new Dictionary<string, AssetBucket>();
11. @@ -464,6 +466,9 @@
12.                          }
13.                      }
14.                  }
15. +
16. +                if (UnityEngine.Profiling.Profiler.usedHeapSizeLong - memory >= 1024 * 1024 * 1024)
17. +                    EditorUtility.UnloadUnusedAssetsImmediate();
18.              }
19.  
20.              foreach (BuildLayout.File file in lookup.Files.Values)
21.  

The following data is from the same test using the workaround above:

 

Hi, @unity_shane, can you take a look at the memory usage issue above?

Update:
I submitted a new bug report for this issue: IN-37708.

 

@Alan-Liu thanks for looking into it that deeply!
@unity_shane We've hotfixed it on our branch now as well, but waiting for the official 1.21.10 release now

 

Here is the public link for the memory usage issue: https://issuetracker.unity3d.com/issues/large-memory-usage-when-using-buildlayoutgenerationtask