BuildPlayerContent - Calculate Asset Dependency Data takes forever

Since we upgraded our project from Unity 2018.4.25f1 to Unity 2019.4.14f1 we are experiencing very long addressables build time caused by Calculate Asset Dependency Data.

In 2018.4 for all platforms addressables build in around 2 hours with ~20k assets and scenes.

Now it takes 10-16 hours to complete because it spend 1-6s on each asset on calculating asset dependency even if only one prefab changes that is not related anywhere. After other changes it builds faster but still is calculating files that are not even close to anything that has changed in the project.

From description in docs Unity should cache that dependency data, but looks like its not working correctly in our case.

What can cause it? In some places we are using OnValidate in some scripts but still it shouldn't cause rebuild of sound files or UI sprite and atlases that are not related to them.

Is there some way to know why assets dependency need to be recalculated?

 

We have tested many versions:
1.12.0
1.16.1
1.16.6
1.16.10
1.16.15

On all of them build looks exactly the same, even if we switch to AssetDatabase v1. Seems that the only difference is between Unity 2018 and 2019

Also something that I noticed is that when I remove project Library and reimport all, first addressable build is long (~15h) but next one is fast (~20min) and with small changes in prefabs we have quite good build times. But then when we change something in one scene build time jumps to 7 hours and stays like that, even if we don't change anything. I have seen that behavior 2 times on 2 different devices.

#edit
Build with same content in Unity 2019.4.16 but without addressables takes 1 hour 5 min total, so 2 hours with them is reasonable. The only thing that looks incorrect is that dependency cache or something that should not be calculated so long and every time.

 

Can we count on any help? Our game is already released and we wanted to release patch before Christmas but preparation of builds for each platform will take us at least a week. On 2019 everything works better than on 2018, but if we didn't find a solution to addressables we will need to roll back to 2018.


Tested AddressableAssetSettings.BuildPlayerContent(); and manual build in Groups window but behavior is the same.

During that dependency calculations build log is spammed with this message. There is so many of them that log file is 30mb after one addressables build.

Unloading 1 unused Assets to reduce memory usage. Loaded Objects now: 209424.

Total: 948.645700 ms (FindLiveObjects: 22.775800 ms CreateObjectMapping: 13.633900 ms MarkObjects: 911.708100 ms  DeleteObjects: 0.527300 ms)



Unloading 0 Unused Serialized files (Serialized files now loaded: 0)

System memory in use before: 4.30 GB.

System memory in use after: 4.32 GB.

We have 378 addressable groups. Game has 7 levels in multi-scene setup, 4 hero (6 skins each) and 16 enemies.

• each level has group for each scene and its assets like lighting data, reflection cube
• another set of groups is for models and textures that are used in each level.
• every heros skin has separate group. For enemies there are 16 groups.
• next are UI atlases and separate group for each one (not everyone is used on each platform) and 2 more for unpacked sprites and for fonts = 16 groups.
• last are groups containing music (17 groups), SFX & VO (58 groups) and dialogs (21 groups)

Thanks to that setup we have few big groups 100-30mb and most of them are 10-2mb so its easier to reduce update size. Total build size is 4gb and 3gb on switch with ~60k assets.

In Addressables Analyze we don't have any issues, no duplicates and we also made our own tools to better track duplicates, missing assets and to test groups config.

 

Hi! We have the same issue. We works on Unity 2018.4 and use Addressables 1.16.15. But after switching to Unity 2019 build time of Addressables became infinity. We also on final stage of our development, so it became a huge problem to switching our game to unity 2019. Any changes?

 

Is there an update? I also came across this problem. We wanted to upgrade from 2018.4 to 2019.4 but with this huge build time it's very unlikely

 

@DavidUnity3d is there any update?

 

@TreyK-47 thanks for reply. We are currently on Unity 2019.4.16 version and we still have this issue sometimes.

But I found workaround in our case:
- if collect dependencies became "infinity"
- then stop process (cancel button)
- close Unity, open Unity
- build Addressables again

I will try newest version of Unity and let you know about result

 

I encountered this issue, seems like an engine side memory leak. In my project, if I build->upgrade, collect dependencies takes forever. If I build->restart unity->upgrade, speed seems reasonable.

Happens on Unity 2020.1.16, Addressables 1.16.15

 

Nothing changed for us after upgrading from 2019.4.16 to 2019.4.18. Addressables 1.16.13. First build took 15 hours, just started another one without any changes in the project and it also looks like it will take another 15h.
Workaround with stopping build and restarting also don't help for us. I remember seeing it work several times but on first builds after removing Library.

 

Made few more tests today. We are using own build pipeline to support many platforms and regions and before every build our code make sure that all text files, scripting defines and projects settings are up do date. When i'm building using it then no mater what it always build long time, editor restart is not helping.

When I tested it only using AddressableAssetSettings.BuildPlayerContent() or Build from Addressable Groups window trick with restarting unity works every time!

We will search what our tool can be doing that is changing it. Nothing should be changing after doing same action again, but maybe its marking some assets as dirty. Seems like modifying any asset and saving project is somehow making everything dirty or clearing build cache?

 

We have some editor scripts, but only for auto grouping assets to Addressables. In our case I don't think that this is cause of problem. But it interesting point.

 

@ozonex I am having similar issues, but for us it is only the Android platform that is having the crazy long build times. Windows is ~30 minutes consistently but Android is 6+hrs.

Do you have similar platform differences? Or is yours long no matter the platform? If you just have one platform, which are you using?

 

We have same issue for all platforms that use addressables: PS4, Xbox, Switch. For windows we don't use them because steam is really good at making patches.

 

We have this issue on both iOS (building on a Mac) and Android (building on a Windows PC).

Using Unity 2019.4.15f1 and Addressables 1.16.15.

Usually, the Calculate Asset Dependency Data completes in a few seconds but sometimes it ends up taking the better part of an hour. Normally happens if Unity has been open for a long time (hasn't been restarted) and after getting latest updated assets from version control.

Restarting Unity doesn't necessarily fix it either. And while it's doing it, I'd rather not force-quit lest I end up with a corrupted asset somewhere.

 

Hi, just send bug report with log & AddressablesBuildTEP.json (case 1311393)

We are unable to share our project and we also cant reproduce it on fresh projects :/

 

Another thing that also might be related to this bug is that we have big issues with making patch of our game for Switch. Every time we try to make a patch build, even when nothing has changed or with some prefab changed patch size is ~2,5gb when whole game size is 3gb. All addressables change for some reason.

Build was made using same project instance and same PC. When we check what is the difference between files it looks like file serialization is different somehow.

Because patch size is limited to 500mb we are actually unable to make any proper patch of our game. Base game was made on 2019.4.14 and when we checked before publishing if patching works we got small 100mb patch.
After upgrading to 2019.4.16 we can't reproduce that, even if we return to 2019.4.14. Even with no changes in files almost all addressables changed

This behavior looks like editor is not updating files every time, like after update between editor versions it leave old files and then sometimes it decide to do it, and there is no way back. It makes some sense, to not update whole game content after engine upgrade, but I'm not sure if this is how it actually works.

 

@Ryanc_unity We finally found a way to reproduce that bug on example project and reported bug with it (case 1313722). It seems its memory management issue.

 

Here is this bug on issuetracker:
https://issuetracker.unity3d.com/issues/addressables-very-slow-build-when-editor-heap-memory-is-full

 

same issue with me but only on Android , in iOS it build normally
using 2020.2.7f1
any workaround ?

 

Any news on that bug report?

 

Getting this issue on both 2019.4.14 and 2019.4.17, are there any updates on a potential cause/solution for this issue?

 

It has already been filed and open for a month now: https://issuetracker.unity3d.com/issues/addressables-very-slow-build-when-editor-heap-memory-is-full

 

In corporate Environments there can be some Antivirus causing I/O locks on Assets. I remember I requested to temporarily disable the Antivirus (for my computer) in my company once and saw a significant difference in the handling of each Asset. Maybe this could hint to some optimizations

 

This bug is from full memory heap in Unity Editor. So one of solution is to not have it full: restart editor before addressable build. It should help but sometimes when you have many assets even building addressables can fill memory heap.

Another solution would be to increase editor memory heap, but there is no setting in editor for it. So even if you have PC with 64GB RAM Unity editor is using it's hardcoded value (I think maximum 1-2GB?).

 

I have received message for this issue:

 

We have met the same problem on 2020.3.5f1(Windows) when build Android package. And we cannot resolve it by restarting Unity or regenerate Addressables Group info.The Unity.log file‘s last content of Unity.log is repeated as follows:

Unloading 1 unused Assets to reduce memory usage. Loaded Objects now: 132188.
Total: 227.944800 ms (FindLiveObjects: 12.789400 ms CreateObjectMapping: 15.368200 ms MarkObjects: 199.589700 ms DeleteObjects: 0.195800 ms)

Unloading 0 Unused Serialized files (Serialized files now loaded: 0)
System memory in use before: 2.55 GB.
System memory in use after: 2.56 GB.

Unloading 0 unused Assets to reduce memory usage. Loaded Objects now: 132188.
Total: 240.341100 ms (FindLiveObjects: 13.661800 ms CreateObjectMapping: 14.406700 ms MarkObjects: 212.108700 ms DeleteObjects: 0.160500 ms)

Unloading 39 Unused Serialized files (Serialized files now loaded: 0)
System memory in use before: 2.55 GB.
System memory in use after: 2.56 GB.

Unloading 1 unused Assets to reduce memory usage. Loaded Objects now: 132222.
Total: 252.804500 ms (FindLiveObjects: 15.786000 ms CreateObjectMapping: 18.568500 ms MarkObjects: 218.276400 ms DeleteObjects: 0.170100 ms)

Unloading 3 Unused Serialized files (Serialized files now loaded: 0)
System memory in use before: 2.55 GB.

 

I faced such issue with the Android Build Target. Fixed this by resolving Android dependencies via ExternalDependencyManager -> Android Resolver.

 

@ryanc-unity can you tell us please are there any news on https://issuetracker.unity3d.com/issues/addressables-very-slow-build-when-editor-heap-memory-is-full ?

 

This sounds very odd, but resetting my window layout got rid of this problem for me. I did this by going to Window => Layouts => Default then restarting the build as usual

 

Might be helpful to some of those affected by this but if the addressables groups window is open when you are doing any group manipulation it rebuilds the treeviews after every modification. So when you are running the deduplication process and its writing new entries it could be slowing things down. In my case I am adding things automatically to addressables groups based on folder so it would slow the import process down by having the window open. IIRC in my case it went from 20 minutes to 2hours+ just by having the window open for reimporting certain folders. I ended up hacking something in that checks for the existence of the window and closing if its open before the build starts. It might be helpful here too.

Code (CSharp):

1.         var window = EditorWindow.GetWindow(typeof(EditorWindow), false, "Addressables Groups");
2.         if (window.titleContent.text == "Addressables Groups")
3.         {
4.             window.Close();
5.             return true;
6.         }

 

It's been a year and a half. Any progress on this @TreyK-47 ?

 

Same issue here, 2021.3.16f1.

 

For those still searching, the problem in my case was that Lightmapping "Auto Generate" option was turned on.

So I think the bundle build process was probbably waiting for light baking to complete.

 

Update !

I faced the same issue. Unity 2021.3.f1, platform = Android. When I try to build bundles via Addresable Groups->Build->New Build. After several steps, it hangs. After several hours of debugging and experimenting, I found out that I was hanging at the stage of collecting dependencies for the scene, and a deeper debugging brought me to the compilation project, deep in the engine, I didn’t dig further, but noticed that my dependencies on Android libraries was outdated. The External Dependency Manager at Unity startup couldn't update them automatically because I didn't have the correct %JAVA_HOME% path specified. I configured this path and reload my PC. The next time Unity was started, the External Dependency Manager updated everything correctly and the bundles were built without problems.



@TreyK-47 - pay attention to this case, probably the dependency update error (falls into the log that Gradle cannot update) is not caught correctly and the process freezes on the side.

 

still happening on 2022 LTS. the cancel button doesnt work either...

 

I'm seeing this issue when building the player for a project. It eventually builds after 8 minutes, but the majority of the build is just "Unloading Unused Serialized files" being spammed in the Editor log. I'm just building for Windows - are there any workarounds here? Anyway to speed this up on successive builds?

 

We have

We're experiencing the same issue after updating to unity lts 2022.3.12 from 2021.
Happens both on the Unity Cloud Build (10h of building Android project!) and local machine after second addressables build.

The workaround we use:
- In unity editor: Clear addressable build cache before building addressables
- Unity Cloud Build: clean builds, slower than expected but at least does not take 10h to build.

I hope this issue will be resolved soon because it really slows the development

 

+1 this

In my project it seems like every time a build fails, it does not properly clear the cache, so next time I try to build it gets stuck forever on calculating one scene's dependencies. Again the cancel button doesn't work either. Clearing the build pipeline cache does the trick, its just annoying to do it every time lol, or forget to do it and need to force close unity

Edit: actually i have to clear All to get anything to work. its very annoying to work with and makes me wonder why its not fixed yet

 

I can confirm this step takes way longer than just performing a clean build and often gets stuck completely. A unity restart helps with the situation so I assume some static state is broken

 

Confirming this issue. Happened suddenly. I thought it was due to our project size but no. 2022.3.13f1, addressables 1.21.19

 

Also seeing this as of 2022.3.13f1 upgrade (we were previously on .11f1 so may just revert back if it fixes it)

 

We have this problem while building addressables (1.21.19) using Unity 2022.3.12f1 on Switch. Our workaround is to clean addressables before relaunching a build which consume a lot of time.

 

To all experiencing this, it's a regression of ScriptableBuildPipeline 1.21.21 (which is mandatory from Unity 2022.3.13 onwards)

You can find a fix there until Unity solves the problem

https://forum.unity.com/threads/bui...buildpipeline-1-21-21-extremely-slow.1512488/

 

Also having this issue. Have to usually clear the build cache or else it hangs forever or takes extremely long.

 

Since building addressables isn't deterministic AFAIK that isn't a viable option when creating patches for certain platforms.

IE: some bundles will be included in the patch because they will be seen as "new data" while they are in fact the same bundle with the same content, just re-created. This will make the patch unnecesarily bigger.

This may not be an issue for all platforms.