Feedback TextMeshPro & Addressables (Asset Bundles)

There was a discussion about this on Addressables forum, but since it's been 2 years since last official response, I'd like to get some recent insights into the situation.
https://forum.unity.com/threads/bug-1-2-4-duplicate-textmeshpro-assets-in-bundles.754211/

Our project stumbled upon this issue because we made the decision to put almost all of the game assets into Addressables (and away from Resources/ BuildSettings) except for a small Boot scene. We have a lot of UI prefabs/scenes with TMP elements that reference materials and font assets. These same assets are meant to be in Resources folder by the way TextMeshPro is currently implemented (as of 3.0.6).

Because there is no way to have Unity reference assets across Resources and AssetBundles, there is inevitable asset duplication. Because we target mobile platforms and some font assets were pretty big, this became an issue. To overcome, I had to resort to some ugly hacking (see thread above). I had to put everything (TMP Settings, materials, Font Assets, Sprites) outside of Resources - it's the only way to get rid of duplication.

But then I realized just how deeply TextMeshPro is tied to Resources folder both at Editor and Runtime:

• TMP Settings must be in a strict hard-coded path in Assets folder (not ideal because we try to put all third party stuff into a dedicated folder..) and must be in Resources. You can ignore it and just move this asset out and load via Addressables, but TMP will complain that "essential assets" needs to be imported and some editor functions won't work (like font asset creation).
  
• You can't dynamically load fallback fonts. This is a big problem if you want to support various languages but don't want to keep ALL fonts in memory (some fonts like Japanese or Chinese have thousands of symbols and better not be loaded until they are needed). To work around this I had to:
  • Create a "template" TMP Settings asset with empty Fallback fonts array;
  • Load this asset via Addressables and then manually inject (by reflection hacking) only the required Font Assets based on current user language into the instantiated clone of the asset;
  • Replace the static instance of TMP Settings with this clone (again, using reflection).
• If you reference materials by name using rich text tags and your materials are not in Resources folder - it won't work. Some more hacking is required.
• Doing a search in code for Resources.Load you can see some even less obvious uses. Each one is a potential issue..

So in the end, current situation with TextMeshPro assets is rather grim:

• TextMeshPro is a must-have package promoted by Unity;
• Addressables is a must-have package promoted by Unity;
• They do "kinda" work together.. except not really, you get a duplicate asset issues which can be (and actually is) critical for "real" production projects on mobile platform;
• TMP is hard-wired into Resources (which Unity manual advises against using since forever...) which does not work at all with Asset Bundles;
• If you need to optimize text asset memory usage / build size - you need to resort to hacking or creating and maintaining local fork of TextMeshPro (both are a lot of work and have pros and cons).

Please consider addressing this.

 

I stumble on the same issue in our project. Due to the usage of adressables, we have multiple font texture/material duplicated in memory.
Since we are running short in term of memory, it would have been great to have some news from Unity about this subject.

 

Same.
Even if they can't fix this, a guide or suggested how-to do it ourselves will be a tremendous helpful

 

We are also waiting for solutions for this.

 

Hope this gets addressed soon. This is a huge deal for multi language support on a handheld, wasting 120+ MB of RAM.

 

Please do something about this. The silence is deafening

 

It says a lot that a detailed post about a critical issue like this hasn't received a single response from Unity in over an year.

Memory is a scarce resource on some platforms and this is wasting 100MB+ for unused fonts just in the rare occasion a fallback is necessary.

As @phobos2077 mentioned, both packages are pretty much a must-have on any professional project, and getting them to properly work together involves a lot of work.

 

waiting for so long

 

With the advent of WaitForCompletion from quite a while ago, replacing all the Resources.Load calls in TMP with Addressables isn't too bad.

Obviously this requires keeping a local copy of the TMP package but it's not a terribly involved process. I came across this post so I spent a little bit of time figuring out a nice way to do it. Basically, I have it set up a little bit like the way the localization package works.


With this setup, I replaced most (there are a few stragglers in my test project) of the Resources.LoadAsset<T> with calls to this class:

Code (CSharp):

1. using UnityEngine;
2.  
3. #if TMP_ADDRESSABLES
4. using UnityEngine.AddressableAssets;
5. #endif
6.  
7. namespace TMPro
8. {
9.     public static class TMP_ResourceLoader
10.     {
11.         public static TMP_Settings LoadSettings()
12.         {
13.             TMP_Settings result = null;
14. #if TMP_ADDRESSABLES
15.             result = Addressables.LoadAssetAsync<TMP_Settings>( $"TMP-Settings" ).WaitForCompletion();
16. #else
17.             result = Resources.Load<TMP_Settings>("TMP Settings");
18. #endif
19.             return result;
20.         }
21.  
22.         public static TMP_FontAsset LoadFontAsset( string name )
23.         {
24.             TMP_FontAsset result = null;
25. #if TMP_ADDRESSABLES
26.             result = Addressables.LoadAssetAsync<TMP_FontAsset>( $"TMP-Font-{name}" ).WaitForCompletion();
27. #else
28.             result = Resources.Load<TMP_FontAsset>( TMP_Settings.defaultFontAssetPath + name );
29. #endif
30.             return result;
31.         }
32.  
33.         public static Material LoadMaterial( string name )
34.         {
35.             Material result = null;
36. #if TMP_ADDRESSABLES
37.             result = Addressables.LoadAssetAsync<Material>( $"TMP-Material-{name}" ).WaitForCompletion();
38. #else
39.             result = Resources.Load<Material>( TMP_Settings.defaultFontAssetPath + name );
40. #endif
41.             return result;
42.         }
43.  
44.         public static TMP_ColorGradient LoadColorGradient( string name )
45.         {
46.             TMP_ColorGradient result = null;
47. #if TMP_ADDRESSABLES
48.             result = Addressables.LoadAssetAsync<TMP_ColorGradient>( $"TMP-Gradient-{name}" ).WaitForCompletion();
49. #else
50.             result = Resources.Load<TMP_ColorGradient>( TMP_Settings.defaultColorGradientPresetsPath + name );
51. #endif
52.             return result;
53.         }
54.  
55.         public static TMP_SpriteAsset LoadSpriteAsset( string name )
56.         {
57.             TMP_SpriteAsset result = null;
58. #if TMP_ADDRESSABLES
59.             result = Addressables.LoadAssetAsync<TMP_SpriteAsset>( $"TMP-Sprite-{name}" ).WaitForCompletion();
60. #else
61.             result = Resources.Load<TMP_SpriteAsset>( TMP_Settings.defaultSpriteAssetPath + name );
62. #endif
63.             return result;
64.         }
65.     }
66. }

The results are pretty promising in that I can still have a default font set up and working correctly. You'd still have to be careful that you don't accidentally touch something TMP related until your bundles are up and running, and you could of course run in to hiccups because none of this is done asynchronously.

I would really like to see Unity address this issue as well - but in the meantime this might be a path for others to explore.

 

I am also running into this issue, where all these TMP assets are being duplicated

 

Stumbled upon this problem. Can we have some response from Unity? @Stephan_B ?

 

OMG. This is so annoying!
@Unity If you implement new features and try to make them the new standard, please make sure they work together with other standard systems!

It's always the same problem: Whenever a new feature is added to Unity (like the Addressables system) and promoted as the way to go, it causes a lot of trouble because the other systems are not meant to work the way the new system requires them to do. This makes the feature (and therefore Unity) not suitable for production. Eventually some years in the future the feature will work as intended. But this is the time it will for sure be replaced by another feature facing the same problems.

In the end the problem is that teams at Unity work on different features and it seems that they work somewhat in isolation. Please, Unity: Make sure to only release a feature and system when it is actually ready to be used with all the other features and systems. You cannot simply add something and ignore that all the other systems have to be adjusted to the new feature.

 

argh, so this is the problem with creating TMP Material Presets via scripts and then using AssetDatabase.SaveAssets to Resources folders in later versions of Unity.

Great... fixing when?

 

Also running into TMP and Addressables issues, but looks like TMP development is being frozen whilst its integrated into the UGUI and UITK packages in various ways, so I wouldn't expect any fixes anytime soon.

 

I was fighting with this issue just now. My solution was to nullify m_defaultFontAsset field in the TMP Settings.asset for the duration of the build. This stops the asset, which is in the Resources folder, from duplicating your addressable fonts in the build.

Here's a helper class for this:

Code (CSharp):

1. public class BuildContext : System.IDisposable
2. {
3.     public BuildContext(string buildVersion = null)
4.     {
5.         if (buildVersion != null) {
6.             PlayerSettings.bundleVersion = buildVersion;
7.         }
8.  
9.         Provider.Checkout(Provider.GetAssetByPath("Assets/TextMesh Pro/Resources/TMP Settings.asset"), CheckoutMode.Asset);
10.         SetFieldValue(TMP_Settings.instance, "m_defaultFontAsset", null);
11.     }
12.  
13.     public void Dispose()
14.     {
15.         Provider.Revert(Provider.GetAssetByPath("ProjectSettings/ProjectSettings.asset"), RevertMode.Normal);
16.         Provider.Revert(Provider.GetAssetByPath("Assets/TextMesh Pro/Resources/TMP Settings.asset"), RevertMode.Normal);
17.     }
18.  
19.     private static void SetFieldValue(Object obj, string fieldName, object value)
20.     {
21.         FieldInfo field = (FieldInfo)obj.GetType().FindMembers(MemberTypes.Field, BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic, (member, _) => member.Name == fieldName, null)[0];
22.         field.SetValue(obj, value);
23.         EditorUtility.SetDirty(obj);
24.         AssetDatabase.SaveAssetIfDirty(obj);
25.     }
26. }

As a bonus, it sets and reverts the project build version also.

 

@uDamian Can you weigh in on this?

 

^ this issue hasn't had an official response in 4 years don't know why they'd start now.

 

Omg, also waiting for a solution...

 

Not only some builds takes 50mins to complete with those milions of Shaders Variantes. Every time it fails, is because of an unity related issue and we never have a way out.

It makes me wonder why haven't I switched to UE5 yet.

 

Even just acknowledgement at this point would be nice.

I wouldn't mind giving that IDisposable BuildContext object above a try just to keep things quiet without invalidating packages, but I don't really know where that script would go or how to reference it. I guess I'd probably make child classes of the default build scripts or something? I can't say I've had to do something like this yet.

 

If you're using the default build command, from the 'File' menu, you can add this script to 'Assets/Editor/Scripts' folder:

Code (CSharp):

1.  
2. using UnityEditor;
3.  
4. public static class BuildPlayerProcessr
5. {
6.     [InitializeOnLoadMethod]
7.     private static void Initialize()
8.     {
9.         BuildPlayerWindow.RegisterBuildPlayerHandler(BuildPlayerHandler);
10.     }
11.    
12.     private static void BuildPlayerHandler(BuildPlayerOptions options)
13.     {
14.         using var buildContext = new BuildCommands.BuildContext(options);
15.         BuildPlayerWindow.DefaultBuildMethods.BuildPlayer(options);
16.     }
17. }
18.  

 

Is there no solution yet for this in Text Mesh Pro duplications?

 

Thought I'd add a solution for using Addressables and TMP that doesn't require embedding the package.
In recent versions of Unity you can override the default ResourcesAPI with a custom class.
Using this, we can handle specific cases for TMP assets.

The following code isn't exactly general purpose. It's built with a specific project structure and addressable layout.
That being:

All assets are located at: "Assets/Plugins/TextMesh Pro/Assets"

Fonts: "Assets/Plugins/TextMesh Pro/Assets/Fonts"
Materials: "Assets/Plugins/TextMesh Pro/Assets/Fonts/Materials"
Sprites: "Assets/Plugins/TextMesh Pro/Assets/Sprites"

These folders are added to Addressables and tagged with "tmp_fonts" and "tmp_sprites" respectively.

the "TMP_Settings" asset is also located at: "Assets/Plugins/TextMesh Pro/Assets"

Basically, this "Assets" folder is acting like the "Resources" folder.

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.AddressableAssets;
3. using UnityEngine.ResourceManagement.ResourceLocations;
4. using UnityEngine.ResourceManagement.AsyncOperations;
5. using TMPro;
6. using System;
7. using System.Collections.Generic;
8. using System.IO;
9.  
10. #if UNITY_EDITOR
11. using UnityEditor;
12. #endif
13.  
14. public static class TextMeshProInitializer {
15.  
16.     private static TextMeshProResourcesAPI resourcesAPI;
17.  
18.     [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)]
19.     static void InitializeRuntime() {
20.  
21.         RegisterAPI();
22.     }
23.  
24. #if UNITY_EDITOR
25.     [InitializeOnLoadMethod]
26.     static void InitializeEditor() {
27.  
28.         RegisterAPI();
29.     }
30. #endif
31.  
32.     private static void RegisterAPI() {
33.  
34.         resourcesAPI = new TextMeshProResourcesAPI();
35.  
36.         ResourcesAPI.overrideAPI = resourcesAPI;
37.     }
38. }
39.  
40. public class TextMeshProResourcesAPI : ResourcesAPI {
41.  
42.     private const string AssetsFolder = "Assets/Plugins/TextMesh Pro/Assets";
43.     private const string MaterialsFolder = "Materials";
44.     private const string AssetExtension = ".asset";
45.     private const string MaterialExtension = ".mat";
46.  
47.     private static readonly Type settingsType = typeof(TMP_Settings);
48.     private static readonly Type fontAssetType = typeof(TMP_FontAsset);
49.     private static readonly Type materialType = typeof(Material);
50.     private static readonly Type spriteAssetType = typeof(TMP_SpriteAsset);
51.  
52.     private bool isLoading = false;
53.     private Dictionary<int, IResourceLocation> locations = null;
54.     private List<AsyncOperationHandle> operations = new List<AsyncOperationHandle>(1);
55.  
56.     public TextMeshProResourcesAPI()
57.         : base() {
58.     }
59.  
60.     public void ReleaseOperations() {
61.  
62.         foreach (AsyncOperationHandle op in operations) {
63.  
64.             if (op.IsValid()) {
65.  
66.                 Addressables.Release(op);
67.             }
68.         }
69.  
70.         operations.Clear();
71.     }
72.  
73.     protected override UnityEngine.Object Load(string path, Type systemTypeInstance) {
74.  
75.         //prevent stack overflow exception if we're already loading through Addressables
76.         //since when Addressables are set to FastMode in editor, that method calls into this ResourcesAPI when running Addressables.LoadAsset calls
77.  
78.         if (isLoading) {
79.             return base.Load(path, systemTypeInstance);
80.         }
81.  
82.         //handle specific types
83.  
84.         if (systemTypeInstance == settingsType) {
85.             return LoadSettings(path);
86.         }
87.         else if (systemTypeInstance == fontAssetType) {
88.             return LoadAssetFromLocation<TMP_FontAsset>(path, default, AssetExtension);
89.         }
90.         else if (systemTypeInstance == spriteAssetType) {
91.             return LoadAssetFromLocation<TMP_SpriteAsset>(path, default, AssetExtension);
92.         }
93.         else if (systemTypeInstance == materialType) {
94.  
95.             string fontAssetsPath = TMP_Settings.defaultFontAssetPath;
96.  
97.             //check if the material is located in the TMP folder
98.  
99.             if (path.StartsWith(fontAssetsPath, StringComparison.Ordinal)) {
100.                 return LoadAssetFromLocation<Material>(path, MaterialsFolder, MaterialExtension);
101.             }
102.         }
103.  
104.         //not a TMP asset, load default
105.  
106.         return base.Load(path, systemTypeInstance);
107.     }
108.  
109.     private TMP_Settings LoadSettings(string path) {
110.  
111. #if UNITY_EDITOR
112.         if (!EditorApplication.isPlaying) {
113.             return LoadAssetFromAssetDatabase<TMP_Settings>(path, default, AssetExtension);
114.         }
115. #endif
116.         return LoadAddressableAsset<TMP_Settings>(path);
117.     }
118.  
119.     private T LoadAssetFromLocation<T>(string path, string subFolder, string extension) where T : UnityEngine.Object {
120.  
121. #if UNITY_EDITOR
122.         if (!EditorApplication.isPlaying) {
123.             return LoadAssetFromAssetDatabase<T>(path, subFolder, extension);
124.         }
125. #endif
126.         string name = Path.GetFileNameWithoutExtension(path);
127.         int hash = TMP_TextUtilities.GetSimpleHashCode(name);
128.  
129.         if (locations == null) {
130.  
131.             LoadResourceLocations();
132.         }
133.  
134.         if (locations.TryGetValue(hash, out IResourceLocation location)) {
135.             return LoadAddressableAsset<T>(location);
136.         }
137.  
138.         return default;
139.     }
140.  
141.     private void LoadResourceLocations() {
142.  
143.         var fontsHandle = Addressables.LoadResourceLocationsAsync("tmp_fonts");
144.         var spritesHandle = Addressables.LoadResourceLocationsAsync("tmp_sprites");
145.  
146.         var fontLocations = fontsHandle.WaitForCompletion();
147.         var spriteLocations = spritesHandle.WaitForCompletion();
148.  
149.         locations = new Dictionary<int, IResourceLocation>(fontLocations.Count + spriteLocations.Count);
150.  
151.         AddLocations<TMP_FontAsset>(fontLocations, AssetExtension, locations);
152.         AddLocations<Material>(fontLocations, MaterialExtension, locations);
153.         AddLocations<TMP_SpriteAsset>(spriteLocations, AssetExtension, locations);
154.  
155.         Addressables.Release(fontsHandle);
156.         Addressables.Release(spritesHandle);
157.     }
158.  
159.     private T LoadAddressableAsset<T>(object key) where T : UnityEngine.Object {
160.  
161.         isLoading = true;
162.  
163.         AsyncOperationHandle<T> op;
164.  
165.         if (key is IResourceLocation location) {
166.  
167.             op = Addressables.LoadAssetAsync<T>(location);
168.         }
169.         else {
170.  
171.             op = Addressables.LoadAssetAsync<T>(key);
172.         }
173.  
174.         operations.Add(op);
175.  
176.         T asset = op.WaitForCompletion();
177.  
178.         isLoading = false;
179.  
180.         return asset;
181.     }
182.  
183. #if UNITY_EDITOR
184.     private T LoadAssetFromAssetDatabase<T>(string resourcePath, string subFolder, string extension) where T : UnityEngine.Object {
185.  
186.         string path = Path.Combine(AssetsFolder, $"{resourcePath}{extension}").Replace('\\', '/');
187.  
188.         if (!string.IsNullOrEmpty(subFolder)) {
189.  
190.             string fileName = Path.GetFileName(path);
191.             string parentDir = Path.GetDirectoryName(path);
192.  
193.             path = Path.Combine(parentDir, subFolder, fileName).Replace('\\', '/');
194.         }
195.  
196.         return AssetDatabase.LoadAssetAtPath<T>(path);
197.     }
198. #endif
199.     private void AddLocations<T>(IList<IResourceLocation> locations, string requiredExtension, Dictionary<int, IResourceLocation> lookup) {
200.  
201.         Type type = typeof(T);
202.  
203.         foreach (IResourceLocation location in locations) {
204.  
205.             //because TMP_FontAssets contain sub assets of material type
206.             //we need to check the required type for this location. If it doesn't match don't add the location
207.             //otherwise, we'd be adding 2 locations for the same asset
208.  
209.             if (location.ResourceType != type) {
210.                 continue;
211.             }
212.  
213.             //also check the extension
214.  
215.             string extension = Path.GetExtension(location.PrimaryKey);
216.  
217.             if (!requiredExtension.Equals(extension, StringComparison.OrdinalIgnoreCase)) {
218.                 continue;
219.             }
220.  
221.             //finally, get the name of the asset and hash it
222.  
223.             string name = Path.GetFileNameWithoutExtension(location.PrimaryKey);
224.             int hash = TMP_TextUtilities.GetSimpleHashCode(name);
225.  
226.             if (!lookup.TryAdd(hash, location)) {
227.  
228.                 Trace.LogError($"Trying to register multiple font assets: {location.PrimaryKey} - {location.ResourceType}. Required type: {type}");
229.             }
230.         }
231.     }
232. }

Hope it helps

 

This solution deserves more accolades! It's nice and elegant, and doesn't require editing of the base TextMeshPro package. Thanks again, I've used it to great effect, and works great with only a few minor changes.

 

Great solution! Thank you for sharing it.

While working with this I've noticed that addressables loading does not handle type check as resource loading do. So for example if you try to load location that is of type Material, but you specify that it is some other type you will get exception. In resource loading you would get null in this case.

So, to handle this case I have added a simple check if location is of correct type, before attempting to load it.
Like so:

 

I'm getting an exception inside WaitForCompletion() method when I give this a try. Does anybody know why that may be the case?

Code (CSharp):

1. InvalidCastException: Unable to cast object of type 'DownloadHandlerBuffer' to type 'DownloadHandlerAssetBundle'.
2.   at UnityEngine.ResourceManagement.Util.UnityWebRequestUtilities.IsAssetBundleDownloaded (UnityEngine.Networking.UnityWebRequestAsyncOperation op) [0x00000] in <00000000000000000000000000000000>:0
3.   at UnityEngine.ResourceManagement.WebRequestQueue.WaitForRequestToBeActive (UnityEngine.ResourceManagement.WebRequestQueueOperation request, System.Int32 millisecondsTimeout) [0x00000] in <00000000000000000000000000000000>:0
4.   at UnityEngine.ResourceManagement.ResourceProviders.AssetBundleResource.WaitForCompletionHandler () [0x00000] in <00000000000000000000000000000000>:0
5.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationBase`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
6.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle.WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
7.   at UnityEngine.ResourceManagement.AsyncOperations.GroupOperation.InvokeWaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
8.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationBase`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
9.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
10.   at UnityEngine.ResourceManagement.AsyncOperations.ProviderOperation`1[TObject].InvokeWaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
11.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationBase`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
12.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
13.   at UnityEngine.ResourceManagement.ChainOperationTypelessDepedency`1[TObject].InvokeWaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
14.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationBase`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
15.   at UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle`1[TObject].WaitForCompletion () [0x00000] in <00000000000000000000000000000000>:0
16.   at TextMeshProResourcesAPI.LoadAddressableAsset[T] (System.Object key) [0x00000] in <00000000000000000000000000000000>:0

Unity version is 2021.3.21f1.
Addressables version is 1.20.3.

 

This isn’t going to work with remote bundles. You need the text mesh pro bundles to be local

 

As far as I can tell, it is placed in a local bundle. Am I missing something here?

 

Try switching off this setting:

 

Yes, that did it. Thanks a lot folks!