Search Unity

TextMesh Pro Version 1.4.1-Preview 1 with Dynamic SDF for Unity 2018.3 now available!

Discussion in 'UGUI & TextMesh Pro' started by Stephan_B, Jan 31, 2019.

  1. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    At long last, the wait is over!

    Today, I am pleased to announce the release of the first preview of TextMesh Pro version 1.4.0 for Unity 2018.3 with support for Dynamic SDF.

    One of the most requested feature since the initial release of TMP has been support for Dynamic SDF which is now available in this new release.

    Dynamic SDF makes it possible to add glyphs and characters to font assets at runtime thus significantly enhancing and simplifying workflows for handling localization and languages like CJK with large character sets. This new functionality is supported throughout TMP and its fallback system.

    In order to support this new feature, structural changes to Font Assets and Sprite Asset have been necessary and will require existing font assets and sprite assets to be upgraded. This upgrade should be automatically handled by the system as these resources are loaded / accessed in your projects. Although I have been testing this process using dozens of user projects where everything worked fine, I am releasing this new version as a preview as I need your help in further testing and making sure this process works as expected. With that said ...

    Steps to Upgrade to this new release
    (1) Backup your project first. This is a preview release so let's not take any chances.
    (2) Open this backup of your project in Unity 2018.3.
    (3) Close any scene that may already be opened in Unity. Although the asset migration process should still work. It is preferable to close any opened scenes.
    (4) Open Package Manager UI via the "Window - Package Manager" menu and as per the image below select the latest preview version available for version 1.4.0.

    upload_2019-1-31_13-33-33.png

    (5) Start opening scenes to see if everything works as expected. If not please report these issues in this thread and as usual I'll get those addressed.

    (6) If your project contains many large font assets where many of them happen to be assigned as fallbacks in the TMP Settings, opening the TMP Settings which would force all these font assets to converted as a group might appear to lock up the editor. So if you run into this, instead of accessing the TMP Settings, just select each of these font assets one at a time in the project tab. Wait for each of them to be converted before selecting the next one.

    This should be it in terms of the migration.

    Next is getting familiar with the new Dynamic SDF system and added flexibility it will offer and new potential workflows in regards to managing these resources. I have already published the following video which covers a big chunk of the new Dynamic System and Fallback.

    The following new video covers the upgrade process as well as provides insight on the structural changes to Font Assets and Sprite Assets. The video also covers some new additional functionality that had not previously been shown / discussed.



    FAQ
    Question 1

    Can the content of a font asset be cleared / reset?

    Yes. In the Editor, you can use the Context Menu option Reset which will clear the Character, Glyph and Glyph Adjustment Tables as well as reset the atlas texture back to size 0.

    Via scripting you can use the following function.
    Code (csharp):
    1. public void ClearFontAssetData(bool setAtlasSizeToZero = false)

    Question 2

    I am trying to switch a previously created font asset to dynamic mode and getting a warning about the atlas texture not being readable. How do I set the texture to readable?

    In the font asset inspector, double click on the Font Atlas to navigate to the atlas texture. Then switch the inspector to debug mode where the Is Readable flag will now be exposed.

    upload_2019-2-1_14-48-38.png

    Question 3
    Can a font asset be created at runtime?

    Yes. The font asset can be created using the following function.
    Code (csharp):
    1. public static TMP_FontAsset CreateFontAsset(Font font, int samplingPointSize, int atlasPadding, GlyphRenderMode renderMode, int atlasWidth, int atlasHeight, AtlasPopulationMode atlasPopulationMode = AtlasPopulationMode.Dynamic)
    As more questions or suggestions come up, I'll add those to the FAQ.

    Feedback
    Please report any issues in this thread. Feel free to ask questions if anything is unclear. I am here and available to help.
     
    Last edited: Mar 26, 2019
  2. Fachmah

    Fachmah

    Joined:
    Jan 11, 2017
    Posts:
    5
    Loving the update! Everything is working really well so far, however I have one question. Is it possible to make a dynamic font file overwrite itself when it has reached max capacity? For instance having a new character overwrite the first character that was added to the atlas.

    Thanks again for the preview - much appreciated!
     
  3. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Glad it is working well and you are most welcome :)

    In terms of the question, the challenge with overwriting a specific glyph is that glyphs come in all shapes and sizes and as such the new glyph could be bigger than the one it is replacing. So doing this could be tricky.

    Having said that, Font Assets can now be Dynamic in terms of adding glyphs and characters to them at runtime but Font Asset themselves can be created at runtime.

    Code (csharp):
    1.  
    2. // Create a new font asset at runtime.
    3. m_DynamicFontAsset = TMP_FontAsset.CreateFontAsset(SourceFontFile, 90, 9, GlyphRenderMode.SDFAA, 1024, 1024, AtlasPopulationMode.Dynamic);
    4.  
    5. // Force some characters to be added to the font asset. Ie. as opposed to some text object forcing those to be added).
    6. m_DynamicFontAsset.TryAddCharacters("1234");
    7.  
    8. //  Assign newly created font asset to some text object.
    9. m_TextComponent.font = m_DynamicFontAsset;
    10.  
    11. // Set the text on the object which in turn will force these to be added to this new font asset dynamically.
    12. m_TextComponent.text = "ABCD";
    13.  
    Given the above code and assuming those characters were available in the font file, the font asset and atlas texture would end up with "1234" and "ABCD" in it.

    You can also clear the font asset data using the following. This clears the glyph, character and glyph adjustment tables as well as atlas texture (reset to 0 X 0 in size) but maintains all the relevant settings so you can use it normally and resume adding characters to it.

    Code (csharp):
    1. m_DynamicFontAsset.ClearFontAssetData();
    Between all of these, you can create new font assets, pre-populate them with some characters. Reset them to clear out their characters and all of that at runtime.
     
    Ubrano likes this.
  4. BBO_Lagoon

    BBO_Lagoon

    Joined:
    Mar 2, 2017
    Posts:
    199
    Hi Stephan,

    First, thanks for all your work !
    I'm really happy with the dynamic generation.

    There is a problem still.
    With texts that are animated (with scale), lot of time on my UIs I have texts that are scaled to 0 and then show with a tween from scale 0 to 1. These texts show as white squares. Texts that start with small scale but not 0 are blurry.
    If I disable / enable the TextMeshPro component all goes weel.

    Would be good to fix that.
     
  5. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Most of the time this issue is the result of the scale / tweening not doing uniform scaling on the X, Y and Z. For certain, a scale of zero on the Y of the text object would cause the behavior you describe.

    Note that I did change the handling of the SDF scaling so if the issue persists, I'll take a look at it and if you could please provide some simple repro project / scene that I could use to reproduce the behavior which usually greatly speeds up the time it takes to resolve this issues.
     
  6. Fachmah

    Fachmah

    Joined:
    Jan 11, 2017
    Posts:
    5
    This makes a lot of sense, I guess I'll work in the ClearFontAssetData() instead. It works the same way as pressing reset in the inspector would right? :--)
     
  7. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    It does indeed. I use it myself when I need to reset / clear a font asset for testing dynamically adding characters in builds / on devices.

    Now just to make sure I don't overlook anything with respect to your use case, what was your thought with regards to doing it one character at a time?

    P.S. Since it is already 4:00 on my end, I am going to go sleep but I'll follow up tomorrow.
     
  8. BBO_Lagoon

    BBO_Lagoon

    Joined:
    Mar 2, 2017
    Posts:
    199
    Here is the repro project. You can see the problem appear with TextMeshPro 1.4 but not with the 1.3
     

    Attached Files:

  9. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Thank you. I'll take a look tomorrow when I wake up.

    Update - I know what the issue is and will have a fix / code change you can make tomorrow.
     
    Last edited: Feb 1, 2019
  10. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    @Binouze Thanks for the time you took to prepare the Repro project as it allowed me to uncover another issue which I would not have otherwise.

    The following changes will be necessary to resolve the issue.

    upload_2019-2-1_16-32-47.png

    Let me know if this resolves the issue. If so these changes will be included in the new preview release.

    Special Note: As of Unity 2018.3, packages are additionally cached on a per project basis where these local cached version are located in the project folder in "Library/PackageCache/..." which is where these changes should be done for the given project.
     
    Last edited: Feb 3, 2019
  11. LeeSinLcj

    LeeSinLcj

    Joined:
    Feb 22, 2017
    Posts:
    5
    Hello Stephan,I'm very glad to use the dynamic font system,but I found a GC.Alloc issue when using it. TIM截图20190202162222.png
    As you can see, this method eat 2.3MB heap memory per call, and when I try to type some chinese word in the IuputField, memory grow up very fast.I notice that in your code there is a TODO work, TIM截图20190202162740.png so could you fixed it or give me some advice?Thank you very much!!!
     
  12. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    By the time I got around to adding the ability to retrieve Glyph Adjustment Data (Kerning) dynamically, it became clear that I was going to need to make changes to the FontEngine and bindings which unfortunately are part of Unity itself and not part of the TMP package. As such this will require a Unity update which I should be able to get approved for 2019.1 but unlikely for 2018.3 given focus now is all on stability.

    As per my TODO note, I am fully aware of the shortcomings of that current implementation and not happy about it but had limited options if I wanted this functionality short term.

    Having said that, I could add new settings to the TMP Settings where trying to retrieve Glyph Adjustment Data dynamically at runtime could be enabled / disabled.

    For some languages Kerning is pretty common but for CJK not as much so maybe this could be a viable short term solution.

    P.S. Can you give me an idea of the number of glyphs your font asset contains?
     
    Last edited: Feb 2, 2019
  13. LeeSinLcj

    LeeSinLcj

    Joined:
    Feb 22, 2017
    Posts:
    5
    We are using Source Hans Sans now and it contains over 60000 glyphs.
     
  14. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    How many glyphs are you planning to add / does your font asset contain?

    For any given project / per language set, you are likely to only end up using a very small subset of those 60,000 glyphs.

    My recommendations remain to have your Primary font assets (for any given language or groups of languages) contain the known characters / glyphs in the project which are the characters found in your the your titles, menus, ui, etc. This font asset should be prepared ahead of time just like before and be static.

    Then to handle user input and all other potential characters, assign to your primary static font asset a fallback that is dynamic and there to catch all other (unknown / not already part of the project) characters.

    This will provide for best performance and quality for all the known characters / text in the project. The primary would already contain the Glyph Adjustment Data (Kerning) since it is static and prepared ahead of time. Then for all other characters, where dynamic fallbacks are used there will be a higher performance overhead but your likely to only have to handle a small number of characters and furthermore given humans don't type that fast relative to computer processing speed, the performance overhead will not be noticeable by any users.

    Although you could rely on the dynamic system to handle all characters without preparing / using any static font assets, I don't think that's the best strategy for managing these resources.

    P.S. Let me know if you feel adding the ability to disable dynamic retrieval of Glyph Adjustment Data in the TMP Settings would be useful?
     
    Last edited: Feb 2, 2019
  15. LeeSinLcj

    LeeSinLcj

    Joined:
    Feb 22, 2017
    Posts:
    5
    Thanks for your advice,I will try to use both static font and dynamic font, but I'm afraid that it could also cause memory leak issues when dynamic character was been created.
     
  16. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Fix an issue where creating Font Assets at runtime were not getting their version number set correctly to version 1.1.0.

    This change will be included in the next preview release but just in case, you wish to experiment with font asset creation at runtime, here are the relevant changes.

    upload_2019-2-2_15-55-58.png

    Special Note: As of Unity 2018.3, packages are additionally cached on a per project basis where these local cached version are located in the project folder in "Library/PackageCache/..." which is where these changes should be done for the given project.
     
    Last edited: Feb 3, 2019
  17. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Adding a new character and glyph to a font asset will result in an allocation of 64 bytes for the glyph and 40 bytes for the character. There is also another allocation of 76 bytes for an internal list which I will try to re-work / remove. So in the next release version 1.4.0-preview.2 (which I'll release this week) the one time allocation per character & glyph added to a font asset will be 180 bytes and if I can do away with the internal list that will reduce it by 76 bytes.

    I have already added the ability to disable retrieving Glyph Adjustment Data (Kerning Pairs) dynamically. This option will be in the TMP Settings. I have already re-worked / removed the use of .ToArray() which was responsible for the bulk of those allocations. The revising implementation will be in another TMP package release which I should be able to get into 2019.1 and will do my best to get in 2018.3. Regardless I'll look for some alternative for 2018.3 just in case.

    Allocations per Glyph Pair Adjustment record is 64 bytes. So if you add a character & glyph that has 10 pairs then it will allocate 640 bytes.

    Unless you are adding new characters, glyphs and adjustment pairs there should be no allocations.
     
  18. BBO_Lagoon

    BBO_Lagoon

    Joined:
    Mar 2, 2017
    Posts:
    199
    Hi Stephan,
    Thanks for the fix, it works now :)
     
    Stephan_B likes this.
  19. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Glyph Adjustment Table Changes
    In preparation for the addition of support for OpenType Font Features, the Glyph Adjustment Table presentation and data structure has been changed as seen below.

    upload_2019-2-11_18-8-16.png

    The updated Glyph Adjustment Table now shows the glyphs for each of the Glyph Pair Adjustment Records. Glyphs will only be shown if they are already present in the Glyph Table.

    Individual pairs now reference the glyph index as opposed to unicode characters. When adding new adjustment pairs, the Character or its UTF16 or UTF32 representation can be used to lookup the glyph index for the given character.

    Note that it is now possible to define / add GlyphPairAdjustmentRecords for glyphs that are not already present in the Glyph Table. This can occurs when the new "Get Font Features at Runtime" option is enabled in the TMP Settings.

    upload_2019-2-11_18-18-48.png

    When this option is enabled and as new glyphs are added to the Glyph Table, Glyph Pair Adjustment Records for those new glyphs will be automatically added to the Glyph Adjustment Table.

    Font Assets that contain existing legacy kerning data will be automatically updated to the new format when these font assets are first loaded in the Editor or at Runtime (if somehow these font assets never ended up being loaded in the Editor during development or testing).

    This new change will be included in the next preview release of TextMesh Pro package. This new release will be version 1.4.0-preview.2 which I am in the final stages of testing.
     
    Last edited: Feb 12, 2019
  20. evilock

    evilock

    Joined:
    Sep 19, 2016
    Posts:
    20
    With the dynamic font, what would be the recommended way to manage the situation where the atlas reaches it's maximum capacity? My use case would be an app where the user can just keep viewing content (in Chinese), gradually filling up the atlas with new glyphs, potentially up to the point that no new ones can be added. Even as a workaround, would there be a way to detect when this happens, so that we could then clear the dynamic atlas?
     
  21. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    As characters and indirectly glyphs are requested, TMP will search (like it did before) the primary font asset and potential fallbacks. In this process if any font asset is set to Dynamic, TMP will try to add the requested character / glyph to it but if it is unable to do so, it simply moves to the next fallback.

    One potential way to handle an ever growing list of characters / glyphs would be to create (ahead of time) several fallbacks that are all empty with the relevant generation settings and to assign them to this primary font asset or wherever in the fallback chain you want them.

    Assuming you ended up creating 10 empty fallback font assets with atlas texture size of 1024 x 1024, one of the issue would have been all those textures being allocated but as it turns out, one of the last changes / improvements I made before releasing this new version was to have the atlas texture of these newly created / empty font assets uninitialized / of size zero until you try to add the first character. As such, this makes creating / using several empty font assets very efficient.

    Furthermore, it is also possible to create font assets at runtime which won't be too useful in this case yet given I have not exposed any API to get callbacks to allow you to manage / create new font assets as needed but something that would enable the dynamic system to create font assets on demand or additional atlas textures which font assets are already setup internally to handle. Regardless, the ability to either add additional atlas textures per font asset or to create fallback font assets on demand is certainly all possible.

    So short term, go with creating whatever number of empty fallbacks you believe will be needed. Keep in mind that these fallbacks can have different sampling point size and padding but still need to have the same ratio of sampling to point size as the primary (reason for that is explained in that video and in many others as well).
     
  22. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    961
    Hi, just wondering, with 1.4.x in the works, is there any plan to support the Addressable system? As in, can we avoid using Resources folder (for TMP essential assets)?
     
    IntegralAR and mandisaw like this.
  23. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    As far as I know Addressables do not currently allow resources to be load synchronously which is an issue when the text object needs its font asset in order to display text. Until then we still need to use the resources folder.

    Having said that, I am likely to add a callback like OnFontAssetRequest, OnSpriteAssetRequest, etc. which users would be able to use to implement their own loading handling of resources like from Asset Bundles and perhaps Addressables given you would have some level of control over how these resources are loaded.
     
    IntegralAR and mandisaw like this.
  24. Shadowing

    Shadowing

    Joined:
    Jan 29, 2015
    Posts:
    1,648
    Any chance we can get Text Mesh Pro to be moved from Windows to go under Tools instead in the editor?
    There is enough stuff under Windows anyways.

    I'm currently going around to all the major used assets on the Asset store and asking if they will do the move as well.
    This way we create a more of a Global Standard.

    Several already use Tools
    Odin
    Opisive
    Natural Manufacture
     
  25. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    New preview release is now available. Release is version 1.4.0-preview.2a.

    It addresses the issues reported thus far along with the changes to the Glyph Adjustment Table I referenced in the post above.

    See the ChangeLog for more details.
     
  26. LeeSinLcj

    LeeSinLcj

    Joined:
    Feb 22, 2017
    Posts:
    5
    Hi Stephan,after I use Unity 2019.1b to build my project on Android with TMP 1.4.0-preview.1b,it will crash when any UI Forms which has used TMP component,and adb logcat says that its an out of memory issue.Do you have any idea about this problem?
     
  27. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Not without more information but let's first check if you get the same behavior using 1.4.0-preview.2a.

    If you still get the issue, you could either submit a bug report with the project and then provide me with the case # or if easier check if you are able to reproduce the behavior in some new project / simple scene which you could provide me to look at.

    P.S. If I was to take a wild guess, it would be extracting of Kerning data at runtime which in 1.4.0-preview.1b results in way too much allocation which should be much better in 1.4.0-preview.2a.
     
    Last edited: Feb 18, 2019
  28. JacketPotatoeFan

    JacketPotatoeFan

    Joined:
    Nov 23, 2016
    Posts:
    34
  29. CanisLupus

    CanisLupus

    Joined:
    Jul 29, 2013
    Posts:
    427
    Hey Stephan, this update is very useful, thank you!

    I ran into a few problems trying to do some things in version 1.4.0-preview.2a:

    1.

    For some fonts assets, like a font for Chinese, we create several empty font assets as fallbacks just to have space for more dynamic characters if TextMesh Pro needs it.

    To avoid filling our project with dummy font assets, I was trying to create a certain number of fallback font assets at runtime, but it seems that adding those at runtime results in "Type mismatch" in the fallback list in the inspector, so I'm not sure if this is working or if it's only that the inspector can't display assets in memory (and not in the project).

    They are created like this:

    Code (CSharp):
    1. TMP_FontAsset originalFontAsset = ...;
    2.  
    3. TMP_FontAsset fallbackFontAsset = TMP_FontAsset.CreateFontAsset(
    4.     originalFontAsset.sourceFontFile,
    5.     originalFontAsset.faceInfo.pointSize,
    6.     originalFontAsset.atlasPadding,
    7.     GlyphRenderMode.SDFAA,
    8.     1024,
    9.     1024,
    10.     AtlasPopulationMode.Dynamic
    11. );
    12. originalFontAsset.fallbackFontAssetTable.Add(fallbackFontAsset);

    2.

    The reason I can't test if this is working is due to another issue.

    When changing language in our app, I read all unique characters from our translation and immediately add those characters to the font to avoid future hitches.

    Learning from posts in this thread, I tried using TryAddCharacters to add them.

    Code (CSharp):
    1. fontAsset.TryAddCharacters(chars);
    However, there was a NullReferenceException on line 1150 of TMP_FontAsset:

    Code (CSharp):
    1. character.glyph = m_GlyphLookupDictionary[glyphIndex];
    Based on the code I then tried calling ReadFontAssetDefinition beforehand and it fixed that:

    Code (CSharp):
    1. fontAsset.ReadFontAssetDefinition();
    2. fontAsset.TryAddCharacters(chars);
    But now when adding the characters I get a KeyNotFoundException in line 1205:

    Code (CSharp):
    1. character.glyph = m_GlyphLookupDictionary[character.glyphIndex];
    It may not be this, but it looks like FontEngine.TryAddGlyphsToTexture couldn't fit all characters in the atlas (since the main font asset is small), so the number of glyphs it returned was less than m_CharactersToAdd.Count, breaking the lookup in the for cycle. But maybe not. ;) Can you look into this?

    3.

    A question. Does TryAddCharacters try to add characters to the fallbacks of the font asset if it fails on the main one? I know TextMesh Pro does that dynamically, but we can't tell what happens when calling TryAddCharacters manually.

    If it doesn't, is there a way to know which characters were correctly added so we can TryAddCharacters in the fallbacks ourselves?

    Thanks a lot!
    Daniel
     
  30. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    This is expected behavior in the Editor / Inspector as non-persistent / scene only assets cannot be referenced by persistent assets. Having said that, this behavior does not prevent the dynamic creation of font assets and their assignment to any other font assets (persistent or not) at runtime.

    This was an issue which I have now fixed. I also added the following overload to this function which will return an array that contains the unicode characters that could not be added to the font asset.

    Code (csharp):
    1. public bool TryAddCharacters(uint[] unicodes, out uint[] missingUnicodes)
    It currently does not.

    If the function returns false, then you know not all characters were added. As per my previous comment above, I add an overload to this function that returns the unicode characters that could not be added.

    Also added similar overload to the other TryAddCharacters function

    Code (csharp):
    1. public bool TryAddCharacters(string characters, out string missingCharacters)
    Out of curiosity, how many unique characters does your largest translation file contain? It might make sense to use (1) static font asset that contains all these characters for each language / groups of languages and then simply rely on one dynamic font asset to capture remaining characters potentially coming from user input.
     
    Last edited: Feb 27, 2019
  31. CanisLupus

    CanisLupus

    Joined:
    Jul 29, 2013
    Posts:
    427
    Thank you very much for the reply.

    I thought this was the case, but good to have the confirmation. In the meantime I tested the runtime fallbacks by having a big enough atlas (2048x2048) that lets everything fit when TryAddCharacters happens. It works fine. When we don't need one of these fallback font assets anymore we call fallbackFontAsset.ClearFontAssetData(setAtlasSizeToZero: true) and it seems to recover all used up memory, which is great.

    OK, that is fantastic! Any ETA on this next version of TextMesh Pro?

    They are not many. A little more than 500 unique characters for chinese. We analysed that possibility, however, we prefer to create the font asset at runtime when the app language changes (and on startup) to avoid having to update a static font asset in the project every time the translation changes. (And also to avoid having all static font assets loaded in the fallbacks and wasting memory, though we could load them dynamically from Resources.)

    We do have static font assets for latin fonts, because they are not very big even with all supported characters included from the start.

    Before this dynamic SDF preview, for chinese we had one atlas with 3500 chars, another with 3000, another with 1605 (all these from a post of yours on another thread, so thanks for that ;)) and then another with punctuation, just to cover "things that might appear on user input". So dynamic atlases are a massive improvement.

    We were previously loading font assets from Resources as required to avoid giant font assets like chinese and its fallbacks wasting memory even when not needed at all. Now with dynamic SDF we have "base" font assets with a fallback for every alphabet and these fallbacks are dynamic and all empty from the start. Then we add characters to the right font asset (from the translation) when a language is set. When a character that is not in the set appears in text (for example a user name or a name in the credits), TextMesh Pro does that for us since the assets are dynamic.

    If you have questions about this setup feel free to ask.

    Thanks!
     
    Last edited: Feb 27, 2019
  32. CanisLupus

    CanisLupus

    Joined:
    Jul 29, 2013
    Posts:
    427
    Another thing. We found what we believe is a bug in input validation. It happens in the preview; not sure if it happens on previous TMPro versions but it seemed to work well before. :)

    The following code will print 0 before any inserted character. So if you write "a" it will print 0 and then 97:

    Code (CSharp):
    1.     void Start()
    2.     {
    3.         inputField.onValidateInput += Validate;
    4.     }
    5.  
    6.     private char Validate(string text, int charIndex, char addedChar)
    7.     {
    8.         Debug.Log((int) addedChar);
    9.         return addedChar;
    10.     }
    Even pressing keys like alt will cause a 0 to be printed. It's possible this is the character that represents the cursor but we're not sure. Either way, we're now ignoring '\0' in our validation method.

    Cheers!
    Daniel
     
  33. Kirsche

    Kirsche

    Joined:
    Apr 14, 2015
    Posts:
    121
    Thanks for sharing this amazing update! I'm happy that this is backwards compatible and it's so easy to re-generate fonts.

    My only (very tiny) issue is that we can't rename the previously created default material. Back then, I thought this would be a good idea:

     
  34. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Since the material and atlas texture are sub assets, the Unity rename function doesn't work on sub objects.

    How did you change that name initially? Was the font asset named that way?
     
  35. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    TMP package version 1.4.0-preview.3a has been released.

    This release includes updated Distance Field and Mobile Distance Field shaders to support single pass stereo rendering.

    Includes a few minor fixes.
     
    Ecnassianer and Kirsche like this.
  36. Kirsche

    Kirsche

    Joined:
    Apr 14, 2015
    Posts:
    121
    Yes, it was the name of the font asset. Anyway, I'll probably try to re-create the font asset and then edit the GUID from the .meta file so references don't break.
     
  37. CanisLupus

    CanisLupus

    Joined:
    Jul 29, 2013
    Posts:
    427
    @Stephan_B You probably didn't see my question in the previous posts. When do you think we'll have the overloads for TryAddCharacters? Next preview? Further along? Thanks!

    @Kirsche We've renamed a font asset default material before. We opened the font asset as text (in any editor), searched for the material name and simply renamed it to what we wanted. Of course, this will only work if your project's assets are serialized as text, but this is the Unity default since some time ago, and for source control they probably should be. :) Unless Stephan says there's something wrong with doing this, it should work.
     
    Kirsche likes this.
  38. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    It is included in preview.3a.
     
  39. CanisLupus

    CanisLupus

    Joined:
    Jul 29, 2013
    Posts:
    427
    I somehow completely missed it. Sorry and thanks.
     
  40. evilock

    evilock

    Joined:
    Sep 19, 2016
    Posts:
    20
    Ending up with several dynamic fallback atlases in use unsurprisingly starts to eventually have somewhat of a performance and memory impact as the glyphs keep adding up, so I’m now trying to figure out a way around that. I do need to be able to support a large amount of (Chinese) characters in my game to show user input (chat), but the amount of individual characters concurrently displayed at any time is never huge. So keeping all the glyphs ever added in the fallback atlases doesn't seem optimal in my scenario.

    For my purposes I'm tempted to keep the fallback capacity low since adding and retrieving with something like a single 1024x1024 fallback atlas is really efficient. Then I'd just clear it when it's filled up, when HasCharacters and TryAddCharacters return false. However as there may be no natural phase in the UI flow to do it, this would then necessitate some way to repopulate the atlas on the fly with characters currently in use in order to not have all displayed text disappear when the fallback is reset. Would TMPro facilitate any convenient way to do this or is there some other approach you would suggest?

    Thanks for the excellent update and support!
     
  41. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    It is important to keep in mind that changes to font assets and their atlas textures in the Editor are persistent. However, that is not the case in builds and as such these changes are not persistent between play sessions resulting in dynamic font assets and fallbacks resetting to their original state for each play session. This should minimize / limit the number of glyphs ultimately being added / needed.

    The following function will allow you to clear a font asset / fallback. In your case, I would not reset the atlas size to zero as you expect to resume adding glyphs into it right away.

    Code (csharp):
    1. public void ClearFontAssetData(bool setAtlasSizeToZero = false)
    To test this in the Editor, I did override the Context Menu Reset option on the font assets which uses the above function. An important part of that however if a callback (in the Editor only) which is as follows:

    Code (csharp):
    1.  
    2. // Called in this Reset function
    3. TMPro_EventManager.ON_FONT_PROPERTY_CHANGED(true, fontAsset);
    4.  
    5. // Implementation in the TMPro_Private.cs and TMPro_UGUI_Private.cs
    6. void ON_FONT_PROPERTY_CHANGED(bool isChanged, TMP_FontAsset font)
    Although this Callback is not available at runtime, you can review the source code to see what it actually calls on the text object. BTW: the UpdateMeshPadding() part can be ignore in this case as the font asset data like point size or padding values would not be changes and this call would not be necessary.

    Assuming you have a list of text objects in your scene, you could implement you own method to force an update of the text objects. You could use ForceMeshUpdate() which pretty much does the same thing as the callback but without checking if the text object is using the font asset that was just reset which is something you should handle.

    Assuming you had a bunch of text objects on screen with text, doing this reset followed by forcing the text objects to update would result in whatever visible glyphs being added again. Ie. no need to figure out manually what should be added or not. Again, for testing create a few text objects that contain some text and then use the Context Menu Reset on the Font Asset and you will see whatever characters / glyphs being used added back in the atlas right away whereas those no longer used not being added.

    I guess at some point I could add a runtime callback but until then you can implement your own manager to force an update on text objects that are using the font asset / fallback that were just reset.
     
  42. evilock

    evilock

    Joined:
    Sep 19, 2016
    Posts:
    20
    Yes, it's indeed even during a single play session I need to consider the atlases filling up.

    All right, that was my conclusion too after poking around the source code. I was hoping I missed something because I kind of wanted to avoid having to do the managing part but that's what I'll do then. Thanks a lot for your input Stephan!
     
  43. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    Are you using a static font atlas that contains the known text in the project? That would reduce the amount of potential glyphs needed to be added.
     
  44. evilock

    evilock

    Joined:
    Sep 19, 2016
    Posts:
    20
    Haven't got to that yet but I'm intending to. Hasn't really been a concern since the known text has just a couple of hundred distinct characters over several UI views, while for the user input I need to support around 8000.

    I'm now refreshing the text fields after clearing the fallback, this approach seems to be working perfect for my purposes with just a single 1k by 1k dynamic atlas. Working very nicely with minimal memory & performance footprint!
     
    Stephan_B likes this.
  45. SniperED007

    SniperED007

    Joined:
    Sep 29, 2013
    Posts:
    345
    When I upgrade I get the following error:

    Library\PackageCache\com.unity.textmeshpro@1.4.0-preview.3a\Scripts\Editor\TMP_SettingsEditor.cs(332,91): error CS0407: 'Object TMP_ResourceImporterProvider.GetTMPSettings()' has the wrong return type
     
  46. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    What version of Unity and TextMesh Pro are you upgrading from?

    The line of code giving this error is simply
    Code (csharp):
    1. return Resources.Load<TMP_Settings>("TMP Settings");
    so maybe to package cache or possibly this file got corrupted.

    Have you modified your TMP Settings? If not, re-import the TMP Essential Resources.
     
  47. SniperED007

    SniperED007

    Joined:
    Sep 29, 2013
    Posts:
    345
    TMP ver 1.3.0
    Unity 2018.3.8f1

    I rolled back to 1.3, tried re-importing the TMP Essentials and then upgrading to 1.4 again... same issue.
     
  48. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    I just installed Unity 2018.3.8f1.

    Created a new project and made sure it was using TMP version 1.3.0. Imported the TMP Essential Resources (from 1.3.0), then upgraded to 1.4.0-preview.3a and no errors.

    Also tested, upgrading to 1.4.0-preview.3a and the importing the TMP Essential Resources and now errors in that case either.

    See if you get the same behavior when creating an empty project and then upgrading TMP in that project to 1.4.0-preview.3a and then importing the updated TMP Essential Resources from that release.

    BTW: The function signature and return type is as follows:
    Code (csharp):
    1. static UnityEngine.Object GetTMPSettings()
    so that should be fine.

    Let me know what happens testing with a new / empty project. If the behavior is the same, then delete the package cache from "...\AppData\Local\Unity\cache"
     
  49. JeromeHdz

    JeromeHdz

    Joined:
    Mar 7, 2019
    Posts:
    4
    Hello Stephan_B

    First of all thank you for your amazing work and tutorials about TMP, it's really appreciated and helpful!

    I'm a beginner with TMP and I had one question about the fallback fonts,

    If we want to earn some spaces (for mobile project for example) with Chinese characters, you suggested to create one main fonts with the characters used in the project and then some fallback fonts with the remains Chinese characters not used in the project.

    My questions are why do we need it? What is the purpose of the fallback fonts if finally the characters will never be used in the final project? (okai finally it's two questions)

    I asked this questions because I want to space a maximum of space in my project.
     
  50. Stephan_B

    Stephan_B

    Joined:
    Feb 26, 2017
    Posts:
    6,595
    The fallback would only be necessary to handle characters not already present in the Primary and coming from user input. If there is no user input then a fallback would not be necessary.

    Having said that, since Fallback font asset atlas textures are not sized until you add characters to them, there is no downside to including an empty fallback just in case somehow your Primary font asset was missing some character.