TextMesh Pro Korean, Japanese and Chinese Localization issues

Thanks again for all of the help with this.

I have these two font assets in my build and I'm wondering if I need both of them

17.9 mb 2.2% Assets/_Core/Fonts/Fonts/_Main/NotoSansCJK-Regular.ttc/NotoSansCJK-Regular.ttc
1.1 mb 0.1% Assets/_Core/Fonts/Fonts/_Main/NotoSansCJK-Regular.ttc/NotoSansCJK-Regular SDF.asset

Can I use TMP with just the 1.1mb asset file or do I need to include both?

 

Hi @Stephan_B ! I encountered an issue importing NotoSansJP (regular) from Google Fonts into my project. After I use FontAssetCreator with the text from file, my font atlas looks ok, but when I press save - it gives an error in the console (see img attached, step 2). And seems like in the generated asset I have no material for the font. Then in runtime I have lots of exceptions telling me that font material is unassigned. I'm using Unity 2020.1.17f1 and TMP 3.0.3
Any help would be highly appreciated.
https://imgur.com/a/6FsSSXy

 

Have you imported the TMP Essential Resources in the project? More specifically are the TMP Shaders present in the project?

 

Thanks so much for the prompt reply. Yes I have them. I didn't mention but I have Alegreya font imported before through Font Asset Creator and I didn't encounter any issues, it works perfect, maybe this info would be of help somehow.

 

As per the image above, most of the shaders appear to be missing. TMP Essential Resources were also updated several releases ago where a few new shaders were added as well as the Shader folder was almost moved outside of the Resources folder. I would suggest deleting this current Shader folder and re-importing the TMP Essential Resources.

Here is an image showing the "TextMesh Pro" folder hierarchy showing the Shader folder outside of the Resources folder and all the shaders.

 

Yay! It works after I imported essentials. Thanks so much Stephen!
But another thing happened. I thought if I specify characters list for my font as a text file with all the texts I have in the game - TMP will make sure I have all those characters included in an atlas of specified size, and if they don't fit - it'll make another atlas of the same size and use multi atlas option.
But seems like it just excludes all the characters that don't fit into the original 1024*1024 atlas. And the option for multi atlas is disabled in the resulting font asset. So how do I go about including everything what I need into a font asset from a file with Japanese texts? The amount of excluded chars is more than 400 even if I choose 2048*2048 atlas.

 

Dynamic Font Assets with Multi Atlas Texture enabled will create new atlas textures as needed to fit those additional glyphs. This functionality is not available in the Font Asset Creator (yet).

There might be a way to cheat by creating a dynamic font asset with multi atlas texture enabled and then creating a script and adding those characters to the font asset. Then switch the font asset to static... I haven't tested it but it should work.

What sampling point size and padding are you using for the CJK characters?

 

Oh that's sad. The system is so well designed and feature rich I thought it should support the basic flow most of mobile game devs use (at least the devs I know personally ). We store all texts for a game in *json files as a pair [key:value]. We use the keys(contain ascii chars only) when assigning texts for UI elements and in runtime we update those keys with real values which are strings from different languages. So for us it would be super nice to have everything generated at the time when we create a font asset. And so it was till I met Japanese It appeared I need to have around 1000 glyphs to show all the text I have in game. And for that I need multi atlases.
The sampling point I use is 70 and padding is 5.
Thanks for the suggestion though, I think I will use it.

 

Adding support for Multi Atlas in the Font Asset Creator should not be difficult.

When Auto Sizing is selected, all glyphs will be packed in a single atlas texture. When Custom Size is selected, then the glyphs will be packed in as many atlas textures as needed to fit all of them based on the specified sampling point size and padding values.

Does the above make sense for your workflow?

I take a look at adding it to the next preview release which will be version 1.6.x for Unity 2018.4, 2.2.x for Unity 2019.4 and version 3.1.x for Unity 2020.x or newer.

 

Yes, that'd be perfect! That's exactly how I imagined it should work

Thanks for the great tool and great support! Hope to see the changes live soon.

 

I change atlas population mode to dynamic and its work

 

Greetings to all,
I ask for help in creating a font. There is such a game Solasta - Crown of the Magister. I made an independent translation according to the instructions at https://github.com/ThyWoof/SolastaUnofficialTranslations
Everything turns on in the game, but instead of the Russian language, squares. As I understand it, you need to find the font somewhere in the game files: globalgamemanagers, globalgamemanagers.assets, globalgamemanagers.assets.resS, resources.assets and other assest files ... tried to extract all ttf and font files, but then nothing turns out already broke my head all over :-( Maybe someone from the guru will help to overcome the problem, I attach the files that I unpacked https://drive.google.com/file/d/18KbMMpV1UKMDcrg8s5L9tUpsYFWTSfiY/view?usp=sharing
Thanks in advance.

 

To revive a bit this situation.
Generally it makes sense when you know the text before hand.
But in cases like where you have to implement user input, like in player names, and chats, this approach fails. You can't have every single TMP font loaded in memory if you have to show random_name, 스티아, Юля, आरवि , 名称 on the same leaderboard you may not even have some of their characters on your font, and even some players love to put names in different languages even on your own language.
Luckly unity text does support, and that is what we are using for this cases, but it would be great to be able to use text mesh pro, so we have all our tools using only text mesh pro and not being duplicating many of the code just to support both.

It would be great if we had a toggle on the TMP settings to allow this, it could even be off by default, and users enable it if they need to, not as the recommended option for all situations.

 

The legacy text system also has to load several fonts in order to find those characters. The key difference is that this is done on the native side where this is not visible to the developer (you). In addition, the list of potential fallback is also hidden from the developer and cannot be modified.

Prior to the recent release of the latest preview versions of the TMP package, you main option was to use Dynamic Font Assets where although these could be shipped empty (ie. with the Atlas Texture at size zero), you still had to include the font file in your build or to try to find it at runtime on the target platform and then creating a dynamic font asset from there.

Now with the introduction of Dynamic OS Font Assets in these latest preview release, these pre-created Dynamic OS Font Assets can still be shipped with their atlas texture uninitialize at size zero but their source font file is no longer included the build. Instead at runtime TMP will try to find the font file by Family name and Style name on the target platform / device.

When using Dynamic OS Font Assets and although all font assets referenced as fallback will be loaded, since their atlas texture can be uninitialized (ie. size zero) the memory overhead is marginal. Furthermore instead of using a Font object, the load the font file from path thus further reducing memory overhead.

Unlike the legacy system, with Dynamic OS Font Assets you now have control over what font files will be used and in which order.

P.S. I do plan on creating an updated video to cover this new Dynamic OS Font Asset type and related updated workflows.

 

Hello, I created the font asset for Chinese and the characters are rendered in Unity Editor and in the build. I created the input field and tried to type some Chinese characters that I know are available in the font. When the android keyboard appears and I just copy past the characters they appear for a milisecond in the android input and then disappear. How do I resolve that? The keyboard is working fine in other applications but not in our game.

 

Do the characters appear correctly if you type them or is the issue specific to copy / paste?

Do you get the same behavior when copy / pasting normal Latin characters?

 

Yeah, sorry for bothering you. It was just a Input field Validator problem. Latin characters were accepted by the project code and Chinese were not.

 

Hi @Stephan_B I'm on Unity 2019.4.33 and I upgraded TMP to 2.2.0 preview 2 via Unity package manager, but I can't generate multiple atlases for Japanese language (Kanji).

I want to support multiple languages (using I2 Localization asset) and I have csv file with all translations. In future I will add more localized text to app via csv file. I read whole topic and I think I should create static and dynamic fonts.

My plan is to have:
1. Latin static font (Noto Sans Regular) for languages: English, Spanish, Polish, French, German, Italian, Turkish, Vietnamise and Portugese
2. Cyrillic static font (also Noto Sans Regular) for Russian.
3. Middle Eastern static font (Noto Naskh Arabic) for Arabic, Persian, etc.
4. Asian static font for Chinese, Korean, Japan
5. Dynamic font for Asian languages

I have a problem with 4th point. I can't generate multiple atlasses using solution you described on version 2.2.0 prev 2. Maybe I should generate static font with only characters I already have in my translation file (single atlas) and add dynamic fonts for future translations? Wdyt?

 

so by default, if I have NotoSansCJKkr-Medium SDF and generated font atlas, if I check Atlas Population mode to Dynamic and Multi Atlas Textures to true, then I can use this dynamic font creation feature with existing font and UIs? Don't need to touch change whole UI text's font file changing works?

 

That is correct.

 

Hi Guys,
I'm just hustling a lot, trying create a Textmesh asset that contains Latin and Japanese.
My problem is to fine the correct range I think.
I used the range mentioned in a earlier post + what I got LiberationSans.
So this:

20-7E,A0-FF,2000-200F,2012-2022,2026,202A-2030,2032-2034,2039-203A,203C,203E,2044,205E,206A-206F,20AC,2122,25A1,3040-309F,30A0–30FF,31F0-31FF

But then a lot of characters are missing and whatever I try, I don't get them. The TTF defiantly includes them:


What am I doing wrong, how can I find out the missing range?

 

Does anyone have a list of the hex codes for Korean and Japanese kanji? I found a great set of txt files from a post about Chinese and would love the same for Korean and Japanese. Any help would be fantastic.

 

Korean : 32-126,8200-9900,12593-12643,44032-55203, use Custom Range Decimal

Japan :
30A0-30FF
3040-309F

use Hex Range