Request: Log how many Glyphs were successfully packed before rendering the atlas

Despite the glyph packing pass occurring separately before the font atlas is rendered, we have to wait for this process to complete before discovering not all the characters may have been successfully packed.

Particularly when rendering character ranges to large atlases (as is often necessary with Chinese etc), it's currently very painful to have to wait upwards of 5 minutes for a large atlas to render before discovering either not all characters were packed, or there is a significant amount of empty space left. Settling on an ideal point/padding size ratio takes considerably longer because of the trial and error this requires.

When logging "Glyph packing completed in X ms", could you also log the number of characters that were packed successfully (e.g. 1300/1400) so that we can cancel the rendering process early?

In a perfect world, being given an estimate of the coverage / space used - e.g. "Atlas packed: 26% empty space") - would also be invaluable, so we can increase the point size before the rendering finishes.

I left an atlas to render for 7 minutes whilst I wrote this post, only to discover that 3 characters weren't packed successfully, this would be a huge QoL improvement!

Thanks for your time!

 

I presume you are using the SDF16 or SDF32 render modes which is why this process takes that time it does. Correct?

In the event you are, I did add multi-threading back to those modes which should make the baking process faster depending on the number of cores in your system. These changes should be in the most recent versions of Unity for 2018.4, 2019.4 and 2020.x.

In terms of the functionality request, would it make sense to perhaps automatically pause the process when we detect either (1) some glyphs not getting packed in the atlas and (2) empty space exceed a % threshold?

Workflow question: Do your font assets only included the known text in the project for a given language or groups of languages or do you try to include most characters like in the case of CJK?

Are you using the dynamic font asset system and the new multi atlas texture support?

 

Yes we use SDF16. It's only for larger atlases where this becomes a problem, although even for quick render processes, I still think it's a better QoL to not have to wait any additional time if the process won't pack everything successfully, as finding a good sampling/padding ratio is a trial and error workflow, so even 30s intervals adds up.

I'm on a high spec machine, but we're using TMP 2.1.0p11 on 2019.2, so unfortunately I may not be able to take advantage of the multithreading if it's only in 19.4. (If you're wondering why we're still on 19.2, we may upgrade eventually, but when we tried to upgrade to 19.3 a couple of months ago, there were too many bugs with the prefab system to make the transition, 19.2 was far more stable).

I'm not so sure about pausing, because some users might want to kick off the process and then leave it to finish regardless of the number of characters that are packed (for instance if you're just packing ASCII and don't care if ~7 obscure characters failed to pack successfully). There is maybe more of an argument for a high empty threshold pausing the process, as 25% wasted space is still inefficient to the user that doesn't mind a few missing characters. Either way, I'm personally happy to cancel the process manually - I just want to know ahead of rendering whether I need to try different settings.

Would it be possible to display the packed arrangement of each glyph's bounding boxes on the preview window before rendering? This would provide the best (human readable) assessment of the quality of the pack.

Re workflow: Yes, we have a tool that aggregates the unique Chinese characters across all the game's strings to a file, which is then fed into TMP. The tool also writes out a Japanese character file, but omits any characters that were found in the Chinese set, to avoid duplicates. Each is then rendered to a unique atlas. We have ~1430 Chinese characters and ~620 Japanese characters. These are set as fallback font assets on our main Latin font file.

I'm afraid I'm not exactly sure what constitutes the "dynamic font asset system", and don't believe we're using multi atlasing.

Thanks for your response!

 

I released 2.1.1 last night which is simply a better version of 2.1.0.p11 so you might want to consider updating to that release.

In terms of multi threading, unfortunately, it only made it into 2018.4, 2019.4, 2020.1 and 2020.2.

Good points. I will investigate displaying the glyph bounding boxes once we have a packing solution. No ETA but I will look into it as it is indeed a good QOL improvement.

Thank you for sharing this information.

The character count is inline with what I typically see for Chinese and Japanese which is way lower than most users assume given the Chinese languages has over 72,000 characters most of which have never made it into any form of electronic publication.

The dynamic system is the ability to use dynamic font assets (as primary or fallback) where these can be populate in the Editor and at runtime. In the Editor, the changes are persistent whereas at runtime, they reset to their default state at the start of each play session. Atlas textures of dynamic font assets are initial at size 0 (empty) until the first glyph is added. This also makes them efficient for shipping and catch all at runtime.

Multi Atlas Texture allows the font asset to create new atlas textures as needed which ultimately makes it possible for a single font asset to handle all characters and glyphs contained in a font file.

See the following videos on the dynamic system, multi atlas texture support and revised suggested workflow in the early development stage which is to run full dynamic until the time comes to switch to dynamic + dynamic fallbacks.

 

Brilliant, thanks for the info!

Awesome, thanks for considering it!

Yeah it's pretty necessary to keep the count down. Just in case more figures are useful for any reason, on our title, Japanese shares 44% of characters with Chinese, so I'd always recommend to other users to account for that if they're baking separate atlases for these languages.

I'll check these out - both sound very useful!

Thanks again for your prompt responses, have a nice day

 

Hi Stephan, sorry to hijack the thread. Because I'm facing an issue seems related to font asset baking on multi-core PC. I upgraded from intel 4-cores CPU (i7-3770) to a 12 cores AMD CPU (3900x). The baking speed is slowing down significantly. Since I'm packing 10k CJK characters, it becomes impossible to finish.

I've tried 2018.4 (with 1.3.0 or 1.4.1) and 2019.4 with(1.5 and 2.0.1), all versions have the same result. The speed is NORMAL on 4-cores PC, it takes about 27s for packing and several minutes to rendering 10k characters. But on my new 12 cores PC, it tooks 10 mins for 10% of the packing process, never tried to finish the whole process. If I use 1.3.0, the progress bar stucked for 90 mins without moving. I've also tried on another AMD PC with a 8 cores 3700x, everything works fine, so I assume it is not related to CPU brand.

The font I'm using is Adobe SourceHanSerifSC-Bold.otf. And the character list is a text file (I've attached). From the image, CPU usage is very low on the 12 cores PC. It seems less than 4 cores are busy. But on my 4 cores (8-hyper threading) PC, all cores are 100% used during baking.

Could you help to check whether there is a bug in the multi-core logic for cores more than a specific number?

Thanks!

 

The change to add the multi threading back was in the FontEngine which is on the Native side which requires the latest version of Unity 2018.4, 2019.4, 2020.1 and 2020.2a.

It also would require version 1.5.x for Unity 2018.4, or version 2.1.x for Unity 2019.4 or version 3.0.1 for 2020.x.

Here is my results using 2018.4.25f1 but the change landed in 2018.4.24f1.

 

Thanks for the quick response! Since my project is using 2018.4.0.f1, the TMP@1.5.1 won't compile on that version, so I'm stucked at 1.4.1 for now. But I've just tried TMP@2.1.1 on 2019.4.0.f1, asset baking still works very slow on my 3900x. The strange part is that everything works fine on my 4-cores and 8-cores PC.

Glad to know that your 3950x could consume 100% CPU with 2018.4.25f1. I'll try with this version. Or it could be caused by something else on my PC. Thanks again for the help!

[Edit]
I've tried 2018.4.25f1, the baking speed is still slow and CPU usage is less than 10%. I'll try to investigate issues in my OS. I'll post the result if I'm lucky enough to find somthing.

[Final Edit]
I found the root cause. Somehow on my new PC the 'Packing Method' is 'Optimum', but my old environment it is 'Fast'. The optimum should be disable when character number is large. Because it is as slow as... you know. I hop I should have checked all the options more carefully.

 

I implemented the following changes to the Font Asset Creator:
- Display glyph rectangles and packing results in the Output / Feedback window as soon as packing is completed and while glyph rendering is still in progress.

 

Amazing! Thanks so much, this will save me (and I'm sure many others) dozens of hours of time Huge QoL improvement!

 

@Stephan_B Thank you very much.