Foreign characters not showing up

Basically, we're having the same problem with this tread
(except that we are using unicode Thai font.) ->

http://forum.unity3d.com/viewtopic.php?t=9138

A lot of foreign letters are not showing up in GUIlabel and etc.
So I thought I should start a new tread so it get noticed by Unity Headquarter.
Hopefully, this would get fixed in 2.1?

 

foreign or chinese? this is actually a massive difference as european letters fall within the regular ASCII range and given you have a real Arial that you use, not the US one, there is little reason not work.

 

Please double-check that you're saving the script in UTF-8 or UTF-16 format. In Unitron you can do this from the menubar->Text->Text Encoding->Unicode. If you change the format and save the script and afterward the font is still not working, then please use the Bug Reporter to submit your problem and attach your project folder. I'll take a look to see what the problem is.

 

I just did a test, and it seems that Unity's C# compiler (at least) won't automatically treat a source file as UTF-8 unless it begins with a BOM, and Unitron doesn't write a BOM when it saves files in the UTF-8 encoding. Nor does TextMate, for that matter. It's therefore best to save as UTF-16 if you want full Unicode support.

It might be help if Unity explicitly used -codepage:utf8 when calling the mcs compiler. Without the -codepage switch, the compiler defaults to whatever encoding it feels is native for the platform it's running on (I think it likes MacRoman on OSX... for some reason).

I imagine the Javascript compiler might behave differently, but I haven't tested it.

Would you like me to file a bug report on this, Sam?

 

I know about Bom. Sadly I'm having a more specific problem with the chat portion though (i.e. reading user's string input.)

On Mac, a lot of characters are not showing up when user type in Thai characters. And when using PC, every character is showing as an alien language.

 

Any words from Unity's HQ?

Do I have to submit a bug report? :?

 

Yes please!

That would be best. If you have a project that doesn't work as you would expect it to, please attach it in the bug report and let me know what it actually does vs what it should be doing.

 

Are you saying that the problem is when you save Thai characters in your Unity scripts or when you try to input them at runtime using the Unity GUI?

Saving text in the scripts should be simple, at least for CJK I just use UTF8 and everything works fine. (Neil - I haven't experienced the problem re BOM in UTF8 vs UTF16, but I generally use Visual Studio which saves the signature by default). In regards to inputting non-western characters, if they require multiple strokes or any kind of OS IME support, you're out of luck - Unity doesn't support this yet.

If the problem is that your characters don't display correctly, it means Unity can't find the character in your font texture - try using ARIALUNI as a last resort - it contains pretty much all east asian languages, but requires a lot of memory.

The only other obscure reason I could think of is also related to the IME - if previous characters are modified by new characters you input (as in Chinese phrasal pinyin, and possibly Thai) then this wouldn't work.

 

My problem is user's key input during run time. Almost all the characters just get lost when typed (nothing appear as if nothing is typed) Got no clue what's happening but some of the letter works some not. (Fonts works fine otherwise)

Making a network game that player can't chat with each other in their native tongue is a killer issue for us. Hope Unity HQ can provide some answer or remedy. I'm not sure if submitting a bug report with sample project file would be any help here since Unity won't have Thai keyboard to try it out there anyway. :?

 

How many times do you have to tap a key to form a character in Thai, and can the newly typed characters affect the previously typed ones? If yes, then I imagine the problem is with IME related. If not then it's probably the font you're using doesn't contain a complete set of Thai.

EDIT: I just did a lookup of Thai IME and it seems to be cell based - which is probably what's causing the problem, since you compose characters, rather than just input one at a time.

 

I doubt that composing characters is the root of this problem though, since all characters compose fine when they're saved with BOM. My problem only happens when user type in characters in game. Some of the characters disappear even without any need to be composed. But I could be wrong :?

 

Maybe I didn't explain very clearly. You definitely should have no problem saving and displaying unicode characters that come from a script save in UTF-8 with the BOM signature.
What I'm talking about comes only when you try to input multistoke or cell-based character into a Unity app at runtime. Unity doesn't have an IME to interpret these kind of characters and therefore could result in strange behavior. Maybe you could post a screen to clarify the problem, but I'm pretty sure it's IME related.

 

Either way, it's not something I can fix with java. Still no words from Unity HQ? :?