Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

TextMesh Pro TMP: Breaking Chinese Bugs for Traditional ZhuYin ("BoPoMoFo" - Taiwan Locale)

Discussion in 'Unity UI (uGUI) & TextMesh Pro' started by MrLucid72, Jul 27, 2019.

  1. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    When using TextMeshPro InputFields for traditional Chinese in Taiwan (called "ZhuYin" or "BoPoMoFo" where you use a combination of phonetics and numbers [indicating tone] to make characters), there are some dominant (but random) bugs:

    Screenshot:



    Screencast:

    https://recordit.co/2e5R5meUn5

    Referencing the screenshot above:
    1. Click in TMPInputField

    2. 1st char will be in Chinese (normal)

    3. 2nd char will become English (expected: Chinese) -- indicated in the example screenshot; notice the number 5, followed by 2 Chinese characters.

    4. If you unfocus the TMPInputField (click away) and return, it'll work for until you submit once -- then steps #2 and #3 (above) will be repeated. There are no special OnValueChange() events associated with this TMPInputField. Submitting will only clear.

    5. If you continue to focus the TMPInputField and type characters, the 1st char will always be an English character until you unfocus >> refocus (then the bug returns after the 1st line).
    • Anytime you see an English char or number @ char[0], it's bugging out.

    • Both the TMPInputField and the submitted txt contain the buggy str (which is why it also reflects in the chat, which is also a read-only TMPInputField)
     
  2. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    Thanks for the feedback / report.

    There are a few issues related to IME that have been present for a while now. These are not specific to TMP or even the legacy text system but to IME handling which is handled by the Input team.

    I will be talking to the Input team shortly and will be sure to bring up these issues as well.

    I'll post back once I have additional information.
     
    MrLucid72 likes this.
  3. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    Ah, thanks for looking into this!
     
  4. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    Hi @Stephan_B , any chance you have a timeline estimate on this? I'm approaching an awkward point where I must soon decide if I should try to find a ghetto workaround or if I should wait for a fix. We're currently in alpha for Chinese testing. Let me know if you require additional details.

    Apologies for the pressure! Just trying to plan :)

    PS - have any temporary workaround suggestions that would be easy to revert later upon a fix?
     
    Last edited: Aug 10, 2019
  5. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    I will take a look over the weekend and provide feedback no later than Sunday night.
     
    MrLucid72 likes this.
  6. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    Can you check if you are getting the same behavior in any Editor text field / inspector? Also please see if you get the same behavior in the UI Input Field.

    What character sequence / keyboard keys are you typing to reproduce these behavior?

    Whenever the input field is un-focused, IME is disabled. However, whenever you re-focus the input field, IME is not re-enabled. Right now I have to use CTRL + Spacebar to re-enable IME.

    There are definite issue with the IME input system.
     
    MrLucid72 likes this.
  7. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    I'll be able to test in a couple days and get back to you~
     
  8. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    @Stephan_B Ok, I got my wife to help out (my Chinese is limited, especially Bopomofo [Taiwan, Traditional Chinese] IME):



    Code (CSharp):
    1. ㄏㄏ>> c space c space
    2. >>xul6
    3. >>ru0 space
    4. >>g4
    5. >>5j/ space
    6. >>jp6
    7. I tried to type "中文" again but it  change to english automatically, so it showed "5" at the next line
    8. >>j/ space
    9. trying to type "中" again, and it worked
    10. trying to type "中" again, it falled and show as "5翁"
    11. (when I type 中, it stared with keyborad "j". )
    12. trying to type "文(jp6)" but it show as "j恩"
    13. how to type 恩>> "p space"
    14. If you need more info or how you want as to type, let me know. Thanks
    15.  
    I'm too deep in a messy project to test anything else, yet, but hope this helps.
     
  9. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    I'll re-test these.

    Were you able to test this in a normal text field in the Editor in some inspector?
     
  10. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    Seems fine in the inspector, so far. By normal, you mean a TMPro UGUI, opposed to inputfield, right?

    To add, I'm also noticing some weird stuff relating to copy+pasting of traditional Chinese characters (possibly any unicode). If I paste some characters inside an editor text field, it'll show as something random. However, if I ctrl+Z, the entire txt is wiped EXCEPT for what I was originally trying to paste!
     
  11. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    To update, what may or may not be related, there's another at least semi-related issue:

    Within the editor, I often cannot copy+paste Unicode values to each other (specifically, to a TMPro UGUI) -- without doing something hacky. Sometimes it works, sometimes it doesn't. However, right now (even after a reboot), it won't:
    • If I copy unicode from outside of Unity, it works fine.
    • If I copy unicode from within Unity to within Unity (eg, from a ScriptableObj to a UGUI text field), this is what happens:

    • ^ What's happening above is me trying to copy+paste the Chinese version of "Sheriff" ( 警長 ) from a ScriptableObject to a TMPro UGUI Text field: When I paste the value, the unicode is replaced with "fw". You can see me copy+paste "asdf" no problem (ASCII English).
    • If I copy FROM Unity to OUTSIDE Unity (eg, notepad), the result is also "fw".
    • If I try to copy a different Unicode from within Unity to Unity, such as "Marshal" ( 刑事官 ), nothing is pasted at all when: In fact, right clicking won't even show a "paste":
    It seems that issues with Unicode extend to the Editor -- but it's not entirely clear what's causing it (I don't have enough time to check patterns further, unfortunately)

    EDIT: To rule out my ScriptableObject having issues, I'll repro this breaking FROM tmpro ugui to tmpro ugui:

    1. copy+paste FROM Notepad TO TMPro UGUI.
    2. Copy from TMPro UGUI to TMPro UGUI. It will not work.



    HOWEVER, notice at the end i got it to show -- sort of: If I CTRL+Z, the field wipes and is replaced with what was *meant* to be pasted. However, this means I cannot paste within the middle and CTRL+Z or any accompanied text will wipe EXCEPT what was supposed to paste.

    So pasting Unicode from plaintext Notepad to a Unity TMPro UGUI Text field will permanently do something very strange to the text. I can no longer copy+paste it anywhere:

    I cannot paste Unicode vals from within a TMProUGUI Text field anywhere within Unity -- and even externally. Pasting here or back to notepad results in an empty paste.
     
    Last edited: Aug 30, 2019
  12. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    Thank you for providing all this great feedback.

    I am slowly going through all of these. Some of these issues have been in Unity since Unity 4.x so tracking those with knowledge of those implementation or getting up to speed on my end on those IME related implementations is slow.

    Please keep on posting in this thread as you uncover additional issues.
     
    MrLucid72 likes this.
  13. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    When I try to paste from Unity to Unity:

    `儲存至桌面為`

    The following is, instead, displayed:

    `2X`

    If I paste here, here's what's displayed:

    `2X` (Notice the +secret tofu box)

    EDIT: Tofu isn't showing. Here's what I see:

     
  14. Stephan_B

    Stephan_B

    Unity Technologies

    Joined:
    Feb 26, 2017
    Posts:
    3,304
    By 2X, you mean duplication of the composition sequence? If that is the case, I did resolve that issue which has been in Unity since at least 5.x.

    P.S. I am focusing on the TMP Input Field issues first and not necessarily Editor / Inspector issues. However, in the case of the duplication issue, I did fix that one in Unity so it is also fixed in the Editor. Editor / Inspector specific issues, I will be passing to the Editor team.
     
  15. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    Oops, no, just a literal string. I wonder if it would partially match a /unicode sequence. I'm not good with that, though, so no idea.
     
    Last edited: Sep 2, 2019
  16. MrLucid72

    MrLucid72

    Joined:
    Jan 12, 2016
    Posts:
    664
    Just wanted to do a quick follow-up: Any luck?