Search Unity

Resolved "TouchScreenKeyboard.hideInput" triggers numerous errors & breaks functions in current Android/Unity

Discussion in 'UI Toolkit' started by mikejm_, Jul 3, 2022.

  1. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    Being able to hide the TouchScreenKeyboard's input field is essential in Android for creating a seamless or basically good experience with text input/messaging/etc. This function to do this is given by "TouchScreenKeyboard.hideInput = true". Documentation suggests:

    This would be absolutely fine. There is nothing wrong with simply providing the illusion of the input field being gone. However, this not currently working in present Android and it is in fact triggering numerous errors which break the basic Keyboard functions.

    1) Errors on Opening the Keyboard
    The first problem is that when you open the keyboard (click a text field), it responds with 7 error messages:

    Code (csharp):
    1.  14211 14211 Error Company.TextBo No package ID 3f found for ID 0x3f050003.
    2.  14211 14211 Error Company.TextBo No package ID 3f found for ID 0x3f050001.
    3.  14211 14211 Error Company.TextBo No package ID 3f found for ID 0x3f050003.
    4.  14211 14211 Error Company.TextBo No package ID 3f found for ID 0x3f050002.
    5.  14211 14211 Error Company.TextBo No package ID 3f found for ID 0x3f050003.
    6.  14211 14301 Error Company.TextBo No package ID 3f found for ID 0x3f050001.
    7.  14211 14301 Error Company.TextBo No package ID 3f found for ID 0x3f050003.
    I am not sure if this one is significant in any way though as these same seven errors are also triggered when TouchScreenKeyboard.hideInput = false. But if it matters, it may be worth mentioning.

    2) Auto-Correct Underlines are Still Visible
    The second problem is that when you start typing, it will show the underline still from the auto-correct suggestions, thus indicating there is still SOMETHING there in the area that the input field would be (it is not completely being hidden).

    Here you can see a horizontal blue underline in a simple bug demo project with "hideInput = true" I made to illustrate which would presumably be under the word "wha" as part of the autocorrect suggestions:

    Screenshot_20220702-231332_TextBox.jpg


    3) Clicking in the Area of Hidden Keyboard Input Field Crashes the Keyboard
    The third and most destructive problem is that if you have the keyboard open with "hideInput = true" and click anywhere within the area that the input field would usually be it closes the keyboard with a cascade of errors/warnings:

    Code (csharp):
    1.  14211 14211 Error ViewRootImpl Calling showInsets(8,true) on window that no longer has views.
    2.  14211 14211 Warn IInputConnectionWrapper getTextBeforeCursor on inactive InputConnection
    3.  14211 14211 Warn IInputConnectionWrapper getTextAfterCursor on inactive InputConnection
    4.  14211 14211 Warn IInputConnectionWrapper getTextBeforeCursor on inactive InputConnection
    5.  14211 14211 Warn IInputConnectionWrapper getTextAfterCursor on inactive InputConnection
    6.  14211 14211 Warn IInputConnectionWrapper requestCursorAnchorInfo on inactive InputConnection
    7.  14211 14211 Error ViewRootImpl sendUserActionEvent() mView returned.
    For example, in this screencap with keyboard input hidden, clicking anywhere in the red box approximately corresponding to the hidden input field will trigger this crash:

    Screenshot_20220702-231332_TextBox red line2.jpg


    Here is the same demo app with the input field NOT hidden - this shows the approximate range of the input field which corresponds to the area that clicking on crashes everything when the input field is hidden:

    Screenshot_20220702-232901_TextBox.jpg

    Presumably this is occurring for the same reason as issue #2 - the keyboard's input field is still there in some sense and capturing clicks in this region. This is then triggering abnormal behavior which is breaking the normal expected functions.

    Help Wanted
    I am wondering if this can be fixed for current Android versions. Alternatively, perhaps you can please provide the code that was used to hide the input field, and we can update it ourselves and write our own code from it that will still work.

    I have been able to write Android code in Unity that tracks keyboard height and hides/shows status bars and nav bars, but I am not sure where to start for the input field. ie. What commands control this or what to manipulate.

    If you are unable to fix this or don't have resources to do so in a timely manner, can you please share the code behind TouchScreenKeyboard.hideInput or explain what commands it was using so those of us who need it to work can at least know where to start?

    This is absolutely necessary for my app so I appreciate any help you can provide. I submitted a bug report for it (IN-9117) with the screencapped simple demo project using UI Toolkit that shows these errors/bugs.

    Thanks.
     
    Last edited: Jul 3, 2022
  2. FreddyC-Unity

    FreddyC-Unity

    Unity Technologies

    Joined:
    Jun 9, 2021
    Posts:
    36
    Hi @mikejm_,

    Thank you for logging this as a bug! It is a known issue that we are currently addressing and we are aware that it is one that have been generating a lot of pain for our users. Unfortunately, there's some limitations for this use case - at this moment - and I will try to explain it as detailed as possible. Also it is worth mentioning that part of the limitations comes from the OS's API, therefore a lot of what we want to implement is being restricted for us.

    As I am trying to follow up with your scenario, the main reasons as to why it's not behaving the way we expect it to (point 2 and 3) and the reason it's throwing the errors (point 1), is because somewhere behind the scene, the visible touch keyboard still holds the previous value/state. To further elaborate, there are some properties that when it's being changed after the keyboard has been instantiated, it won't take full effect unless it's re-instantiated. After doing some quick testing, it will behave in an unwanted manner when changing the keyboard type, hide input, auto-complete, and at times just the text value. You will experience these unplanned bugs when you have more than one input field and you try assigning different behaviors to each of them.

    While there's no immediate solution we can provide for you now, I can share that there's a set of improvements that will be made available in a very near future release. For your scenario, we will automatically address this issue and in addition, you will be able to enable and disable the OS input through the builder or text field (will be exposing the current valid keyboard for customization - this way when opening it, it will always hold your desired value and give you the desired outcome). The changes should make it less dependent of the OS input and give you some more freedom.
     
    HugoBD-Unity likes this.
  3. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    Thanks FreddyC. I appreciate you guys are aware of this and trying to find a solution. That gives me a lot of peace of mind.

    I would just like to clarify a bit about my usage and that this will be a global solution then that won't require any specific new steps on our end.

    For my part, I am not using the built in "TextField" in UIElements or any the old UI system. (Although these premade elements trigger the same errors as my custom made ones, which is why I realized it was a bigger background issue.)

    I have built my own custom TextFields which in the simplest manner, upon clicking in a mobile environment, open the touch screen keyboard with code like:

    Code (csharp):
    1. TouchScreenKeyboard.hideInput = true;
    2. TouchScreenKeyboard keyboard = TouchScreenKeyboard.Open(editString, TouchScreenKeyboardType.Default, true, true, false, false, "", 1000);
    Then I can read from keyboard.text on a frame by frame basis to update my custom TextField based on the keyboard input.

    It is very critical I keep using my custom TextFields as they have numerous features that integrate with other design elements and click management which I have built and I can't use the default ones at this point.

    So I just want to be clear that the solution you are providing will be global and not tied to any particular "TextField" type object, but rather will just mean that if we use hideInput = true without any other specific change in our methods, it will then avoid these crashes. ie. It will just work.

    I presume some may appreciate the option to disable the system OS touch screen keyboard but I actually rather like using the built in ones as then I don't have to try to create 50+ keyboards for every language (which Android/iOS already provide users for free). That would be a nightmare! I am happy to let Google/Apple/Samsung deal with that ...

    So I actually like the current integration with the OS keyboard. I would just like it not to obviously be displaying abnormal underlines or crashing when you click in the zone the hideInput is supposed to be clearing.

    Thanks again for your hard work. I will just keeping working on other parts of my app in the mean time. It is a bit stressful as I am making a mobile app that depends on touch screen keyboards and SMS type communication so having this working is absolutely critical for me. But I appreciate that you are working on a solution and I am looking forward to it.

    Thanks.
     
  4. junwchina

    junwchina

    Joined:
    Jul 30, 2014
    Posts:
    1
  5. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    They marked my bug report as fixed internally and pending for release a week or two ago so fingers crossed.
     
    ChGuidi likes this.
  6. arenart

    arenart

    Joined:
    Nov 1, 2017
    Posts:
    3
    Hi @mikejm_
    is it resolved? If so, how?
     
  7. co43687

    co43687

    Joined:
    Mar 11, 2021
    Posts:
    3
    please fix it asap. i get same problem as well. its critical.
     
    mikejm_ likes this.
  8. gisinator

    gisinator

    Joined:
    Jan 23, 2021
    Posts:
    11
    mikejm_ likes this.
  9. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    The bug was marked as "solved" in December. I cautiously waited for the "solved" versions of Unity to be released. I finally worked up the courage to go test it, and unfortunately, no it is not in any way solved at all.

    This is now 9 months after I reported and Freddy said at that point Unity was already aware of the problem. So this problem is just sitting and no one is fixing it. I and many others absolutely need a working onscreen keyboard where we can control the way the input is shown on screen.

    I repeat my request: If Unity is not willing to fix this, can you please share the code you are using for your partial/broken solution publicly so the community can try to work from what you started with and fix it instead? If I am unable to solve it with this information, I would even pay a considerable amount to a 3rd party Android developer to help find a solution as I am dead in the water without this being fixed for the project I have been working on the past 2 years.

    I will not be able to release without a working on screen keyboard for the largest mobile operating system on earth.

    Current Problems (Unity 2022.2.9f1 - Released: March 3, 2023)
    I reopened the test project I submitted to Unity in this recent version as well as my real projects and found bizarre new behavior with this "fix". Testing using Samsung S21 Android phone.

    If this is actually "fixed" in a different version of Unity, please let me know which version I should be using as that would certainly be a nice quick solution. However, I presume this version has the "fix" as it is now behaving completely different than before. Just not correctly still.

    1) TOUCHSCREENKEYBOARD.HIDEINPUT RESETS ITSELF TO FALSE WHEN KEYBOARD IS OPENED
    Bizarrely, I noticed immediately I am no longer able to hide the input field at all. When I debug out, I can see that although my code clearly sets TouchScreenKeyboard.hideInput = true, something is constantly resetting this to false. I have my own internal bool I set in the test project above and it is not changing, but Unity keeps changing the assignment for the global variable beyond my control as soon as you activate the Keyboard.

    Here I have in my Update() loop of the original demo project:

    Code (csharp):
    1. Debug.Log("UPDATE" + " | HIDE INPUT STATUS " + TouchScreenKeyboard.hideInput + " | HIDE INPUT BOOL " + hideInputField);
    You can see for endless iterations (two shown here) it debugs out that both of these are "true" as I have set them to true on Awake() and none of my code will change this. Yet, as soon as I click the TextField at the bottom to open the keyboard, this then changes to false for no clear reason:

    keyboard new - click turns hide input false2.png
    This should remain "true" for both as I have never told Unity to set it to false (I only ever told it to set it to true).

    Thus this test project I submitted no longer even hides input at all. This is now what it looks like even after I set hideInput to true (it is no longer hidden, whereas before in the OP it would at least try to hide):

    keyboard new - no hide input.jpg

    I suspect this is due to something in TextField resetting it, as in my real project where I am not using TextField but rather a custom made text field class, this does not happen.

    2) ON PROJECTS WHERE HIDEINPUT REMAINS TRUE, THE SAME VISUAL ISSUES AND ERRORS REMAIN
    As noted, on my main real project, hideInput does remain true in the same version of Unity. I believe this is because I am using a custom text field class I made rather than a standard TextField (which seems to now reset hideInput to false on clicking). Either way, at least this lets me see the other behaviors. Sadly, the problems reported seem to have changed a bit in behavior, but remain unresolved and this is still unusable.

    One could presumably test this without TextField just by running TouchScreenKeyboard.Open as this is how I open keyboards in my custom text field.

    (i) Previously Reported: Blocked Clicks
    If I click in the region of the "hidden input", my clicks are blocked from going through to Unity objects underneath but not always. Sometimes the 2nd or 3rd or 4th click goes through while the others are blocked. This is somewhat improved as previously clicking in this area would always crash the keyboard and at least now it does not always do so.

    No errors are debugged out when these clicks don't go through, so until I debug it further, I can't say for sure if this is my problem or yours. Although the next issue suggests it probably still is. My guess is this issue would be fixed if everything else is fixed too.

    Additionally, clicking in the "hidden zone" repeatedly can sometimes trigger a total application crash. I haven't yet caught that on debugger to see why it's happening. But something is likely still wrong here too.

    (ii) Previously Reported: Can Still See Autocorrect Lines
    For example, here I have typed 'hnv' in a textfield in a scroll window above offscreen and the keyboard is trying to autocorrect me to "have" which is fine in the grey region where it should be, except it is still showing the blue autocorrect underlines for the imaginary suggestions in the "hidden" zone as previously reported was still an issue 9 months ago. This again shows there is still something there and it is still broken.

    I am guessing this is related to the click blocking behavior above. If something is still there (which it clearly visibly is), it could still be blocking clicks too.

    keyboard new - autocorrect.jpg

    3) NEW OTHER ERRORS
    When the TouchScreenKeyboard is open and Unity has set "hideInput" to false on its own accord (as in the original demo project for some reason now), there are still errors - they are just new ones I don't recall from before. For example, if in the original test project you open the keyboard and then click any random place else on the screen, the keyboard collapses and it gives:

    keyboard new - error tap screen while open.PNG

    CONCLUSION
    A couple final points:
    I know that the challenge you voice is that you must interface with Android behind the scenes to move these unwanted objects off the screen. But compared to all the other things Unity has accomplished, I can hardly see this task as being so insurmountable as to cripple your entire organization and high quality coders. And if it is so challenging and insurmountable to you (which I don't believe), please give us the code and let us work on it instead.

    Either way, can we please, please, please have a working touchscreen keyboard with hideInput that allows us to design our apps appropriately in the way users expect?

    BUG REPORT

    The same test project from IN-9117 remains relevant. The only problem is now, as noted, this project is no longer sustaining hideInput = true and Unity is magically changing this to false for some reason. So you will not see the rest of the problems until you can fix that first. I cannot find any explanation for why hideInput remains true in my real project but no longer in this demo project.

    I can submit it again with a new bug report but I am wary of doing so. It would be the same test project. It also takes about 6-9 months for a bug report to be "resolved," and this "resolution" was anything but a resolution. I do not want to be stuck waiting another 6 months just to have someone look at it again for something we all already know is a problem and remains a problem.

    Thanks for any help.
     
    Last edited: Aug 10, 2023
    Unifikation and gisinator like this.
  10. huwp

    huwp

    Joined:
    May 22, 2013
    Posts:
    33
    Regarding the crash after leaving for a few minutes:
    Code (CSharp):
    1. AndroidJavaObject View = UnityClass.GetStatic<AndroidJavaObject>("currentActivity").Get<AndroidJavaObject>("mUnityPlayer").Call<AndroidJavaObject>("getView");
    2.  
    There's no using statement so Dispose() isn't being called for this object. Also, these objects are being recreated every single frame so it's *hammering* the app's memory. They should be initialized in OnEnable() and disposed in OnDisable().
     
    mikejm_ likes this.
  11. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    Thanks! I know that was not responsible code, but I did not realize Garbage Collector would not solve this at least crudely. Are those things not caught by Garbage Collector?

    I updated the code in a more responsible way (and my post above) and those memory overloads seem to have stopped.

    Memory usage still creeps slowly over time but I presume this may be due to Unity's known issue with memory fragmentation and non-compacting garbage collection (ie. other things beyond my control).

    One less problem. :)
     
  12. huwp

    huwp

    Joined:
    May 22, 2013
    Posts:
    33
    Well they are, but creating and destroying those objects are actually invoking TWO garbage collectors, Unity's and Android's java runtime. Trying to wrangle both to keep references to each-other is uh, a delicate task to say the least. : P

    I needed to learn about TextField sooner or later, so here's my attempt:
    • Tested on Unity 2021.3 LTS Android.
    • hideInput seems to stay hidden.
    • Made the layout more flex-able.
    • Used RuntimePanelUtils.ScreenToPanel() to scale the keyboard height properly.
    • Unfortunately it looks like the TextField doesn't recognize when the soft keyboard's submit button is pressed so that's a different bug?
    All three items should be attached to the same GameObject:

    HideInputTest.uxml
    Code (csharp):
    1. <ui:UXML xmlns:ui="UnityEngine.UIElements" xmlns:uie="UnityEditor.UIElements" xsi="http://www.w3.org/2001/XMLSchema-instance" engine="UnityEngine.UIElements" editor="UnityEditor.UIElements" noNamespaceSchemaLocation="../../../UIElementsSchema/UIElements.xsd" editor-extension-mode="False">
    2.     <ui:VisualElement picking-mode="Ignore" style="background-color: #333; flex-grow: 1;">
    3.         <ui:Label name="lbl_output" picking-mode="Ignore" style="flex-grow: 1; margin: 8px; padding: 4px; border-width: 8px; border-radius: 8px; background-color: #777; color: black; -unity-text-align: top-left;" />
    4.     </ui:VisualElement>
    5.     <ui:VisualElement picking-mode="Ignore" style="flex-direction: row;">
    6.         <ui:TextField name="txt_input" style="flex-grow: 1; margin: 4px; padding: 0;" />
    7.         <ButtonVE name="bve_submit" style="margin: 4px; padding: 8px; border-radius: 4px; background-color: #777; justify-content: center;">
    8.             <ui:Label picking-mode="Ignore" text="Submit" style="color: black; -unity-text-align: middle-center;" />
    9.         </ButtonVE>
    10.     </ui:VisualElement>
    11. </ui:UXML>
    HideInputTest.cs
    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3. using UnityEngine.UIElements;
    4.  
    5. public sealed class HideInputTest : MonoBehaviour
    6. {
    7.     #region Components
    8.  
    9.     UIDocument _ui;
    10.     SoftKeyboardHelper _kbh;
    11.  
    12.     private void GetComponents()
    13.     {
    14.         _ui = GetComponent<UIDocument>();
    15.         _kbh = GetComponent<SoftKeyboardHelper>();
    16.     }
    17.  
    18.     #endregion
    19.  
    20.     #region UI
    21.  
    22.     Label _lbl_output;
    23.     TextField _txt_input;
    24.     ButtonVE _bve_submit;
    25.  
    26.     private void GetUIElements()
    27.     {
    28.         var root = _ui.rootVisualElement;
    29.         _lbl_output = root.Q<Label>("lbl_output");
    30.         _txt_input = root.Q<TextField>("txt_input");
    31.         _bve_submit = root.Q<ButtonVE>("bve_submit");
    32.     }
    33.  
    34.     #endregion
    35.  
    36.     private void Awake()
    37.     {
    38.         GetComponents();
    39.         GetUIElements();
    40.     }
    41.  
    42.     private void Start()
    43.     {
    44.         _kbh.KeyboardHeightChanged += Kbh_KeyboardHeightChanged;
    45.  
    46.         _bve_submit.OnClick += SubmitText;
    47.         _txt_input.RegisterCallback<NavigationSubmitEvent>(_ => SubmitText());
    48.  
    49.         SubmitText("Hello");
    50.         _txt_input.Focus();
    51.     }
    52.  
    53.     private void Kbh_KeyboardHeightChanged(int height)
    54.     {
    55.         _ui.rootVisualElement.style.marginBottom =
    56.             RuntimePanelUtils.ScreenToPanel(_ui.rootVisualElement.panel, new Vector2(0, height)).y;
    57.     }
    58.  
    59.     private void SubmitText(string txt)
    60.     {
    61.         _lbl_output.text += txt + "\n";
    62.     }
    63.  
    64.     private void SubmitText()
    65.     {
    66.         SubmitText(_txt_input.value);
    67.         _txt_input.value = "";
    68.         _txt_input.Focus();
    69.     }
    70. }
    71.  
    72. public class ButtonVE : VisualElement
    73. {
    74.     public new class UxmlFactory : UxmlFactory<ButtonVE> { }
    75.  
    76.     public event Action OnClick;
    77.  
    78.     public ButtonVE()
    79.     {
    80.         RegisterCallback<NavigationSubmitEvent>(_ => OnClick?.Invoke());
    81.         RegisterCallback<PointerDownEvent>(e => this.CapturePointer(e.pointerId));
    82.         RegisterCallback<PointerUpEvent>(e =>
    83.         {
    84.             if (this.HasPointerCapture(e.pointerId))
    85.             {
    86.                 this.ReleasePointer(e.pointerId);
    87.                 OnClick?.Invoke();
    88.             }
    89.         });
    90.         focusable = true;
    91.     }
    92. }
    SoftKeyboardHelper.cs
    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3.  
    4. public sealed class SoftKeyboardHelper : MonoBehaviour
    5. {
    6.     public event Action<int> KeyboardHeightChanged;
    7.  
    8.     private void Start()
    9.     {
    10.         TouchScreenKeyboard.hideInput = true;
    11.     }
    12.  
    13.     int _kbHeight = 0;
    14.  
    15.     private void Update()
    16.     {
    17.         var h = GetHeight();
    18.         if (_kbHeight != h)
    19.         {
    20.             KeyboardHeightChanged?.Invoke(h);
    21.             _kbHeight = h;
    22.         }
    23.     }
    24.  
    25. #if UNITY_ANDROID && !UNITY_EDITOR
    26.     private AndroidJavaClass _unity;
    27.     private AndroidJavaObject _view;
    28.     private AndroidJavaObject _rect;
    29.  
    30.     private int GetHeight()
    31.     {
    32.         if (!TouchScreenKeyboard.visible) return 0;
    33.  
    34.         _view.Call("getWindowVisibleDisplayFrame", _rect);
    35.         return Screen.height - _rect.Call<int>("height");
    36.     }
    37.  
    38.     private void OnEnable()
    39.     {
    40.         _unity = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
    41.         _view = _unity
    42.             .GetStatic<AndroidJavaObject>("currentActivity")
    43.             .Get<AndroidJavaObject>("mUnityPlayer")
    44.             .Call<AndroidJavaObject>("getView");
    45.         _rect = new AndroidJavaObject("android.graphics.Rect");
    46.     }
    47.  
    48.     private void OnDisable()
    49.     {
    50.         _rect?.Dispose();
    51.         _view?.Dispose();
    52.         _unity?.Dispose();
    53.     }
    54. #else
    55.     private int GetHeight()
    56.     {
    57.         return TouchScreenKeyboard.visible
    58.             ? (int)TouchScreenKeyboard.area.height : 0;
    59.     }
    60. #endif
    61. }
     
    Last edited: Apr 12, 2023
    Blacklister001 and gisinator like this.
  13. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    FYI huwp and just for reference, that approach still has the bug where TouchScreenKeyboard.hideInput gets set to "false" on clicking the TextField if you build to an Android device (the new glitch added in the last "fix"). I just tested it as I was trying to create some bug reports.

    I was told by @HugoBD-Unity to submit new bug reports for all remaining problems to have them reviewed next. So here they are.

    1) IN-41019: "TouchScreenKeyboard.hideInput" is set to "false" by clicking a TextField
    I created a new bug report for the phenomenon of TouchScreenKeyboard.hideInput being auto-set to "false" now on TextField clicks in Android. This bug was created by the last "fix" as described here. It was not happening previously.

    This is personally minor for me as I don't even use TextFields (I use my own class). But I mention it in the interest of being comprehensive.

    2) IN-41023: Numerous Problems with TouchScreenKeyboard Functions in Android

    This bug report avoids the problem from the first bug by instead using TouchScreenKeyboard.Open() to directly open the keyboard. The "text field" is now just a dummy element that does nothing. The only operational control is the Open/Close (previously Submit) button at the bottom right which opens or closes the keyboard.

    (i) Setting "TouchScreenKeyboard.active" to false triggers an error in Android
    If you click the "Open/Close " button at the bottom right, this will open or close a new keyboard with:

    Code (csharp):
    1. if (keyboard == null) {
    2.                 keyboard = TouchScreenKeyboard.Open("");
    3.                 Debug.Log("KEYBOARD OPENED");
    4.             }
    5.             else {
    6.                 keyboard.active = false;
    7.                 keyboard = null;
    8.                 Debug.Log("KEYBOARD CLOSED");
    9.  
    10.             }
    However, closing the keyboard creates the following error sequence in Android Logcat:

    Code (csharp):
    1. 2023/05/13 23:51:20.340 13497 13497 Info ViewRootImpl@1abbbb3[UnityPlayerActivity] MSG_WINDOW_FOCUS_CHANGED 1 1
    2. 2023/05/13 23:51:20.340 13497 13497 Info Unity windowFocusChanged: true
    3. 2023/05/13 23:51:20.341 13497 13497 Debug InputMethodManager startInputInner - Id : 0
    4. 2023/05/13 23:51:20.341 13497 13497 Info InputMethodManager startInputInner - mService.startInputOrWindowGainedFocus
    5. 2023/05/13 23:51:20.357 13497 13497 Warn IInputConnectionWrapper requestCursorAnchorInfo on inactive InputConnection
    6. 2023/05/13 23:51:20.395 13497 13497 Error ViewRootImpl sendUserActionEvent() mView returned.
    Thus there is no error free way to close the TouchScreenKeyboard.

    (ii) Auto-correct suggestions still show up over the "hidden zone"
    In this project the "inputField" remains hidden, but you can still see autocorrect prompts pop up in this area if you type a sequence that will trigger them. This suggests the Android objects that must be hidden (eg. moved off screen) have not been adequately hidden.

    For example, on my Samsung S21, if I type "its" I get a little line under where it is trying to suggest me capital "it's" instead:

    Screenshot_20230513-235625_TextBox its autocorrect.jpg

    This obviously should not be happening. If you are testing this, I am not sure of any certain triggers for the autocorrect to show the lines. It seems to have some inconsistency over time due to its own logic I presume, but "its" and "i" seem to trigger it pretty consistently.

    (iii) Clicks in the hidden zone remain blocked
    This is the biggest issue and likely directly related to (ii). When the TouchScreenKeyboard is open, any clicks in the region where you would see for example the autocorrect lines of (ii) are blocked from going through to UI Toolkit.

    On any click release, I have the following Debug:

    Code (csharp):
    1. VisualElement clickedVE = rootVE.panel.Pick(UIClickPosScaled);
    2.             if (clickedVE == openButtonVE) {
    3.                 Debug.Log("RELEASED ACTIVE OPEN BUTTON" + " random: " + UnityEngine.Random.Range(10000, 99999));
    4.                 openButtonVE.runButtonClick();
    5.                 textFieldOpen = false;
    6.             }
    7.             else if (rootVE.panel.Pick(UIClickPosScaled) == textFieldButtonFieldVE) {
    8.                 Debug.Log("RELEASED TEXT FIELD REGION CLICK " + " random: " + UnityEngine.Random.Range(10000, 99999));
    9.             }
    10.             else {
    11.                 if (clickedVE != null) {
    12.                     Debug.Log("RELEASED OTHER CLICK REGION " + clickedVE.name + " random: " + UnityEngine.Random.Range(10000, 99999));
    13.                 }
    14.                 else {
    15.                     Debug.Log("RELEASED NULL REGION " + " random: " + UnityEngine.Random.Range(10000, 99999));
    16.                 }
    17.             }
    So all clicks should be debugged out with some region and VE naming Debug line. However, once the keyboard is open, this no longer works in the hidden zone.

    A click in the typically hidden zone when the keyboard is closed looks like this:

    Code (csharp):
    1. 2023/05/14 00:03:32.939 16191 16191 Info ViewRootImpl@1abbbb3[UnityPlayerActivity] ViewPostIme pointer 0
    2. 2023/05/14 00:03:33.027 16191 16191 Info ViewRootImpl@1abbbb3[UnityPlayerActivity] ViewPostIme pointer 1
    3. 2023/05/14 00:03:33.112 16191 16219 Info Unity RELEASED TEXT FIELD REGION CLICK  random: 64264
    4. 2023/05/14 00:03:33.112 16191 16219 Info Unity UnityEngine.StackTraceUtility:ExtractStackTrace () (at /home/bokken/build/output/unity/unity/Runtime/Export/Scripting/StackTrace.cs:37)
    5. 2023/05/14 00:03:33.112 16191 16219 Info Unity UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
    6. 2023/05/14 00:03:33.112 16191 16219 Info Unity UnityEngine.Logger:Log (UnityEngine.LogType,object)
    7. 2023/05/14 00:03:33.112 16191 16219 Info Unity UnityEngine.Debug:Log (object)
    We can see the "RELEASED TEXT FIELD REGION CLICK" Debug line clearly. However when the keyboard is open, we no longer get this by clicking in the same region. Instead we get:

    Code (csharp):
    1. 2023/05/14 00:04:53.565 16191 16191 Info ViewRootImpl@4def854[UnityPlayerActivity] ViewPostIme pointer 0
    2. 2023/05/14 00:04:53.647 16191 16191 Info ViewRootImpl@4def854[UnityPlayerActivity] ViewPostIme pointer 1
    And nothing else. It seems Unity has no awareness of this click at all. It is being blocked, likely by the same Android object that is showing the autocorrect underlines from (ii). Unity will start to become aware of the clicks as you go upwards to leave the "hidden zone".

    This is crippling as any TextField in the region we would typically want it will be unusable.

    Thanks for any further assessment and solutions. I hope for expedience as this has already been 9 months ago reported.
     
    Last edited: May 14, 2023
    huwp and Unifikation like this.
  14. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    I just received an update on my bug for:

    IN-42553 - [Android] Auto-correct suggestions show up over the "hidden zone" when typing text with TouchScreenKeyboard UPDATE: Closed as Won't Fix

    Won't Fix. Resolver: Julius Miknevicius Resolution Notes: Based on the team priorities we will not be able to investigate this issue in the foreseeable future.

    Can anyone at Unity clarify why this is or what the reason is? This is very important for me as I have invested 2 years into an app that relies on social media and chat functions in UI Toolkit and I cannot now complete this project without a working Text Field and hideInput function.
    • Is there a custom fee I can pay Unity to look at what the problem is?
    • Alternatively, can you share details of the existing "hideInput" code so I can build a working version myself from it as a starting point?
    • Or is there any detail that can be share on why you feel this is not fixable or what the challenge is?
    Having a "hideInput" that only partially works and your company will not fix is no use to anyone.

    Is there any chance this could be looked at again and reconsidered? Or if not, would it maybe make sense to opensource that part of the code and let the community fix it together here on the forum so we can all benefit?
     
    Last edited: Jun 8, 2023
    RyanCantCode likes this.
  15. HugoBD-Unity

    HugoBD-Unity

    Unity Technologies

    Joined:
    May 14, 2018
    Posts:
    499
    Hi @mikejm_, looking at our tracking system, it seems like IN-42553 was closed as it's a duplicate of UUM-37652 which is currently open.

    A ticket has also been opened for the second issue you mentioned UUM-39642.
     
    mikejm_ likes this.
  16. mikejm_

    mikejm_

    Joined:
    Oct 9, 2021
    Posts:
    346
    It looks like this has all been fixed as far as I can tell now in 2023.1.7f1 which is absolutely amazing. I just demoed it and as far as I can tell no errors, no blocked clicks, and no autocorrect lines.

    Absolutely amazing. :)

    Thanks again for everything and to the whole Unity team.

    You guys always come through and it's appreciated.
     
    Last edited: Aug 13, 2023
    HugoBD-Unity likes this.