Search Unity

UWP, XboxOne, InputFields, and onscreen keyboards (or lack thereof)

Discussion in 'Windows' started by IQpierce, Dec 15, 2017.

  1. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    I've been porting a large project to the XboxOne on the UWP platform. Our game uses "D3D", rather than "XAML", for performance reasons. I've been working with Unity 5.6.3p4 mostly, though all of the above seems to be true for 2017.2 as well.

    I submitted a bug for an issue I've been struggling with: on this platform, selecting an InputField does not launch the On-Screen Keyboard (OSK) for text entry. Unity launches a touch keyboard automatically on mobile platforms (including, I think, UWP on Windows Phone)... but it definitely does not automatically handle text entry on Xbox One.

    While TouchScreenKeyboard.isSupported reports FALSE on this platform, I found that using TouchScreenKeyboard.Open() does work for opening the OSK on this platform and retrieving the entered text after the user presses the Menu/submit button.

    Using this to create a workaround seems like it would be trivial; and indeed doing that is what Unity rep Tautvydas-Zilys suggests in this forum thread.

    However, there are two major issues I've had trying this, which aren't discussed elsewhere:
    1. Cancelling the keyboard with the "B" button is completely not recognized by Unity. The keyboard disappears, and no other controller input is accepted; but the TouchScreenKeyboard instance, and the static variables, are all unchanged when this happens, so there's no way to detect this state. (Tried the "visible", "active", "done", "status", and "wasCanceled" flags, to no success.)
    2. The OSK's text entry field is not shown (ie the white box above the keyboard that shows the text being entered). For our InputFields on the lower half of the screen, this means the player can't read the text they're entering.
    So my question (preferably for a Unity dev) is: I successfully created a native plugin to let me fix #1. Do I have to do the same to fix #2?

    More importantly, does Unity plan to implement this missing functionality? This is important UI functionality, and I assume the relatively few mentions of it on the forums only because there aren't very many XboxOne UWP games made in Unity. The keyboard can obviously be launched and used to some extent since TouchScreenKeyboard (mostly) functions; why isn't the full functionality there to do all this automatically, without the need for a native plugin and a workaround script?

    This issue seems to imply that it's "by design" that D3D apps in UWP are unable to support the onscreen keyboard, and that this is only supported on XAML. However we need to use D3D, and we need this functionality.

    I hope a Unity dev can advise on the right way to go here, or at least someone who's dealt with this problem and can confirm I'm going down a reasonable path with this workaround.
     
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    I wasn't aware that this wasn't working. The keyboard on d3d build type has been fixed a while ago - in Unity 5.5. All of this sounds very bizarre.

    The answer is yes, and I strongly believe that the current behaviour is simply a bug. Could we get a bug report on that?
     
  3. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    Thanks for the prompt response. The thread from February about this issue led me to believe that was missing functionality, and that a TouchScreenKeyboard workaround was the only solution.

    The bug I submitted recently is here: https://fogbugz.unity3d.com/default.asp?978369_h663u031puqcddtr

    If it makes a difference: I am testing on an xbox kit flashed with OS version 16299.3036. I can restore it to a different version of the OS if you believe this is an issue with the OS.

    I also forgot to mention that I'm using InControl for gamepad interaction - but I believe this is just using standard Unity methods for activating an InputField.

    I just tried making a new repro project for this. It does not appear to happen in XAML projects, but it definitely happens to this D3D test project... attached.
     

    Attached Files:

  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    Thanks, we'll take a look.
     
    BrandyStarbrite likes this.
  5. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    (And just to be clear - the repro project I added does NOT use the InControl plugin, and still demonstrates this problem, so InControl isn't the culprit here.)

    Thanks again for looking into this @Tautvydas-Zilys!
     
  6. Ayrik

    Ayrik

    Joined:
    Aug 31, 2008
    Posts:
    426
    @Tautvydas-Zilys it appears this is still broken. Obviously not a lot of people need this functionality but I just wanted to state, for the record, my desire to have it fixed. Thanks!
     
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    Yeah, I'm aware. It's pretty high in priority to look into - I hope to get to it next week.
     
  8. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    Good news: I fixed it last week and I'm going to be backporting the fix all the way to 2017.4 in coming weeks.
     
  9. Ayrik

    Ayrik

    Joined:
    Aug 31, 2008
    Posts:
    426
    Thank you very much! \o/
     
  10. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    I use a latest Unity 2018.1.2f1 and TMP_InputField (latest version) on Creators Xbox One and it does not bring the screen keyboard like in other platforms eg mobile. This is very important.
     
  11. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    A year and a half after this bug was found, Unity has released new major versions (2018) but this bug is still there in D3D! This makes the Creators program from Microsoft via Unity completely useless in D3D. All Apps and Games need an onscreen keyboard.
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    The fix is landing to patch releases very soon. I fixed it a couple of weeks ago.
     
    Ayrik likes this.
  13. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    Thats good to hear but I already made my own custom keyboard for these platforms.
     
  14. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    Its been 2 weeks but I didnt see it in the latest patches. When do you expect it?
     
    philly2013 and Ayrik like this.
  15. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    I am curious as well. My project is still using a workaround script to bring this keyboard up and dismiss it; and bringing up the keyboard this way has at least one issue (it doesn't display the text entry, so for InputFields in the lower half of the screen, the player can't see the text they're entering as they're entering it).

    And the workaround doesn't work for bringing up the OSK on Windows 10 at all. (Is that supported? It's something we'd like to do through the OS when the player is using a gamepad.)
     
    Ayrik likes this.
  16. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    The fix should work on Windows Desktop, Mobile and Xbox one.
     
  17. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    The fix landed to 2018.1.8f1 and 2017.4.7f1.

    We don't have any support for it on the Standalone player. As for UWP, while we do have support for it, I don't think that will work. I believe Windows itself won't show a keyboard if a physical one is connected (at least that was the case few versions ago, I don't know if it changed). Feel free to report a bug if you think we should investigate this scenario further.
     
  18. bphillips09

    bphillips09

    Joined:
    Aug 31, 2014
    Posts:
    33
    It seems it is still not possible to detect when the "B" or "Back" button is pressed and the OSK is dismissed. When dismissed, the TSK instance still shows Active as True and Status as Visible even though the keyboard is gone. Is this desired behavior? Is there another way to detect when the OSK is dismissed on Xbox One?
     
  19. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    That is totally unexpected. Which Unity version are you on?
     
  20. bphillips09

    bphillips09

    Joined:
    Aug 31, 2014
    Posts:
    33
    Unity 2018.2.14f1
     
  21. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    Can we get a bug report on it?
     
  22. bphillips09

    bphillips09

    Joined:
    Aug 31, 2014
    Posts:
    33
    Sure, just tried it on 2018.2.17f1 and the issue still exists. Here's the case number for the bug report (it happens on the example project I made for the bug report too): 1104971

    Another possibly related issue: once text is entered and the caret is moved with right/left bumper, entering new text will ignore the caret's position and will reset the caret to the end of the text and enter the text there. I assume this is not intended behavior either?
     
  23. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    You're right, that doesn't sound right either.
     
  24. Homicide

    Homicide

    Joined:
    Oct 11, 2012
    Posts:
    243
    I don't know whats going on here, but i followed this thread, and assumed that all input field issues would have been resolved by now, but it seems this is not so.

    https://issuetracker.unity3d.com/is...putfield-is-touched-with-android-api-level-28

    I am trying to build to android , and migrate around the app using a gamepad, everything works great, except InputFields. They dont work at all. I have been able to force the Keyboard to open like this ...

    Code (CSharp):
    1.       InputField _inputField;
    2.         void Start()
    3.         {
    4.             _inputField = GetComponent<InputField>();
    5.         }
    6.      
    7.         void Update()
    8.         {
    9.             if (Input.GetButtonDown("Submit"))
    10.             {
    11.                 TouchScreenKeyboard.Open("", TouchScreenKeyboardType.Default, false, false);
    12.             }
    13.  
    14.  
    15.         }
    16.     }
    But, once the virtual keyboard opens, it can not be navigated with a controller at all. The controller is simply stuck between the text preview field , and the 'ok' button to submit the text.


    Really frustrating, as this prevents anyone from using my app, ultimately making it a collosal waste of time, as it requires login and room creations with unique names, etc. And specifically targets Android / SHIELD TV.

    Any ideas or pointers guys?
     
  25. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    This thread/forum specifically discussed issues with UWP/Xbox One.I suggest posting on an android forum about this.
     
  26. Homicide

    Homicide

    Joined:
    Oct 11, 2012
    Posts:
    243
    Ahh yes, i just did that , sorry bout that :D
     
  27. kenzstudiosgaming

    kenzstudiosgaming

    Joined:
    Apr 16, 2019
    Posts:
    2
    We seem to be still hitting this issue today with both 2018 Unity and 2019. Are there any workarounds working for you guys?
     
  28. giorgos_gs

    giorgos_gs

    Joined:
    Apr 23, 2014
    Posts:
    560
    The only stable workaround is to make your own keyboard and use it on whatever platform you have problems with.
     
  29. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    What scenario isn't working specifically? All known issues that were reported were fixed.
     
  30. WarmedxMints

    WarmedxMints

    Joined:
    Feb 6, 2017
    Posts:
    595
    Apologies for bumping and old thread, however, most of my google searches lead me here.

    I am also having an issue. I can use the TouchScreenKeyboard class to open the on screen keyboard on an XB1 however I cannot seem to get it to display an input field and while I can get it to put the text into an input field. I cannot delete characters nor does pressing the view button or enter on the on screen keyboard close itself.

    I am using Unity 2019.2.0f1 and the following code

    Code (CSharp):
    1.     [SerializeField]
    2.     private InputField TextInputBox;
    3.  
    4.     private TouchScreenKeyboard keyboard;
    5.  
    6.     private void Start()
    7.     {
    8.         TouchScreenKeyboard.hideInput = false;
    9.     }
    10.  
    11.     private void Update()
    12.     {
    13.         if (Input.GetButtonDown("MENU"))
    14.         {
    15.             keyboard = TouchScreenKeyboard.Open(TextInputBox.text, TouchScreenKeyboardType.Default, false, false, false);
    16.         }
    17.         if (keyboard == null)
    18.         {
    19.             return;
    20.         }
    21.  
    22.         TextInputBox.text = keyboard.text;
    23.     }

    Just to note, I am building a D3D project.
     
  31. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    I can't remember exactly, but I think input from keyboard on UWP comes from Input.inputString. Pressing backspace shows up as '\b' character.