Search Unity

Feedback Mouse and Keyboard support for iOS

Discussion in 'Input System' started by AcidArrow, Oct 11, 2020.

  1. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,794
    Is mouse and keyboard supported with the Input System for iOS/iPadOS 13+?

    If not, why not?
     
    AcidArrow, Oct 11, 2020
    #1

  2. unity_r71wO1KFVJaK9w

    unity_r71wO1KFVJaK9w

    Joined:
    Dec 15, 2020
    Posts:
    7
    Unity input system does not support keyboard and Mouse functionality in input system.
    https://docs.unity3d.com/Packages/com.unity.inputsystem@1.0/manual/SupportedDevices.html
    You have to make a native plugin with wrapper using GCKeyboard and GCMouse (just introduced in IOS 14) which will callback to unity code as all the native ads plugin wrappers does.
    I think unity should update the input system as iOS 14 supports Keyboard and Mouse.
    The only issue I am facing now to lock the pointer inside unity Game.
     
    unity_r71wO1KFVJaK9w, Jan 10, 2021
    #2

  3. anatolyV

    anatolyV

    Joined:
    Nov 29, 2014
    Posts:
    63
    Hi, from this and your other posts I'm getting the impression that you've written a native wrapper for GCKeyboard. Would you be able to share it or sell on Asset Store for example?
     
    anatolyV, Jul 14, 2021
    #3

  4. Bastienre4

    Bastienre4

    Joined:
    Jul 8, 2014
    Posts:
    191
    I'm also interested for this :)
    It's sad that Unity does not handle this.
    I get keyboard events but I need to let the key rest by something like 0.5s between each stroke.
    But i'm mostly interested by the mouse support.
     
    Bastienre4, Sep 24, 2021
    #4

  5. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    Keyboard on iOS works acceptable last time I've checked, both old and new systems go through same code, though they go via UIKeyCommand which might introduce some quirks. I've tried it via pairing Apple bluetooth keyboard to iPad.

    The only downside right now is that we're not layout independent there, e.g. if you physically change your keyboard input language to something like AZERTY, input system package will fail to correctly recognize Q/A locations, though this setting is hidden very deep in iOS settings (this is different one to your onscreen keyboard language), same story on Android.

    Mouse on Android works also.
    Mouse on iOS might work, I haven't tried, but it's likely to be the same way as mouse would work with any iOS application then.

    GCKeyboard and GCMouse are not supported as of right now. Would you folks mind reporting a bug via Help->Report a Bug so we can keep track of it? Thanks!
     
    Last edited: Sep 27, 2021
    dmytro_at_unity, Sep 27, 2021
    #5
    Bastienre4 likes this.

  6. Bastienre4

    Bastienre4

    Joined:
    Jul 8, 2014
    Posts:
    191
    First, thanks for your answer.

    I will do a bug report (containing a link to this conversation) about this.

    About the keyboard:
    - The fact that it's not layout independent is a problem, but if it's the only issue here, it's acceptable. We can do a rebind menu.
    - It's not the only issue with the keyboard though. I noticed that if I press a key (in this example, ' = ' key), the action is sent only if I wait like 0.5s between each physical key press. I've made a record with the Input debugger connected to the iPad (that is an AWESOME tool by the way, good job here). I press and release the ' = ' key. Sometimes I wait AFTER releasing the key, sometimes I spam the key. Notice how the only difference is the "Format=KEYS". If this is present, the action works as expected.
    - Please note that if we're using the keyboard in an input field it works fine. What I said is only valid for "gameplay" inputs (like WASD movement for instance)
    - I tested with the magic keyboard and with a bluetooth keyboard.



    About the mouse:
    - It's recognized as a touch. I checked all the InputDevice and there's no mouse. I checked with a mouse connected in bluetooth and with the magic keyboard (trackpad). I can't get the cursor position unless there's a touch (= click).
    - I would love for the mouse to be recognized as a "Mouse" device.

    Have a nice day.
     
    Bastienre4, Sep 27, 2021
    #6
    dmytro_at_unity likes this.

  7. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    Ok I see, likely UIKeyCommand goes through Cocoa shortcut/combo recognizers or something first and that delays the input. Definitely GCKeyboard should be better here. Thanks for the bug, we will have a look.
     
    dmytro_at_unity, Sep 28, 2021
    #7
    Bastienre4 likes this.

  8. anatolyV

    anatolyV

    Joined:
    Nov 29, 2014
    Posts:
    63
    I think this issue is also causing issues in the 'iOS apps on M1 Mac' scenario, the keyboard inputs aren't recognized properly, although I haven't delved too deep into the cause of this.
     
    anatolyV, Sep 28, 2021
    #8

  9. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    @aVolnov is it the same behavior on iOS apps on Mac? aka input of some keys is delayed?
     
    dmytro_at_unity, Sep 29, 2021
    #9

  10. Bastienre4

    Bastienre4

    Joined:
    Jul 8, 2014
    Posts:
    191
    BTW I reported the bug. Case 1369022.
     
    Bastienre4, Sep 29, 2021
    #10
    dmytro_at_unity likes this.

  11. anatolyV

    anatolyV

    Joined:
    Nov 29, 2014
    Posts:
    63
    No, sorry. Keyboard inputs are completely ignored under macOS.
     
    anatolyV, Sep 29, 2021
    #11
    dmytro_at_unity likes this.

  12. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    @aVolnov would you mind filing a separate bug report for keyboard support for iOS apps running under macOS on M1? Both old and new input systems are driven by same cocoa touch code here, so if one doesn't work for you it's likely another also doesn't work. I've asked around and there is at least one report that it worked a while ago, but we need to recheck then.
     
    dmytro_at_unity, Sep 30, 2021
    #12

  13. anatolyV

    anatolyV

    Joined:
    Nov 29, 2014
    Posts:
    63
    Alright, and just as another bit of info, input fields do work as expected. If I file a bug report on this particular, should I submit a repro project as well? Wouldn't be too hard to create one considering what the issue is but I would still need to carve out a bit of time for that.
     
    anatolyV, Sep 30, 2021
    #13

  14. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    @aVolnov Yes bug reports do need a repro case so our QAs can test them, I guess something like this will do:
    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3. using UnityEngine.InputSystem;
    4.  
    5. public class TestScript : MonoBehaviour
    6. {
    7.     void Update()
    8.     {
    9.         foreach (var keyCode in (KeyCode[]) Enum.GetValues(typeof(KeyCode)))
    10.         {
    11.             if (Input.GetKeyDown(keyCode))
    12.                 Debug.Log($"old input, {keyCode} pressed");
    13.  
    14.             if (Input.GetKeyUp(keyCode))
    15.                 Debug.Log($"old input, {keyCode} released");
    16.         }
    17.  
    18.         if (Keyboard.current == null)
    19.             return;
    20.        
    21.         foreach (var keyCode in (Key[]) Enum.GetValues(typeof(Key)))
    22.         {
    23.             if (keyCode == Key.None || keyCode == Key.IMESelected)
    24.                 continue;
    25.        
    26.             if (Keyboard.current[keyCode].wasPressedThisFrame)
    27.                 Debug.Log($"new input, {keyCode} pressed");
    28.        
    29.             if (Keyboard.current[keyCode].wasReleasedThisFrame)
    30.                 Debug.Log($"new input, {keyCode} released");
    31.         }
    32.     }
    33. }
    34.  
    35. // and in separate file
    36. using UnityEngine;
    37. public class LogPrinter : MonoBehaviour
    38. {
    39.     private string log;
    40.     void OnEnable () {
    41.         Application.logMessageReceived += HandleLog;
    42.     }
    43.    
    44.     void OnDisable () {
    45.         Application.logMessageReceived -= HandleLog;
    46.     }
    47.     void HandleLog(string logString, string stackTrace, LogType type){
    48.         log = $"\n{Time.timeAsDouble:F2} {logString}{log}";
    49.         if (log.Length > 1000)
    50.             log = log.Substring(0, 1000);
    51.     }
    52.     void OnGUI () {
    53.         GUILayout.Label(log);
    54.     }
    55. }
    56.  
    I don't immediately have direct access to M1 macbook so unfortunately can't test it right now
     
    dmytro_at_unity, Oct 4, 2021
    #14
    Bastienre4 likes this.

  15. Vincent-Parker-WB

    Vincent-Parker-WB

    Joined:
    Nov 15, 2018
    Posts:
    1
    After installing iOS 15 on my iPad and iPhone, all external input from a Bluetooth keyboard is ignored unless you are in an active typing field. Tried the legacy input system, the new input system, as well as rewired plugin. They all work in iOS 14.x but once I updated to iOS 15, all keyboard input except for input fields do not register. (as in key being pressed to do something in code vs. typing in a text field) Tried 2019.4.32f1 & 2020.3.21f1.

    Old Legacy Input System
    Code (CSharp):
    1. void Update()
    2. {
    3.      if (Input.GetKeyDown(KeyCode.A))
    4.      {
    5.           //Do Something
    6.      }
    7. }
    New Input System
    Code (CSharp):
    1. void Update()
    2. {
    3.      var keyboard = Keyboard.current;
    4.      if (keyboard == null)
    5.           return;
    6.  
    7.      if (keyboard.aKey.wasPressedThisFrame)
    8.      {
    9.           //Do Something
    10.      }
    11. }
    I can't get either of these to work in iOS 15 on a Bluetooth keyboard, but they work on macOS, Windows, & Android. and iOS 14.x and earlier.

    Is there something I'm missing?
     
    Last edited: Oct 29, 2021
    Vincent-Parker-WB, Oct 29, 2021
    #15

  16. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    @Vincent-Parker-WB Hi, would you mind to create a bug report for iOS 15 problem specifically so I can forward it to mobile team? Thanks!
     
    dmytro_at_unity, Nov 2, 2021
    #16

  17. bkachmar

    bkachmar

    Joined:
    Mar 15, 2013
    Posts:
    43
    I have the same issue. Our music app uses bluetooth pedal to turn pages.
    And after updating to iOS 15 the "Up Arrow" and "Down Arrow" commands that come from pedal are ignored and the users are complaining that they can use pedal in other apps, but not in ours.
    We're using Unity 2020.3.7f1 and
    if (Input.GetKeyUp(KeyCode.UpArrow))
     
    bkachmar, Nov 22, 2021
    #17

  18. hippogames

    hippogames

    Joined:
    Feb 5, 2015
    Posts:
    234
    hippogames, Dec 10, 2021
    #18

  19. dmytro_at_unity

    dmytro_at_unity

    Unity Technologies

    Joined:
    Feb 12, 2021
    Posts:
    212
    Hi folks, I found where the issue is, because we currently using UIKeyCommand for keyboard input (I know I know, we plan to use GCKeyboard going forward, but it's still some time into the future), iOS 15 has a change in behavior where system handlers take precedence over ours, so none of the input is reaching any of backends. The current fix is to set "wantsPriorityOverSystemBehavior" to YES for all commands. If you're blocked by this you can do it manually in UnityView+Keyboard.mm that is part of iOS player.

    In meanwhile I'm fixing it as part of https://issuetracker.unity3d.com/is...ing-when-using-an-external-keyboard-in-ios-15 and will backport to all supported versions.
     
    dmytro_at_unity, Jan 19, 2022
    #19

  20. hippogames

    hippogames

    Joined:
    Feb 5, 2015
    Posts:
    234
    Good news! I'll wait for resolution, thanks!
     
    hippogames, Jan 23, 2022
    #20

  21. HWIvan

    HWIvan

    Joined:
    Jan 21, 2019
    Posts:
    12
    Any updated ETA on when we can expect to see updated support for iOS and tvOS devices such as GCKeyboard, GCMouse, GCMicroGamepad and GCDirectionalGamepad?
     
    HWIvan, Jun 30, 2022
    #21
    Inter-Illusion, Brainferno, sexton_unity and 1 other person like this.

  22. sexton_unity

    sexton_unity

    Joined:
    Dec 17, 2018
    Posts:
    8
    has anyone done a UIHoverGestureRecognizer plugin to get OnPointerEnter and OnPointerExit for ios/ipados?
     
    sexton_unity, Nov 26, 2022
    #22

  23. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,794
    Many years later, keyboard works on iPad, but mouse doesn't, please add support for it for both the old input manager and the new input tragedy.
     
    AcidArrow, Jan 20, 2024
    #23