iOS Bluetooth Keyboard

I'm looking to implement hardware keyboards on an iOS app for accessibility reasons. I've tested on an iPad with a Bluetooth keyboard. The keyboard works for typing into text fields, but I'm not getting any inputs i.e. Input.GetKeyDown.

I found a bunch of old forum posts mentioning this issue with no solutions. Does anyone know of a solution or work around?

 

I have created an asset that does exactly this - respond to keystroke events from connected Bluetooth accessories. I was surprised that this functionality doesn't exist in Unity by default. The asset was submitted to the Asset Store for review about 10 days ago and I'm expecting a response any time now. It'll be free and open source when it is published. I will update this thread as soon as it's available.

 

It's available on the Asset Store now!

http://u3d.as/1cUf

 

Hello aihodge!

Thank you so much for your asset - it is exactly what I'm looking for, and I can't wait to use it!!

However, I'm having some trouble building it in xCode. I can't seem to synch up the correct versions of xCode/Unity/iPhone/iOS/Deployment Target, and I'm curious if you could let me know which you used, or what your asset supports?

I'm building with:

xCode 9.2
Unity 2017.3.0
iPhone 5c
iOS 10.3.3
Deployment Target 10.3

I get an number of warnings related to 'Unsupported Configuration' and 'Deprecations'

When I build the scene you provided: 'ExternalInputExample, I DO get the key presses in the console log.
So I know that it works.

However, when I add my own scripted key interactions using the bluetooth connected keyboard, my key presses are ignored.

As a sanity check, I did build a Mac build direct from Unity and the key presses respond appropriately.

Please let me know if you have any advice on best-set-up-practices.
Or any other tips for that matter.

Thank you so much for your help!!

Sincerely,
kropcke

 

Hi kropcke,

Thank you for downloading the iOS External Input asset. The version on the Asset Store was built with Unity 5.6.4, and tested on iPhone 7 Plus running iOS 11.4. I am aware of an error which Xcode will throw when using an earlier version of iOS as the deployment target, and it involves the KeyInputViewController xib's conforming to the "Safe Area" Auto Layout guides that have been added for the iPhone X unique screen dimensions. The simple fix is to deselect "Safe Area Layout Guide" and "Safe Area Relative Margins" for the xib in Xcode. If you could post the specific warnings you are seeing related to unsupported configurations and deprecations I could look into that further.

Regarding best practices, I'd recommend implementing a C# event/delegate structure in your game following this example:

https://unity3d.com/learn/tutorials/topics/scripting/events

In the iOSExternalInputController class, you’d add delegate and event properties to the class and broadcast the event in DidReceiveKeystroke(). Then in your class which handles your input in your game, you’d subscribe to the event from iOSExternalInputController, and define a method which gets called whenever that script receives an event. From there you can have your script do whatever you want with the input.

Conceptually, you can think of the asset's native plugin as "pushing" or "forwarding" key events from connected Bluetooth accessories to the iOSExternalInputController class. However, they are forwarded as strings rather than KeyCode types.

Hope this helps! Please let me know if you have any other questions.

 

Hi aihodge!

Thank you so much for the thoughtful response. I'm still struggling a bit, but I want to dig into it further before I come back with more questions. Truth be told, I haven't had a chance to work on it this past week and am just getting started again.

Mostly, I wanted to say 'thanks!' and that I will reply after I try it out some more.
Thank you again for all your help. Talk soon!

-kropcke

 

Sounds good! Let me know if I can help with deploying it in your project.

 

Hello! Just downloaded your asset, it looks like it's exactly what i need, however i am getting the same problems where do i find "Safe Area Relative Margins"? I found the first item but can't as of now find that second checkbox...

 

Hi @bennett_apps, it's under the Size Inspector panel for the View in the xib file. See attached screenshot.

 

LOL how the heck did i miss that? Thanks!

I unchecked both boxes, attempted a build, and got the error "Linker command failed with exit code 1 (use -v to see invocation)"
Any clue what that even means?

 

It's okay if you don't, just let me know if you have any more information on this please

 

Hi @bennett_apps, does Xcode provide any more details with this error? What is your deployment target? If you haven't yet, try rebuilding from Unity with the modified xib in the Assets/Plugins/iOS folder.

 

Trying to build to iPhone 5 (actual device).
I'm not sure I understand...the xib that was modified changed nothing in unity...only the xcode project that unity created. Do you mean somehow copy that modified xib into unity and rebuild?

 

Yes, that's correct. The modified xib can go in Assets/Plugins/iOS with the rest of the native plugin assets, replacing the existing xib there.

 

Like import it in? OK ill try that.

 

Yes, by replacing the existing xib in the Plugins folder. Let me know how you get on with this specific issue or if you have any other questions relating to the iOS External Input asset.

 

My apologies for taking so long. i've been working on other things. seeing an email that your asset was "just updated" reminded me! I tried importing xib and re-building in unity, but same affect I really want to get this to work, but I haven't found a way yet! What Xcode did you use when testing this asset? And any other ideas?

 

I always use the current (non-beta) version of Xcode. Why don’t we arrange a time to screen share and figure this out? Feel free to DM me and I’ll share my contact info.

 

currently, I do most of my work on a windows laptop, and if I want to build for iPhone, I transfer xcode project over to really old mac and build from there. So it's probably gonna be pretty hard to screen share. fortunately, I am getting a newish MacBook air soon, one that will run latest xcode. I didn't know there was a DM in forums, let me check it out...

 

Replied!

 

Hi, I also tried to find this asset. Thanks!
I successfully built my app and got into my iphone & ipad, but it doesn't work..

I used your ExternalInputExample.scene



And my gameobject sprite is like this.




I just want to change the round square's color when I press the right/left arrow in the bluetooth keyboard.

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class TestKey : MonoBehaviour {
6.     private SpriteRenderer spr;
7.  
8.     // Use this for initialization
9.     void Start () {
10.         spr = GetComponent<SpriteRenderer> ();
11.         spr.color = Color.blue;
12.         ExternalInputController.instance.OnExternalInputDidReceiveKey += HandleExternalInput;
13.     }
14.  
15.     void HandleExternalInput(KeyCode receivedKeyCode)
16.     {
17.         // respond to received KeyCodes here
18.         if(receivedKeyCode == KeyCode.LeftArrow){
19.             spr.color = Color.red;
20.         } else if(receivedKeyCode == KeyCode.RightArrow){
21.             spr.color = Color.blue;
22.         }
23.     }
24. }

I wonder if I'm wrong or missing some implementation?

 

Hi @ricegood, here's a few things you can try on your end to troubleshoot:

1) Double check that the asset's native plugin files are being added to the generated Xcode project.
2) Add some logging to the ExternalInputController class and to your TestKey class' HandleExternalInput methods to check that the key codes are being received on the managed (C#) side.
3) Double check that the bluetooth keyboard that you're using is properly connected to your iOS device and is sending key codes to other apps (ex: open up the Notes app and try typing there).

Let me know the results of those steps. I'm happy to continue troubleshooting this over DMs if you like.

 

Hi @aihodge, thanks a lot for your asset.
I'm now building an iOS version TANKS game based on the official tutorial, and I want to use a bluetooth keyboard to control the tank and it's shooting.
To achieve that, the original version(on computer) is using Input.GetKeyDown, Input.GetKey and Input.GetKeyUp in update function to control the shooting force.

Code (CSharp):

1. if (Input.GetKeyDown(Keycode.Space)) {
2.   // press fire for the first time?
3.   m_Fired = false;
4.   m_CurrentLaunchForce = m_MinLaunchForce;
5.  
6. } else if (Input.GetKey(Keycode.Space)) {
7.   // holding the fire button, not yet fired
8.    
9.   m_CurrentLaunchForce += m_ChargeSpeed * Time.deltaTime;
10.   m_AimSlider.value = m_CurrentLaunchForce;
11.  
12. } else if (Input.GetKeyUp(Keycode.Space) && !m_Fired) {
13.   // release the button, fire!
14.   Fire();
15. }

I wonder if I can achieve the same effect with your asset?

 

Hi @duck105, yes the iOS External Input asset can achieve the control scheme you described. Conceptually, you can think of the asset's native plugin code as "pushing" the key presses from the Bluetooth keyboard over the plugin's native-to-managed bridge onwards to the ExternalInputController C# script where you can filter the different Keycodes that you're using as your controls in a switch statement (or register another class to respond to OnExternalInputDidReceiveKey events), and assigning the corresponding functions to be called by the appropriate Keycodes.

Happy to answer any other questions you have or to help with your specific implementation.

 

Just popping in to thank you @aihodge for making this asset. I'm about to port Mini Micro to iOS, and being able to access key-down and key-up events on Bluetooth keyboards is absolutely essential.

I don't suppose you know of a similar asset for Android?

 

Hi there! Thank you for making this asset, it overcomes a major flaw in Apple's iOS I/O design.
I still have to try this, just wanting to ask beforehand: does this work with HID devices connected via USB to an iOS device?

 

I'm finally getting around to actually trying this! In response to @duck105 query about KeyUp/KeyDown events...

But now that I look at it, I don't see how. ExternalInputController has only a "did receive keystroke" event. Nothing for actual key-down and key-up.

If we want to do typical game controls, where a character moves as long as the key is held down and then stops moving when the key is released, is that impossible with a Bluetooth keyboard?

EDIT: To partially answer my own question, it looks like it is possible since iOS 13.4. But this plugin predates that, so... :|

 

You're correct, the version of iOS External Input currently on the Asset Store cannot differentiate between Up/Down states of the key being pressed - what it does with DidReceiveKeystroke() is pass the corresponding key code to the managed C# environment. My response to @duck105 failed to make that clear, sorry about that! This was because it wasn't possible to get key states on iOS when I first made this plugin, but responding to key commands is a common access pathway in the world of hardware/software accessibility which is what I had in mind initially.

In regards to the new support in iOS 13.4, this is great news and I'll certainly be updating the asset to differentiate between UIKey states, along with better tvOS support soon. Thank you very much for bringing this to my attention!

 

I expect that it should work with USB connected devices as well, although I don't have the hardware to test this myself right now. Please let me know if you attempt it and I will update the asset's description if it works.

 

Hi @aihodge !
First of all thank you for your work! Moreover to publish it as free asset and open source. You're an angel.
One quick question though, does this plugin only work with MFI controller?

I tried to got it to work with old, unbranded Chinese made VR controller to no avail. DidReceiveKeyoke isn't called at all. I've also made sure that mm, xib, h files exist ion Xcode.

Also, for additional info my controller is detected as unidentified keyboard when connected to MacBook, the only controls available are volume up, volume down, mute, play/pause music (iOS & Mac) which all are unrecorded by this Plugin.

 

Heya @aihodge !

Quick question whenever you get a chance, we've added this package for grabbing input via keyboards on iPadOS and it seems to be working fine, however, we don't seem to be getting any input when running on macOS.

Specifically, we have an iPad/iPhone app that runs on the new M1 Macs with Apple Silicon and were hoping the capabilities might translate over to that version. I tried with both the built-in keyboard on the Macbook Air as well as an external Apple bluetooth keyboard.

Is this expected? I know this was created well before the new Apple Silicon machines were released so it's hard to anticipate that use-case... I'm just wondering if there's anything you can think of that might explicitly forbid it from working there, or if maybe it's something that should work but some sort of bug or issue is getting in the way.

Thanks again for all your contributions!

 

Hi @kukuhbr and @oneilnyc !

Thank you both for the feedback. You actually reminded me that I need to update the package on the Asset Store as the support for down/up key phases, as well as other features introduced in iOS 13.4, have been successfully integrated into iOS External Input (thanks to @JoeStrout for help with beta testing!), and these new capabilities may be relevant to arriving at solutions for your respective use cases. I'll be submitting the update today.

The asset should work with any HID-compliant input device that can be configured to send key codes. If you are able to determine the specific key codes corresponding to volume up/down/mute/etc., you could add these to the native plugin code where it constructs a lists of keys to "listen" for - I don't think volume keys are included in the list currently.

With the version currently on the Asset Store, yes this is expected behaviour. With the new update that I'll be submitting today, I have a strong feeling that it could work on Apple Silicon as the new APIs from iOS 13.4 appear to be an attempt to bring parity to the input capabilities of iOS/iPadOS and macOS (Apple's API documentation for the new features that I was using as a reference in updating iOS External Input is in fact categorized under Mac Catalyst apps), though I don't have an Apple Silicon-based computer with which to test. Will be replying to the email that you sent me with some more thoughts around getting it to work on Apple Silicon.

 

Awesome, thanks! We'll give the new package a shot and report back.

 

I'm having a crash on my iOS app as soon as the ExternalInputController is activated. The crash log doesn't seem to contain any useful clues (it just shows a very long stack of objc methods). But I am running 15.3.1, and I notice the latest review on the asset store claims that it crashes for them on iOS 15 devices.

He also points to this Unity issue with Bluetooth keyboards not working in iOS 15 with normal Unity input, too. So I guess Apple has changed (broken) something here, requiring developers to update their code.

@aihodge, any chance for an update to this excellent plugin to work with iOS 15? And, is there any work-around in the meantime — some way to configure Xcode to target an older SDK or something?

 

Thank you @JoeStrout for bring this to my attention. For anyone coming across this before it's updated on the Asset Store, please read on for a solution:

iOS External Input relies on a .xib file within the UnityFramework bundle. This .xib file is the native view of size (0,0) that is overlayed onto the UnityPlayer view of your Unity iOS game and its class defines the native interface for the Bluetooth input devices which get sent to Unity's managed C# environment. It looks like the plugin’s post processing code was using an incorrect path to load the xib, and the solution is to fix the path so it reflects the location within the UnityFramework bundle.

To fix the path, open ExternalInputController.mm, and find the line where ‘keyInput’ is allocated and initialized with the xib. It should be line 29, or around there. Replace it with this new line of code:

Code (CSharp):

1. keyInput = [[KeyInputViewController alloc] initWithNibName:@"KeyInputViewController" bundle:[NSBundle bundleForClass:KeyInputViewController.class]];

The change is loading the bundle by determining which bundle to load based on wherever KeyInputViewController class resides, in our case this is UnityFramework bundle.