New Input System - Making a custom HID Device work in Android

Hello!

I have been experimenting with creating my own HID Device to be recognized by Unity applications on Android.

I managed to get a working HID Device in Unity Editor without too much trouble by creating my own layout. However, whenever I build on an Android and connect the device to it, I can't get it to work. I tried logging at different steps in my scripts and none of those show up. On the other hand, when using USB or HID info applications from the play store, our device is correctly recognized by these apps.

Am I missing something obvious?

Thanks in advance!

 

Unfortunately, ATM we don't have HID support on Android. Windows, Mac, and UWP are the only platforms that support raw HID at this point.

It's mentioned in the docs but looking at it, could definitely be made more prominent.

 

Thanks for the repy. Is Android support planned anytime soon? If not, could you advise an alternative way of getting HID to work on Android?

Thanks a lot in advance!

 

Unfortunately, nothing concrete planned as of yet. Think it'll depend on how demand for it will turn out after 1.0.

ATM it'd be down to create a custom Java or C++ Unity plugin for your Android app in order to surface the data.

What's the HID you're looking to get working with the input system on Android?

 

We're working on a positioning box that outputs XYZ and quaternions of a user's position. Just output, no user feedback going back to the device. At the moment we have a working version with a Java library that does the USB work but we thought going HID would make it more standard and cross platform.

We did notice that the device is recognized by Unity on Android, but as a gamepad. We have full control over the device descriptor but we can never get the callbacks to register. On windows it all works perfectly though.

Is there some way of accessing the raw HID buffer so we can parse it ourselves with a c# script?

Thanks a lot for your help and work on the input system!

 

In the input system? (i.e. not the old one)

If so, you may actually be pretty close to hooking this up without needing actual, raw HID support.

What we do for gamepads on Android is simply relay all the data from their Android key and motion events. The key event data comes out in a button array and the axis data comes out in an axis array. You can see the data we surface for controllers in the AndroidGameControllerState struct.

So, if your device shows up as a gamepad, you should already be seeing state events with AndroidGameControllerState payloads flowing in. And in those, all the axes and buttons that you have configured in your HID should be included (up to 48 axes and 220 buttons).

If this is indeed the case, then all you need is a custom layout that overrides the built-in Android gamepad layout for your device and diverts control to you. Simply copy AndroidGameControllerState and customize the InputControlAttributes to your need and set up a custom InputDevice-derived calss that references the struct in its InputControlLayoutAttribute.

Then register it like so.

Code (CSharp):

1.             InputSystem.RegisterLayout<YourDevice>(
2.                 matches: new InputDeviceMatcher()
3.                     .WithInterface("Android")
4.                     .WithDeviceClass("AndroidGameController")
5.                     .WithProduct("YourProductString");

////CAVEAT: I just noticed that the Android support in the input system registers a custom callback that performs some layout searching of its own and will disregard if the input system already found a matching layout. This may interfere with the code above. If so, hook into InputSystem.onFindLayoutForDevice to override the result. I'll make sure we fix that problem on our side.

 

Fix for non overridable Android gamepad layouts in flight here. Should be in next package.

 

Thank you very much for all this information! This will probably get us going. I'll try it all out and will get back to you with the results, probably monday afternoon.

 

OK, I made some good progress, it's almost working but not yet fully. With your changes to AndroidSupport.cs the device is recognized properly and the proper layout is selected but the report does not seem to get properly parsed. The format is declared in the input debugger as AGC. Should I create a different layout for android?

I can send an input trace from the debugger if you want. Attached is my custom layout. It's not fully finished but should be working.

 

The format tag in this case needs to match the one that is used by native to send the data. So yes, it'll have to be AGC.

Also, the format of the struct must follow AndroidGameControllerState. The HID input report format in this case will *not* come through in managed code as the native code in Unity is picking up input through Android events.

Overall, I'd recommend either using AndroidGameControllerState as is like below or creating a customized version of it (which, however, still has to follow the memory layout already prescribed by the existing struct).

////EDIT: actually, I just realized that using AndroidGameControllerState as is will make it pick up the controls from Android gamepads which you don't want. Adjusted the code below.

Code (CSharp):

1.  
2. struct AHIAHIDDeviceState : IInputStateTypeInfo
3. {
4.     public FourCC format => new FourCC('A', 'G', 'C');
5.  
6.     internal const uint kAxisOffset = sizeof(uint) * (uint)((MaxButtons + 31) / 32);
7.  
8.     public fixed uint buttons[(MaxButtons + 31) / 32];
9.  
10.     // Need to determine which 3 axes in the axis array correspond to the device position.
11.     // If they are not consecutive in the array, that'll be a problem.
12.     // Let's assume is axis 0, 1, 2 here.
13.     [InputControl(name = "devicePosition", layout = "Vector3", offset = kAxisOffset]
14.     // Let's say deviceRotation is axis 3, 4, 5, 6.
15.     [InputControl(name = "deviceRotation", layout = "Quaternion", offset = kAxisOffset + 3 * sizeof(float))]
16.     // Do same for batteryLevel and trackingState.
17.     public fixed float axis[MaxAxes];
18. }
19.  
20. [InputControlLayout(stateType = typeof(AHIAHIDDeviceState))]
21. #if UNITY_EDITOR
22. [InitializeOnLoad]
23. #endif
24. public class AHIAHIDDevice : TrackedDevice // Or InputDevice, whichever you prefer.
25. {
26.     public AxisControl batteryLevel { get; private set; }
27.  
28.     // Same for batterLevel and trackingState...
29.  
30.     protected override void FinishSetup()
31.     {
32.         base.FinishSetup();
33.         batteryLevel = GetChildControl<AxisControl>("batteryLevel);
34.    }
35.  
36.    static AHIAHIDDevice()
37.    {
38.        InputSystem.RegisterLayout<AHIAHIDDevice>(
39.            matches: new InputDeviceMatcher()
40.                .WithInterface("Android")
41.                .WithDeviceClass("AndroidGameController")
42.                .WithCapability("vendorId", 0x4040)
43.                .WithCapability("productId", 0x4040)
44.            );
45.    }
46.    [RuntimeInitializeOnLoadMethod]
47.    static void Init() { }
48. }
49.  

 

Step by step it's getting closer but some issues still remain. I had to change a few things to the code you wrote above and now it's recogized properly but I'm getting some errors in the debug. I'll keep investigating but all your help up until now has been very helpful!

Code (CSharp):

1.  
2. 03-25 14:51:40.241 12978 12993 D Unity   : Input: attaching joystick ["Demute AHIA"][0x00000018:e932f40a13f42cec0cef64c9c8225a2fc0afe85e] at pos [0x01]
3. 03-25 14:51:42.295 12978 12993 D Unity   : UnloadTime: 4.458291 ms
4. 03-25 14:51:43.720 12978 12993 E Unity   : NotImplementedException: exponents
5. 03-25 14:51:43.720 12978 12993 E Unity   :   at UnityEngine.InputSystem.Utilities.JsonParser.ParseNumber (UnityEngine.InputSystem.Utilities.JsonParser+JsonValue& result) [0x0023f] in /home/waldek/bin/unity3d/ahia_hid/Library/PackageCache/com.unity.inputsystem@1.0.0-preview.6/InputSystem/Utilities/JsonParser.cs:419
6. 03-25 14:51:43.720 12978 12993 E Unity   :   at UnityEngine.InputSystem.Utilities.JsonParser.ParseValue (UnityEngine.InputSystem.Utilities.JsonParser+JsonValue& result) [0x000ca] in /home/waldek/bin/unity3d/ahia_hid/Library/PackageCache/com.unity.inputsystem@1.0.0-preview.6/InputSystem/Utilities/JsonParser.cs:231
7. 03-25 14:51:43.720 12978 12993 E Unity   :   at UnityEngine.InputSystem.Utilities.JsonParser.ParseValue () [0x00001] in /home/waldek/bin/unity3d/ahia_hid/Library/PackageCache/com.unity.inputsystem@1.0.0-preview.6/InputSystem/Utilities/JsonParser.cs:195
8. 03-25 14:51:43.720 12978 12993 E Unity   :   at UnityEngine.InputSystem.Utilities.JsonParser.NavigateToProperty (System.String path) [0x00298] in /home/waldek/bin/unity3d/ahia_hid/Library/PackageCache/com.unity.inputsystem@1.0.0-preview.6/InputSystem/Utilities/JsonParser.c
9. 03-25 14:51:43.743 12978 12993 D Unity   : Sensor :        Accelerometer ( 1) ; 0.001197 / 0.00s ; MPU6500 Acceleration Sensor / Invensense
10. 03-25 14:51:43.746 12978 12993 D Unity   : Choreographer available: Enabling VSYNC timing
11.  

 

I'll go fix that.

 

Fix in flight.

 

Thank you so much for all these fixes! I'm not a Unity3d expert so I'm not really certain when an issue is my fault or not. I'll integrate your latest fix and will report back asap, probably over the weekend. Again, your help is very much appreciated!

 

OK, I've been trying some more and the NotImplementedException is gone when I use the fix-json-parser branch but it's still not fully functional. I get some odd values in from time to time but I suspect those are just random memory locations. What I do find odd is that my HID report going out is 34 bytes (272 bits) in size, but the incoming AGC events are 244 bits. Is there some sort of buffer limit somwhere? I know my input action scheme should be working because I included a mouse and those events get triggered when I connect one to my Android phone.
I suspect we're very close to getting this to properly work though!

 

244 bits? How do you arrive at that number?

To emphasize, the incoming data has no relation to the HID input report that your device sends. The input report gets lost at the Android OS level. What we see and transmit is data collection from Android MotionEvents. This means that each input event will have a full AndroidGameControllerState record with the axis and button values stored in the respective arrays. At least if the device is surfaced as an Android game controller, that *should* be the outcome...

 

I got this number from the input debugger window in the events list at the bottom. That is the incoming data right? The data is labeled as AGC. When I was debugging on windows, I used that window to see if all data was coming in nicely. Am I on the wrong track here?

 

Hmm... could you post a screenshot?

It sounds quite odd. Just to be extra clear, this is with the input debugger *remotely* connected to the Android player. And then, with the device debugger open on your device (which is reported as a game controller), you get STAT events tagged with the AGC format and a size of 244 (which would be bytes, though)?

This below is the data we send for game controllers on Android and should be what you see coming out on the managed size as a payload in those STAT events.

Code (CSharp):

1.         // See platforms\android-16\arch-arm\usr\include\android\input.h 'Constants that identify each individual axis of a motion event.' enum
2.         const static int kMaxAndroidAxes = 48;
3.         // See platforms\android-16\arch-arm\usr\include\android\keycodes.h  Key codes enum
4.         const static int kMaxAndroidButtons = 220;
5.         const static int kInputAndroidGameControllerState = 'AGC ';
6.  
7.         struct GameControllerState
8.         {
9.             UInt32 buttons[(kMaxAndroidButtons + 31) / 32];
10.             float axes[kMaxAndroidAxes];
11.         };
12.  

 

Indeed, I'm attached to my phone when I see the STAT events tagged with AGC and a size of 244. A screenshot is attached. I hope this helps. I'll continue on my end.

Thanks a lot!

 

Hi,

I have taken over this project but unfortunately I have not yet succeed in going further than my colleagues. Just to be clear, I'm always compiling and uploading to the Android device in question and monitor the inputdebugger over a wifi network.

I have couple of question about what has been said previously in this thread:

1. Why use AxisControl rather than IntegerControl ?
2. How do I get the incoming data from the AxisControl (e.g. batteryLevel value between 0 and 100) ?
3. What does the [InputControl(name = "devicePosition", layout = "Vector3", offset = kAxisOffset + 1)] really do ? I think I understand if fully, could you elaborate a bit further.

Here is the last interface I tried to use:

Code (CSharp):

1. [StructLayout(LayoutKind.Sequential, Size = 4)]
2. unsafe struct HIDDeviceState : IInputStateTypeInfo
3. {
4.     public FourCC format => new FourCC('A', 'G', 'C');
5.  
6.     public const uint kAxisOffset = 0;
7.  
8.     [InputControl(name = "devicePosition", layout = "Vector3", offset = kAxisOffset + 1)] //Vector3
9.     [InputControl(name = "deviceRotation", layout = "Quaternion", offset = kAxisOffset + 13)] //Quaternion
10.     [InputControl(name = "batteryLevel", layout = "Analog", offset = kAxisOffset + 29)] //Analog
11.     [InputControl(name = "trackingState", layout = "Integer", offset = kAxisOffset + 33)] //Integer
12.     public fixed float axis[37]; // 37 -> total bytes send by the device
13. }
14.  
15. [InputControlLayout(stateType = typeof(HIDDeviceState))]
16. #if UNITY_EDITOR
17. [InitializeOnLoad]
18. #endif
19. public class HIDDevice : InputDevice
20. {
21.     public AxisControl batteryLevel { get; private set; }
22.     public IntegerControl trackingState { get; private set; }
23.  
24.     protected override void FinishSetup()
25.     {
26.         base.FinishSetup();
27.         batteryLevel = GetChildControl<AxisControl>("batteryLevel");
28.         trackingState = GetChildControl<IntegerControl>("trackingState");
29.         Debug.Log("batteryLevel: " + batteryLevel.ToString());
30.     }
31.  
32.     static HIDDevice()
33.     {
34.         InputSystem.RegisterLayout<HIDDevice>(
35.             matches: new InputDeviceMatcher()
36.                 .WithInterface("Android")
37.                 .WithCapability("vendorId", 0x4040)
38.                 .WithCapability("productId", 0x4040)
39.             );
40.     }
41.     [RuntimeInitializeOnLoadMethod]
42.     static void Init() { }
43. }

 

Bump thread. Need recreate JoystickButton0 and find this thread. I made some progress but failed to find a suitable InputControlAttribute parameter.

 

@Rene-Damm I have to make a Virtual Computing Network app on android. I have successfully mapped all the keyboard key and they are working on the editor. But on Android some keys of keyboard are not getting call backs but instead android system functionality occurs like when system(windows) key is pressed then google assistant pops up breaking the app. Similarly the mouse input of right mouse button is not working on android as by default android has mapped the back functionality with mouse right key. Is there any solution to it. Looking forward.
Please also integrate GCKeyboard and GCMouse with the input system on IOS and it is introduced in IOS 14