Question XR Hands - how to listen for Menu Button?

Working with XR Hands - integration has been nice & smooth so far, but I can't find any way to listen for the Oculus Menu Button on the left hand. The Oculus button on the right hand works properly. Currently my implementation uses action-based input and listens for <XRController>/menuButton. I can't find an input that works for XR Hands. I'm thinking I may be missing a configuration step, but I'm not sure which one. I've also tried

hand.TryGetFeatureValue(UnityEngine.XR.CommonUsages.menuButton, out var menu)

but this only ever returns false.

Is there an input action that works or is there some other mechanism to pick this up?

* Edited to add:

I should note that I am currently testing in the editor using Link. I haven't tested in headset yet.

 

I don't know the answer to your question, but I wonder if your left hand menu button is bound to oculus home and is being intercepted by the headset. What happens when you try swapping the pause and home button in your headset's settings?

UPDATE: I guess that wouldn't rule out if your headset was intercepting the input, since you still don't know what the proper input event should be anyway. Hmm...

 

I don't think this is what's happening--the Home button is working fine (takes me to the Oculus dash) and I can see the Menu button animation is activated on click.

 

I think you should take a deep dive into your controller bindings settings via the current XRInteractions file you have set to the left / right controller. As jasonboukheir3 said it might be a conflict between what is actually bound up and what you want to use as if I have understood correctly that you are setting via code.

 

The question is: which input binding is *expected* to deliver this event using the XR Hands package? All XR controllers I've tested use <XRController>/menuButton, but this does not work for XR Hands + Oculus. I can't find one that works in this configuration. The code snippet above is simply an additional attempt I made to determine if the user activated the menu, which also failed.

 

To clarify the left hand/right hand distinction of the original post--if you're not familiar with Oculus hand tracking, it renders a menu button on your left hand and the Oculus Home button on the right. This matches the menu button on the left controller and home button on the right controller. But the normal mapping for the controller menu button (<XRController>/menuButton) does not work with the hands integration.

 

I would need to look into your scripts and controller settings to troubleshoot this to give you a good answer. There's not much I can do without more insight in your setup. I am currently working on a VR project myself with support for the Quest 2, might bump into the same issue if I am lucky!

 

It's not related to my setup, the problem is that the mapping is not documented in the first place. It turns out the answer is that you need to monitor <MetaAimHand>/aimFlags and check for flag 0x100. The details are here:

https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_FB_hand_tracking_aim

Now just to figure out if I can map this to a button input...

 

I think I stumbled upon something that may help you. As I said earlier, I do not know what your settings is and you still won't post it, so I am working blind here to assist. Depending on the XR plugins you use, the controller setup is different and also has to be coded/wired different.
Just try setting every input to the most specific you can get eg. XR Controller -> Oculus Touch Controller -> Oculus Touch Controller Left Hand -> [action] etc. Hope it helps somehow

 

I posted the answer above. Again the question wasn't about my settings or any configuration on my side. The question was about the (lack of) documentation for the XR Hands package and which event is raised when the user clicks the "menu" or "system" button on the left hand for the Meta Aim Hand controller. If you're not familiar with the XR Hands package, please install and test it out to see what I mean.

 

In case this is helpful, I created a composite input that takes an Integer + a Mask and turns it into a float that can be used as a button input:

Code (CSharp):

1. using UnityEditor;
2. using UnityEngine;
3. using UnityEngine.InputSystem;
4. using UnityEngine.InputSystem.Utilities;
5. #if UNITY_EDITOR
6. [InitializeOnLoad] // Automatically register in editor.
7. #endif
8. [DisplayStringFormat("{Input}+{Mask}")]
9. public class IntFlagsButtonComposite : InputBindingComposite<float>
10. {
11.     [UnityEngine.InputSystem.Layouts.InputControl(layout = "Integer")]
12.     public int Input;
13.  
14.     //[InputControl(layout = "Button")]
15.     public int Mask;
16.    
17.     public override float ReadValue(ref InputBindingCompositeContext context)
18.     {
19.         var val = context.ReadValue<int>(Input);
20.  
21.         return (val & Mask) == Mask ? 1 : 0;
22.     }
23.  
24.     public override float EvaluateMagnitude(ref InputBindingCompositeContext context)
25.     {
26.         return ReadValue(ref context);
27.     }
28.  
29.     static IntFlagsButtonComposite()
30.     {
31.         // Can give custom name or use default (type name with "Composite" clipped off).
32.         // Same composite can be registered multiple times with different names to introduce
33.         // aliases.
34.         //
35.         // NOTE: Registering from the static constructor using InitializeOnLoad and
36.         //       RuntimeInitializeOnLoadMethod is only one way. You can register the
37.         //       composite from wherever it works best for you. Note, however, that
38.         //       the registration has to take place before the composite is first used
39.         //       in a binding. Also, for the composite to show in the editor, it has
40.         //       to be registered from code that runs in edit mode.
41.         InputSystem.RegisterBindingComposite<IntFlagsButtonComposite>();
42.     }
43.  
44.     [RuntimeInitializeOnLoadMethod]
45.     static void Init() { }
46. }
47.  

Then in my Input Actions I set up something like this along with the other inputs for Menu:



Not sure if this composite will be helpful outside of this use case at all, but here we are.

 

This looks good to me, as you are saying the fault may be somewhere else then.

 

Same issue here. This should be way easier to do and better documented.

 

I followed this example but no luck for me. The event never triggers. Is there possibly some pre-requisite setup I'm missing? I'm using MRTK v3 in Unity 2021.3 LTS. MRTK uses the Mixed Reality OpenXR Plugin, which seems to wrap the Unity OpenXR Plugin v1.6.

 

This was an ugly hack and I'm not even sure it still works, but one thing to check is to make sure "Meta Hand Tracking Aim" is selected in your OpenXR Feature Groups:



That said, this project wasn't using MRTK at all, definitely possible those mappings confuse things.

 

Ah, I'm sure this is what I'm missing, unfortunately I can't select it that way because the MRTK OpenXR wrapper is what controls that UI there and it doesn't expose that feature. I'll have to see if maybe there's some other way to turn that on, and if I do turn it on, I'll have to see if it then breaks MRTK.

For anyone thinking about using MRTK, just.. don't... I have regrets.

 

After re-visiting this, it wasn't MRTK that was the issue. The reason I couldn't see these hand tracking OpenXR features is besides the OpenXR package being added to the project, the XR Hands (com.unity.xr.hands) package also needs to be added. This package doesn't even show up in the package manager and I had to add it by name.

 

Unfortunately it doesn't work on XRHands 1.3 and 1,4 pre.

It shouldn't be that hard, but I finally find a way to read the left menu button on the XR Hand.

with the latest version of XR Hands (1.3 and 1.4 pre) menu button on left hand doesn't work. I tried all available action properties.

so what is the answer?

1. Create a new Action e.g Menu on Left Hand, set the Action Type to Value and Control Type to Integer
2. Add a Binding to it and set it to aimFlags [LeftHand Meta Aim Hand] and check the Generic XR controller check box
3. Don't forget to click on Save Asset button

in the code read value of the Menu action that you create in the first step like below:

Code (CSharp):

1. if ( handMenuButton.action.ReadValue<int>()==261 )
2. {
3.         // left hand menu button triggered
4. }

 

Any other suggestions? I can't get any of these working.

 

Thanks! This worked for me. One change I'd suggest is to do a bit check, rather than comparing to a constant:

Code (CSharp):

1. if ((_metaLeftHandMenuInputActionReference.action.ReadValue<int>() & (int)MetaAimFlags.MenuPressed) != 0)