Search Unity

Any good way to find out if a keyboard key is pressed with the mouse over a VisualElement?

Discussion in 'UI Toolkit' started by Baste, Feb 23, 2021.

  1. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    I've got an editor window that shows a bunch of VisualElements in a grid. I want to know if the user presses "b" with the mouse over one of the elements, and if they do, I want to know which VisualElement it is.

    I wanted to add a KeyDownEvent to the VisualElements themselves, but that only fires if they have "focus", which isn't applicable at all here. So my current hack is:

    Code (csharp):
    1. // in the specific visual elements:
    2. RegisterCallback<MouseEnterEvent>(evt => window.mousedOver = this);
    3. RegisterCallback<MouseLeaveEvent>(evt => window.mousedOver = null);
    4.  
    5. // in OnEnable() on the window itself:
    6. if (rootVisualElement.parent == null) // this is null in OnEnable... sometimes
    7.     EditorApplication.delayCall += SubscribeKeyDownEvent;
    8. else
    9.     SubscribeKeyDownEvent();
    10.  
    11. ...
    12.  
    13. private void SubscribeKeyDownEvent() {
    14.     var keyDownReceiver = rootVisualElement;
    15.     while (keyDownReceiver.parent != null) {
    16.         keyDownReceiver = keyDownReceiver.parent;
    17.     }
    18.  
    19.     keyDownReceiver.RegisterCallback<KeyDownEvent>(KeyDown);
    20. }
    21.  
    22. ...
    23.  
    24. private void KeyDown(KeyDownEvent evt) {
    25.     if (evt.keyCode == KeyCode.B && mousedOver != null) {
    26.         HandleBPressedOver(mousedOver);
    27.     }
    28. }
    This is just painful. It will also probably break if stuff starts having focus? idk. There's gotta be a better way to handle this! In imgui I'd check if Event.Current was a keydown event with the b keyCode, and then I'd painstakingly figure out the mouse position vs. the element positions. The "what is the mouse over" thing has become a lot easier, but getting a notif when the key is down only being available for focused things seems like a very silly restriction.
     
    Baste, Feb 23, 2021
    #1

  2. Digika

    Digika

    Joined:
    Jan 7, 2018
    Posts:
    225
    Digika, Feb 23, 2021
    #2

  3. uBenoitA

    uBenoitA

    Unity Technologies

    Joined:
    Apr 15, 2020
    Posts:
    220
    Hi Baste,

    You can use 
    rootVisualElement.panel.visualTree.RegisterCallback<KeyDownEvent>(KeyDown, TrickleDown.TrickleDown);
    to subscribe to the event and make sure 
    KeyDown
    gets called first no matter what child is or is not focused.

    As for 
    rootVisualElement.parent
    being null in OnEnable, if that is the case then you can do your SubscribeKeyDownEvent as a reaction to the AttachedToPanelEvent, instead of using a 
    delayCall
    , which is much more consistent and also won't prevent your project from building if your script is not in an "Editor" folder.
     
    uBenoitA, Feb 23, 2021
    #3
    twistcap likes this.

  4. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    This is an EditorWindow, so I'm not too worried about trying to make it compile in builds :p

    AttachToPanelEvent seems to be called every time I dock the window in a new spot. originPanel is always null and destinationPanel is always a new object. Does the previous panel always get destroyed?

    It seems like the keyDownEvents doesn't get registered if I dock the EditorWindow. If I have the window as a free floating element, they do work.

    Also, the KeyDownEvent is always invoked twice - both times have the same phase (at target), but the second one is missing the keycode.

    Here's a test implementation. My real window uses stylesheets, but here I've just done everything inline:

    Code (csharp):
    1. using UnityEditor;
    2. using UnityEngine;
    3. using UnityEngine.UIElements;
    4.  
    5. public class MyEditorWindow : EditorWindow {
    6.  
    7.     [MenuItem("Test/Window")]
    8.     public static void TestWindow() {
    9.         GetWindow<MyEditorWindow>();
    10.     }
    11.  
    12.     private void OnEnable() {
    13.         if (rootVisualElement.panel == null) {
    14.             rootVisualElement.RegisterCallback<AttachToPanelEvent>(SubscribeKeyDownEvent);
    15.         }
    16.         else
    17.             SubscribeKeyDownEvent(null, rootVisualElement.panel);
    18.  
    19.         var root = new VisualElement();
    20.         rootVisualElement.Add(root);
    21.  
    22.         for (int i = 0; i < 10; i++) {
    23.             var row = new VisualElement();
    24.             row.style.flexDirection = FlexDirection.Row;
    25.             root.Add(row);
    26.  
    27.             for (int j = 0; j < 10; j++) {
    28.                 var cell = new VisualElement();
    29.                 cell.style.minHeight = 16;
    30.                 cell.style.minWidth = 16;
    31.                 cell.style.maxHeight = 16;
    32.                 cell.style.maxWidth = 16;
    33.  
    34.                 cell.style.backgroundColor = Color.HSVToRGB(Random.value, 1f, 1f);
    35.                 row.Add(cell);
    36.             }
    37.         }
    38.     }
    39.  
    40.     private void SubscribeKeyDownEvent(AttachToPanelEvent evt) {
    41.         SubscribeKeyDownEvent(evt.originPanel, evt.destinationPanel);
    42.     }
    43.  
    44.     private void SubscribeKeyDownEvent(IPanel oldPanel, IPanel newPanel) {
    45.         oldPanel?.visualTree.UnregisterCallback<KeyDownEvent>(KeyDown, TrickleDown.TrickleDown);
    46.         newPanel .visualTree.RegisterCallback  <KeyDownEvent>(KeyDown, TrickleDown.TrickleDown);
    47.     }
    48.  
    49.     private void KeyDown(KeyDownEvent evt) {
    50.         Debug.Log($"{evt.propagationPhase}/{evt.keyCode}/{evt.character}/{evt.target}/{(evt.imguiEvent != null ? evt.imguiEvent.type.ToString() : "NULL")}");
    51.  
    52.     }
    53. }
    So bug 1: Hit Test->Window, click a button. Note that you get two Debug.Log's. The first one is WILD:

    upload_2021-2-24_14-0-50.png

    Why is it printing less? Turns out that evt.character is '\0'! Also turns out that that terminates the string! Is that a bug in Debug.Log? We're logging C# strings, those are not null-terminated!

    The actual bug is that there's two invocations of the method, one with the data packed in one way and another one with it packed in another. That's strange and unexpected.

    Bug 2: Dock the window next to the inspector or any other window, give it focus and try to hit any keys. No respone. Undock it again, it gets a response.
     
    Baste, Feb 24, 2021
    #4

  5. uBenoitA

    uBenoitA

    Unity Technologies

    Joined:
    Apr 15, 2020
    Posts:
    220
    Bug 1 is not a bug, it's the way KeyDownEvents work. They are just a copy of GUI Events, which work exactly like that, if you try
    Event e = new Event();
    
void Update()
    
{
    
    while(Event.PopEvent(e))
    
        Debug.Log($"{e.typ}/{e.keyCode}/{e.character}");
    
}

    , you should see similar results. You can ignore the KeyDownEvent if keyCode==None, if you're not interested in the text content of the keyboard event, or ignore the one with character=='\0' if you're looking for text input.

    Bug 2 seems more of a problem. Could you print `rootVisualElement.panel.focusController.focusedElement` to see if there's something that has the focus and that does something strange with it? For EditorWindows, there are a few IMGUIContainers that handle window-related events on the top of the visual tree hierarchy (you can see them in the UI Toolkit Debugger, usually there's 1), which can be receiving events before the rest and use them. If all keyboard events are systematically used by that element, then that's probably a real problem, and we might need to open a bug about it, indeed.
     
    uBenoitA, Feb 24, 2021
    #5

  6. broots

    broots

    Joined:
    Dec 20, 2019
    Posts:
    54
    +1ing on this, I had to switch to using IMGUI events because of this exact problem, keystrokes were sometimes missing/probably consumed by something before UIE triggered them. This seemed especially true in cases where I wanted to react to say "cntrl+s" before the editor, and UIE seemed to never receive the input in time, but IMGUI can consume these events just fine and "snatch" them from the editor.
     
    broots, Feb 25, 2021
    #6

  7. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    After I dock the element, focusedElement is null. Should I send a bug report?
     
    Baste, Feb 25, 2021
    #7

  8. uBenoitA

    uBenoitA

    Unity Technologies

    Joined:
    Apr 15, 2020
    Posts:
    220
    Small correction to what I said earlier: you shouldn't use 
    panel.visualTree.RegisterCallback<KeyDownEvent>
    , but rather use 
    rootVisualElement.RegisterCallback<KeyDownEvent>
    , because the panel.visualTree is shared by all windows docked in the same DockArea, that is, when you have multiple tabs grouped together in the editor, you don't want some inactive tabs to be listening to events targeted at the other active tabs.

    You can use 
    rootVisualElement.focusable = true; rootVisualElement.pickingMode = PickingMode.Position; rootVisualElement.Focus();

    to make sure that the window's background is clickable and that doing that focuses the rootVisualElement.

    You can also react to 
    EditorWindow.OnFocus()
    to make sure rootVisualElement or one of its children reacquires the focus when your window tab is selected. Something like this:

    Code (CSharp):
    1. using UnityEditor;
    2. using UnityEngine;
    3. using UnityEngine.UIElements;
    4.  
    5. public class TestKeyDownWindow : EditorWindow
    6. {
    7.     // Called immediately after OnEnable, once UIDocument's panel has been assigned
    8.     public void CreateGUI()
    9.     {
    10.         rootVisualElement.Add(new Label("Some Label"));
    11.         rootVisualElement.Add(new TextField());
    12.  
    13.         rootVisualElement.RegisterCallback<KeyDownEvent>(OnKeyDown, TrickleDown.TrickleDown);
    14.  
    15.         // Make sure rootVisualElement has the focus when we start.
    16.         rootVisualElement.focusable = true;
    17.         rootVisualElement.pickingMode = PickingMode.Position;
    18.         rootVisualElement.Focus();
    19.     }
    20.  
    21.     private void OnKeyDown(KeyDownEvent evt)
    22.     {
    23.         Debug.Log("UITK event: type=" + evt.GetType() + ", keyCode=" + evt.keyCode + ", character=" +
    24.                   (evt.character == 0 ? "0" : "" + evt.character) + ", modifiers=" + evt.modifiers);
    25.     }
    26.  
    27.     // Called when EditorWindow gets keyboard focus, for example when changing tab. Make sure we reacquire focus.
    28.     public void OnFocus()
    29.     {
    30.         ReacquireFocusOnWindowRootOrChildren();
    31.     }
    32.  
    33.     private void ReacquireFocusOnWindowRootOrChildren()
    34.     {
    35.         var focusedElement = rootVisualElement.focusController.focusedElement as VisualElement;
    36.         if (focusedElement == null || !IsChildOfWindowRoot(focusedElement))
    37.             rootVisualElement.Focus();
    38.     }
    39.  
    40.     private bool IsChildOfWindowRoot(VisualElement ve)
    41.     {
    42.         return ve == rootVisualElement || ve != null && IsChildOfWindowRoot(ve.parent);
    43.     }
    44.  
    45.     [MenuItem("Window/TestKeyDownWindow")]
    46.     static void Open()
    47.     {
    48.         GetWindow<TestKeyDownWindow>().Show();
    49.     }
    50. }

    That being said, there is still a situation where the DockArea of the EditorWindow will receive all keyboard events and stop them from going further down, that is, when you explicitly click on the window tab and nothing else. This is by design, as it's the only way to tell the editor that you want to interact with the docking system itself and not the window.
     
    uBenoitA, Mar 3, 2021
    #8

  9. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    That works!

    Though, in general, it is a bit clunky! Especially since I had to go on the forums to understand what was going on.


    So a follow-up question: this came up because I wanted my UI Toolkit based editor window have keyboard shortcuts. What's the intended way to do that?

    In this case, imagine you're making an MS Paint clone in an editor window using UI Toolkit, and you want pressing "b" to select the bucket fill tool, when the window is focused.

    Is the intended workflow for that to require setting the focusable and pickingMode fields to the correct values and calling Focus()? None of those are in any way intuitive prerequisites to have a callback do something.
     
    Baste, Mar 4, 2021
    #9