Search Unity

Bug WarpCursorPosition working?

Discussion in 'Linux' started by Enzi, Jan 15, 2024.

  1. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    958
    Is
    Code (CSharp):
    1. Mouse.current.WarpCursorPosition(...);
    working in Linux?
    Not working here within editor, Ubuntu 22.04 and Unity 2023.3.b2

    Doc says, it should work on all platforms.

    edit: Turns out it does work but only for X11 and not Wayland.
    Please fix :)
     
    Last edited: Jan 15, 2024
  2. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    958
  3. karliss_coldwild

    karliss_coldwild

    Joined:
    Oct 1, 2020
    Posts:
    599
    Cursor locking doesn't have to be implemented using WarpPosition. Having a cursor locked or confined to a window are common operations so they have a dedicated API.
     
    tylerinthezoo likes this.
  4. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    958
    You say common but that's exactly the question. Whatever it is using, it still has to make a call to the window manager that is running. X11 or Wayland.
    With Cursor.lockState this call is successful and with WarpPosition it isn't. Both are setting a position, so why do they differ?
     
  5. karliss_coldwild

    karliss_coldwild

    Joined:
    Oct 1, 2020
    Posts:
    599
    Setting mouse position and confining or freezing it is not the same.

    Cursor.lockState only needs to confine the cursor so that you can't click outside the window. As sideeffect of having the cursor being locked after it's unlocked it might reappear somewhere else. But it doesn't mean the API is suitable for freely setting the position.

    If you take a look at wayland documentation for cursor freezing/confining protocol https://wayland.app/protocols/pointer-constraints-unstable-v1 it provides optional hint where to restore the cursor after unfreezing. But that's only a hint and the desktop environment is in control whether to use it and when to use it. Could you emulate cursor warping by abusing the restore position hint - probably. Would it work robustly and reliably - most likely not.

    Unity is probably not communicating with wayland directly. There were some talks on them working on native wayland support, but I don't think that's available unless you are using latest experimental unity version. LTS like 2021 is still using X11 api and xwayland. Which exact API Unity is using to implement the cursor lock mode I don't know. You will need to pay Unity undisclosed amount for the source code access, or reverse engineer Unity to be sure. But since cursor lock mode works, but the warpCursor doesn't I am quite confident that X11 provides some kind of API which allows constraining cursors without arbitrary setting it's position and thus supported by xwayland.

    Here is one test project I did in Unity.

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.InputSystem;
    5.  
    6. public class CursorTest : MonoBehaviour
    7. {
    8.     // Start is called before the first frame update
    9.     bool lockCursor = false;
    10.     bool warpCursor = false;
    11.  
    12.     public GameObject lockedIndicator;
    13.     public RectTransform t1;
    14.     public RectTransform t2;
    15.     public UnityEngine.UI.Text text;
    16.  
    17.     void Start()
    18.     {
    19.  
    20.     }
    21.  
    22.     // Update is called once per frame
    23.     void Update()
    24.     {
    25.         if (Input.GetKeyDown(KeyCode.A))
    26.         {
    27.             lockCursor = !lockCursor;
    28.             Cursor.lockState = lockCursor ? CursorLockMode.Locked : CursorLockMode.None;
    29.             lockedIndicator.SetActive(lockCursor);
    30.         }
    31.  
    32.         if (Input.GetKeyDown(KeyCode.S))
    33.         {
    34.             warpCursor = !warpCursor;
    35.         }
    36.         if (warpCursor)
    37.         {
    38.             Mouse.current.WarpCursorPosition(new Vector2(Screen.width / 2, Screen.height / 2));
    39.         }
    40.  
    41.         t1.position = Mouse.current.position.ReadValue();
    42.         var scr = new Vector3(Screen.width * 0.5f, Screen.height * 0.5f, 0);
    43.         var mouseRel = (Vector3)Mouse.current.delta.ReadValue();
    44.         t2.position = scr + mouseRel;
    45.         string tx = $"pos: {t1.position.x} {t1.position.y} | delta: x:{mouseRel.x} y:{mouseRel.y}";
    46.         if (lockCursor)
    47.         {
    48.             tx += " | lock";
    49.         }
    50.         if (warpCursor)
    51.         {
    52.             tx += " | warp";
    53.         }
    54.         text.text = tx;
    55.     }
    56. }
    Which allows you to play with the cursor lock and cursor warp functionality. What I observed was that:
    * on Wayland while the cursor lock was active the reported absolute position was fixed roughly in the center of window and after disabling it cursor was in the center. That doesn't necesarily mean unity can freely change the position. As expected warping doesn't work.
    * On x11 locked mode behavior was similar. Enabling warping demonstrated that simulating cursor lock by each frame warping it to center of window isn't a good solution, if you move the mouse slowly it's mostly fine. But if you move it quickly it's posible to move and click outside the window, which is undesirable for a game.
    * On windows results were mostly simular. One interesting difference was that on Windows in the cursor locked mode, the marker t1 slightly moved in the direction I was moving the mouse by a few pixels. The movement was significantly smaller than when simulating lock by warping, but it was there.


    One thing you can easily look at is how open source libraries handle this situation. SDL https://www.libsdl.org/ is a commonly used crossplatform library used for stuff like window creation and input. From the Unity changelogs it seems like under the hood Unity is also using SDL for some stuff on Linux although it's unclear how much and in which situations. Since the observed behavior is different I guess this is one of the situations where they have Unity has their own implementation. Anyway SDL is a good example how these functions could be implemented using the APIs provided by system.


    Code (Cpp):
    1. #include <SDL.h>
    2. #include <SDL_events.h>
    3. #include <SDL_keycode.h>
    4. #include <SDL_mouse.h>
    5. #include <SDL_stdinc.h>
    6.  
    7. #include <format>
    8. #include <iostream>
    9.  
    10. int main() {
    11.  
    12. //SDL_HINT_MOUSE_RELATIVE_MODE_WARP
    13.   SDL_Window *SDLWindow{nullptr};
    14.   SDL_Surface *SDLWindowSurface{nullptr};
    15.   SDL_Init(SDL_INIT_VIDEO);
    16.  
    17.   SDLWindow = SDL_CreateWindow("Hello Window", 0, 0, 500, 300, 0);
    18.   SDLWindowSurface = SDL_GetWindowSurface(SDLWindow);
    19.  
    20.   SDL_FillRect(SDLWindowSurface, nullptr,
    21.                SDL_MapRGB(SDLWindowSurface->format, 40, 40, 40));
    22.   SDL_Event event;
    23.   bool exit = false;
    24.   bool lock = false;
    25.   bool warp = false;
    26.   SDL_MouseMotionEvent mouseEvent;
    27.   while (!exit) {
    28.     bool hadMouse = false;
    29.     while (SDL_PollEvent(&event)) {
    30.       switch (event.type) {
    31.       case SDL_QUIT:
    32.         exit = true;
    33.         break;
    34.       case SDL_KEYDOWN:
    35.         switch (event.key.keysym.sym) {
    36.         case SDLK_a:
    37.           lock = !lock;
    38.           SDL_SetRelativeMouseMode(lock ? SDL_TRUE : SDL_FALSE);
    39.           break;
    40.         }
    41.         break;
    42.       case SDL_MOUSEMOTION:
    43.         hadMouse = true;
    44.         mouseEvent = event.motion;
    45.         break;
    46.       }
    47.     }
    48.     std::cout << std::format("mouse: {} {} | rel: {} {} | lock:{} warp:{}",
    49.                              mouseEvent.x, mouseEvent.y, mouseEvent.xrel,
    50.                              mouseEvent.yrel, lock, warp)
    51.               << std::endl;
    52.     SDL_UpdateWindowSurface(SDLWindow);
    53.   }
    54.   SDL_Quit();
    55. }
    With SDL behavior is a bit different. When relative cursor mode is activated both on X11 and Wayland SDL hides the cursor and confines it to window. Reported mouse position changed within the dimensions of window, but the relative movement was correct even if you pushed it against corner. But you aren't really supposed to look at the absolute mouse position while the relative mode is active and you can't interact with anthing anyway, so it doesn't matter too much whether cursor is fully frozen or whether it can move within bounds of windows as long as relative movement is correct.

    If you look at the SDL source code for Wayland support you will find that the code matches with practical observation. WarpMouse is for the most part unsupported https://github.com/libsdl-org/SDL/b...791/src/video/wayland/SDL_waylandmouse.c#L545 .

    And SetRelative locks the cursor https://github.com/libsdl-org/SDL/b...791/src/video/wayland/SDL_waylandmouse.c#L577 and if you dig deeper you will find that it uses the methods from the paged I linked before https://wayland.app/protocols/point...onstraints_v1:request:lock_pointer:arg:region

    Looking at X11 implementation I am not quite sure how it works. X11_SetRelativeMouseMode either returns an error or does nothing claiming to have done the thing. The observed behavior is similar to as on Wayland, but the code for doing it isn't there https://github.com/libsdl-org/SDL/b...59ee84c172b/src/video/x11/SDL_x11mouse.c#L388 Maybe it's happening somewhere else based on a flag.

    Seems like SDL does have a fallback mode https://github.com/libsdl-org/SDL/b...bd6a0b3abc56a791/src/events/SDL_mouse.c#L1139 which implements cursor locking by warping it to center of screen, but it should only activate on platforms that don't have native cursor locking support. In theory there is flag to force it, but I failed to activate it and test how well it works compared to the naive version I made in Unity. Maybe it works slightly better if lower level engine code does it at optimal moment, compared to doing it somewhere in Update.
     
    Ares2048 and Enzi like this.
  6. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    958
    Excellent post! Thanks!
     
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,670
    FWIW we use gdk_display_warp_pointer in the Linux Editor and SDL_WarpMouseInWindow in the Linux player (which ends up in XWarpPointer).
     
    tylerinthezoo likes this.