Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

[Solved] Virtual mouse cursor that triggers UI events?

Discussion in 'UGUI & TextMesh Pro' started by blueJgrrl, Apr 11, 2016.

  1. blueJgrrl

    blueJgrrl

    Joined:
    Oct 3, 2012
    Posts:
    19
    [Solved: Here's the code to create a virtual mouse cursor that triggers UI events.]

    Is it possible to create a virtual mouse cursor (as a UI element on the canvas) that can trigger UI events such as hover and click? More importantly, a virtual mouse cursor that does not map 1:1 with the screen position of the actual mouse cursor?

    My game has an unusual setup, because I'm recreating the rectangular pixel ratio of old CRT screens, and the UI is in a pixel-perfect 320x200 VGA-esque style. The UI canvas is rendered to a RenderTexture, which is then displayed on a scaled quad to create the correct pixel aspect ratio.

    Here's a screenshot with the canvas in editor view, and the final rendered quad in game view:



    The final result with post-process effects looks authentic:



    At present the fake mouse cursor is an Image on the canvas. I have it moving according to mouse input, but have not been able to figure out how to make it trigger UI events.

    I've been googling this like mad, trying to learn the API for EventSystems, and have not been able to crack this nut. Do I manually raycast from the Image and invoke events via script? If so, I haven't been able to figure out what in the API can actually do that.

    Any help is appreciated, whether it's just pointing me in the right direction, or a more thorough response.
     
    Last edited: Apr 12, 2016
  2. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,400
    ModLunar likes this.
  3. blueJgrrl

    blueJgrrl

    Joined:
    Oct 3, 2012
    Posts:
    19
    ModLunar and karl_jones like this.
  4. blueJgrrl

    blueJgrrl

    Joined:
    Oct 3, 2012
    Posts:
    19
    I have the virtual cursor sending events instead of the mouse cursor. There's a bug right now where hover events flicker on and off. Once I fix that and make sure nothing else is buggy, I'll upload my code for others to use.

    Here's the virtual cursor triggering a hover event on an image button:
     
    karl_jones likes this.
  5. blueJgrrl

    blueJgrrl

    Joined:
    Oct 3, 2012
    Posts:
    19
    The problem was simple: The cursor was still a raycast target. Code works now.

    Here's the modified script, for anyone that wants to have a virtual mouse cursor.
     
  6. Giganten

    Giganten

    Joined:
    Sep 29, 2015
    Posts:
    1
    How can I adapted the code so I get the ui button clicked if I press a button on gamepad instead of the mouse using the virtual cursor?
     
  7. rerwandi

    rerwandi

    Joined:
    Dec 8, 2014
    Posts:
    544
    I used your script, but its only work with Screen Space- Camera render mode canvas. And also its only detect click when the virtual cursor is behind the button. If i set the sorting layer for my virtual cursor in front of the button, it doesnt detect the click.
    Any idea how to fix this ?
     
    jinnindo likes this.
  8. mh114

    mh114

    Joined:
    Nov 17, 2013
    Posts:
    295
    Off-topic; just wanted to say this looks interesting, reminds me a lot of Ultima Underworld which I presume gave some inspiration for the style..? :)
     
    TomTrottel and Chris_Payne_QS like this.
  9. Zimbres

    Zimbres

    Joined:
    Nov 17, 2014
    Posts:
    181
    May I ask about the script? I'm not used with the event system model, and the script supplied is not working for me. I've tested on a clean project, and no luck. The only thing that works, is the Virtual Cursor (from rect transform field). It register the mouse over event on the UI buttons. But it does not move itself.

    I do want to do a dirty little fix, and simply move this rect transform on the SendMoveEventToSelectedObject() method, but I'm not sure if this would be bad usage of the PointerInputModule inheritance.

    Also, bringing back reynolderwandi's question, why the mouse over is only registered when the cursor is behind the button? That's odd.
     
  10. rerwandi

    rerwandi

    Joined:
    Dec 8, 2014
    Posts:
    544
    A modified StandaloneInputModule that allows UI events to be driven by a virtual mouse cursor. Does not handle cursor movement.
    So you are supposed to make your own virtual cursor movement. I already done that by repositioning the virtual cursor based on mouse delay movement.
    And in my case, i finally solved it by ignoring the raycast for my virtual cursor image (because we chose the first object that raycasted)
     
    Last edited: Sep 25, 2016
  11. Zimbres

    Zimbres

    Joined:
    Nov 17, 2014
    Posts:
    181
    Really? I'm feeling very noob right now, hehehe. Good to know!

    Hum.... a last question? how to use the gamepad buttons as "mouse0"/left click?
     
  12. my_little_kafka

    my_little_kafka

    Joined:
    Feb 6, 2014
    Posts:
    92
    This is a very good solution for creating a virtual cursor, worked great on 5.4, but it's broken now on 5.5

    I looked at the source code of 5.5 UI system and couldn't find anything new in the systems - don't know what to do.

    The problem is that virtual cursor doesn't send PointerEnter and PointerExit events to UI objects. It doesn't work with EventTrigger component, and it even doesn't work with standart Button component (it doesn't change the normal color to highlighted color).

    The strangest thing is that this events started to work only when any of the mouse buttons are pressed and then released. I can press any mouse button on with virtual cursor on the any part of the screen, but if I release the button while the cursor is placed on object with PointerEnter even - the event is called then, and standart Button changes its color to a highlighted color.
     
  13. moi6991

    moi6991

    Joined:
    Feb 18, 2016
    Posts:
    2
    Hi, this works lovely for me,im developing a game with kinect, and i need two cursors(one for each hand) how can i use your code and make it work for two virtual cursors instead of one.Thanks
     
    danielmz25 and KUFgoddess like this.
  14. XiongGuiYang

    XiongGuiYang

    Joined:
    Sep 5, 2016
    Posts:
    14
    Button btn_Gm = gmIns.GetComponent<Button>();
    btn_Gm.Invoke();//Inveke onClick Events
     
    Tehelee likes this.
  15. TheHats

    TheHats

    Joined:
    Jul 4, 2013
    Posts:
    2
    Recently used this code to use a controller in my game's menus. As my_little_kafka pointed out, it doesn't seem to work in 5.5. I was not getting proper enter and exit events on the currently raycasted object.

    I did a diff on the 5.5 and 5.2 code and found that in ProcessMouseEvent() they added this under the two variables like so:

    Code (CSharp):
    1.  protected void ProcessMouseEvent(int id) {
    2.             var mouseData = GetMousePointerEventData(id);
    3.             var leftButtonData = mouseData.GetButtonState(PointerEventData.InputButton.Left).eventData;
    4.  
    5.             if (ForceAutoSelect())
    6.                 eventSystem.SetSelectedGameObject(leftButtonData.buttonData.pointerCurrentRaycast.gameObject, leftButtonData.buttonData);
    and when you look at Force AutoSelect() it's just this:

    Code (CSharp):
    1. protected virtual bool ForceAutoSelect() {
    2.             return false;
    3.         }
    So I changed that to TRUE and suddenly the current raycasted target from my controller move events was actually working~ Hooray. One note though, it doesn't seem to parse through raycastable UI objects that don't have buttons or events on them so be extra sure you keep a tight house when it comes to "raycast target"s

    Hope I save someone the pain I just had spending 5 hours to realize I have to flip an easily overlooked bool.
     
    Last edited: Mar 10, 2017
    HomeBearJosh likes this.
  16. cattagames

    cattagames

    Joined:
    Jul 27, 2016
    Posts:
    8
    How did you manage to have the Image object (the virtual cursor) have the same position as the mouse?
     
    tredpro likes this.
  17. HomeBearJosh

    HomeBearJosh

    Joined:
    Dec 13, 2017
    Posts:
    34
    Did you ever figure out how to do this with this script?
    Thanks!
     
  18. FoxCastz

    FoxCastz

    Joined:
    Jun 18, 2015
    Posts:
    75
    I tried the VirtualInputModule and it isn't doing anything. I created a script to move the virtual mouse and nothing interacts with it.
     
  19. blueJgrrl

    blueJgrrl

    Joined:
    Oct 3, 2012
    Posts:
    19
    It's probably no longer compatible with the latest versions of Unity. I haven't touched that code in a very long time. If anyone has the fix for it, feel free.
     
  20. FoxCastz

    FoxCastz

    Joined:
    Jun 18, 2015
    Posts:
    75
    I am just trying to add controller support, to a pc game. This requires some form of cursor moveable via controller that can interact with Unity's UI.
     
    ModLunar likes this.
  21. whatarenames

    whatarenames

    Joined:
    May 6, 2018
    Posts:
    1
    It appears to be semi-compatible? I'm not sure what the exact behavior was in the past but if I set
    Cursor.lockState = CursorLockMode.Locked;
    , it doesn't trigger certain pointer events. The click event still works but
    IPointerExitHandler
    and
    IPointerEnterHandler
    don't trigger (I haven't tested others). I have to set the mouse state to Confined to get them to work but in-editor but Confined doesn't function in editor, only builds.
    I would very much like to be able to work this with a Locked cursor but unable to find if there's a way to make it happen.
     
    MaxPirat likes this.
  22. XBlay

    XBlay

    Joined:
    Jul 6, 2015
    Posts:
    1
    This solution worked for me: https://answers.unity.com/questions/1262549/cursorlockstate-and-onpointerenter-not-working-tog.html


    Code (CSharp):
    1. private CursorLockMode _currentLockState = CursorLockMode.None;
    2.  
    3. public override void Process()
    4.         {
    5.             _currentLockState = Cursor.lockState;
    6.             Cursor.lockState = CursorLockMode.None;
    7.             bool usedEvent = SendUpdateEventToSelectedObject();
    8.  
    9.             if (eventSystem.sendNavigationEvents)
    10.             {
    11.                 if (!usedEvent)
    12.                     usedEvent |= SendMoveEventToSelectedObject();
    13.  
    14.                 if (!usedEvent)
    15.                     SendSubmitEventToSelectedObject();
    16.             }
    17.  
    18.             ProcessMouseEvent();
    19.             Cursor.lockState = _currentLockState;
    20.         }
     
    kostas-vs and MaxPirat like this.
  23. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    so now unity 2020.2.1. how to make virtual mouse cursor and make it work now?
     
  24. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    Me too. How to make it?
     
  25. This sample is working for me properly with PS4 wireless controller on Windows. It even shows how to move a soft cursor around.

    result.png
     
  26. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    @Lurking-Ninja Thx for reply, is this unity built-in package? can I just import it through package window?
     
  27. Hi, yes, of course. It's a sample for the new input system. It's not a complete solution, but it's something to give you idea and a base to build your own on the top of it.
     
  28. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    So I imported this package, then what? I run the game and connect nintendo switch pro controller to pc, but mouse cursor does not move by controller's L stick.
     
  29. leandrogonzaga

    leandrogonzaga

    Joined:
    Feb 26, 2021
    Posts:
    1
    You need to set up the 'Cursor Graphic' and 'Cursor Transform' fields on the 'Virtual Mouse' component. Also, in order to work properly, you need to keep the Canvas Resize as constant size, i guess.

    See the 'README' on the 'Gamepad Mouse Cursor' folder for more info
     
  30. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    Where is Virtual Mouse component?
    Where is [Input] package initially? There is none.
    Where is Gamepad Mouser Cursor folder?
     
  31. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    Eventually, How to mimic mouse over state of window system's real mouse?
     
  32. leegod

    leegod

    Joined:
    May 5, 2010
    Posts:
    2,488
    I just want some 2d cursor texture moving by controller's Lstick and when it is over some UI element, calling mouse over state, but Virtual Mouse component is too complicated and over weighted, and mostly, there seems no Mouse Over mimic function in there.
     
  33. Rickmc3280

    Rickmc3280

    Joined:
    Jun 28, 2014
    Posts:
    192
    I am interested in making this work with the new input system as well. I was given advice for the old input system and have been unable to wrap my mind around it (after I tried their advice and it failed because I have both old and new input systems enabled and am reliant on the new one - thus have that script in the scene). I have another script that creates the Vector2 coordinates and I need to send those to..(???) something to simulate a mouse.

    In this example I have a trigger enter into a box with a collider. OnTriggerStay is called and updates the position of the cursor... This Box is sized the same size as the canvas and I am able to match the coordinates of the screen, similar to a mouse ( I compared it to the input from the mouse using Input System and they had the same ranges). So instead of doing all of the raycasting, since I have the coordinates, how do I send it to the canvas/event system?

    Edit: I am going to try all the steps mentioned above, again, but initially I did not get this working.
     
  34. Neonage

    Neonage

    Joined:
    May 22, 2020
    Posts:
    288
    We can simply override BaseInput with custom class. Works perfectly

    Code (CSharp):
    1.  
    2. using UnityEngine;
    3. using UnityEngine.EventSystems;
    4.  
    5. public class BaseInputCustom : BaseInput
    6. {
    7.     public override Vector2 mousePosition => GameCursor.position;
    8.  
    9.     protected override void OnEnable()
    10.     {
    11.         base.OnEnable();
    12.         GetComponent<StandaloneInputModule>().inputOverride = this;
    13.     }
    14.  
    15.     public override float GetAxisRaw(string axisName)
    16.     {
    17.         if (axisName == "Horizontal") return InputBuffer.x.dpad.x + InputBuffer.x.move.x;
    18.         if (axisName == "Vertical") return InputBuffer.x.dpad.y + InputBuffer.x.move.y;
    19.         return base.GetAxisRaw(axisName);
    20.     }
    21.  
    22.     public override bool GetMouseButtonDown(int button)
    23.     {
    24.         if (button == 0)
    25.             return InputBuffer.x.leftClick.isDown;
    26.         return base.GetMouseButtonDown(button);
    27.     }
    28.     public override bool GetMouseButtonUp(int button)
    29.     {
    30.         if (button == 0)
    31.             return InputBuffer.x.leftClick.isUp;
    32.         return base.GetMouseButtonUp(button);
    33.     }
    34. }
    35.  
     
    EZaca, BunceoNoicel2 and zezba9000 like this.