[Solved] Virtual mouse cursor that triggers UI events?

[Solved: Here's the code to create a virtual mouse cursor that triggers UI events.]

Is it possible to create a virtual mouse cursor (as a UI element on the canvas) that can trigger UI events such as hover and click? More importantly, a virtual mouse cursor that does not map 1:1 with the screen position of the actual mouse cursor?

My game has an unusual setup, because I'm recreating the rectangular pixel ratio of old CRT screens, and the UI is in a pixel-perfect 320x200 VGA-esque style. The UI canvas is rendered to a RenderTexture, which is then displayed on a scaled quad to create the correct pixel aspect ratio.

Here's a screenshot with the canvas in editor view, and the final rendered quad in game view:



The final result with post-process effects looks authentic:



At present the fake mouse cursor is an Image on the canvas. I have it moving according to mouse input, but have not been able to figure out how to make it trigger UI events.

I've been googling this like mad, trying to learn the API for EventSystems, and have not been able to crack this nut. Do I manually raycast from the Image and invoke events via script? If so, I haven't been able to figure out what in the API can actually do that.

Any help is appreciated, whether it's just pointing me in the right direction, or a more thorough response.

 

Awesome, this should help! Thank you. I'll report back when I get some results.

 

I have the virtual cursor sending events instead of the mouse cursor. There's a bug right now where hover events flicker on and off. Once I fix that and make sure nothing else is buggy, I'll upload my code for others to use.

Here's the virtual cursor triggering a hover event on an image button:

 

The problem was simple: The cursor was still a raycast target. Code works now.

Here's the modified script, for anyone that wants to have a virtual mouse cursor.

 

How can I adapted the code so I get the ui button clicked if I press a button on gamepad instead of the mouse using the virtual cursor?

 

I used your script, but its only work with Screen Space- Camera render mode canvas. And also its only detect click when the virtual cursor is behind the button. If i set the sorting layer for my virtual cursor in front of the button, it doesnt detect the click.
Any idea how to fix this ?

 

Off-topic; just wanted to say this looks interesting, reminds me a lot of Ultima Underworld which I presume gave some inspiration for the style..?

 

May I ask about the script? I'm not used with the event system model, and the script supplied is not working for me. I've tested on a clean project, and no luck. The only thing that works, is the Virtual Cursor (from rect transform field). It register the mouse over event on the UI buttons. But it does not move itself.

I do want to do a dirty little fix, and simply move this rect transform on the SendMoveEventToSelectedObject() method, but I'm not sure if this would be bad usage of the PointerInputModule inheritance.

Also, bringing back reynolderwandi's question, why the mouse over is only registered when the cursor is behind the button? That's odd.

 

A modified StandaloneInputModule that allows UI events to be driven by a virtual mouse cursor. Does not handle cursor movement.
So you are supposed to make your own virtual cursor movement. I already done that by repositioning the virtual cursor based on mouse delay movement.
And in my case, i finally solved it by ignoring the raycast for my virtual cursor image (because we chose the first object that raycasted)

 

Really? I'm feeling very noob right now, hehehe. Good to know!

Hum.... a last question? how to use the gamepad buttons as "mouse0"/left click?

 

This is a very good solution for creating a virtual cursor, worked great on 5.4, but it's broken now on 5.5

I looked at the source code of 5.5 UI system and couldn't find anything new in the systems - don't know what to do.

The problem is that virtual cursor doesn't send PointerEnter and PointerExit events to UI objects. It doesn't work with EventTrigger component, and it even doesn't work with standart Button component (it doesn't change the normal color to highlighted color).

The strangest thing is that this events started to work only when any of the mouse buttons are pressed and then released. I can press any mouse button on with virtual cursor on the any part of the screen, but if I release the button while the cursor is placed on object with PointerEnter even - the event is called then, and standart Button changes its color to a highlighted color.

 

Hi, this works lovely for me,im developing a game with kinect, and i need two cursors(one for each hand) how can i use your code and make it work for two virtual cursors instead of one.Thanks

 

Button btn_Gm = gmIns.GetComponent<Button>();
btn_Gm.Invoke();//Inveke onClick Events

 

Recently used this code to use a controller in my game's menus. As my_little_kafka pointed out, it doesn't seem to work in 5.5. I was not getting proper enter and exit events on the currently raycasted object.

I did a diff on the 5.5 and 5.2 code and found that in ProcessMouseEvent() they added this under the two variables like so:

Code (CSharp):

1.  protected void ProcessMouseEvent(int id) {
2.             var mouseData = GetMousePointerEventData(id);
3.             var leftButtonData = mouseData.GetButtonState(PointerEventData.InputButton.Left).eventData;
4.  
5.             if (ForceAutoSelect())
6.                 eventSystem.SetSelectedGameObject(leftButtonData.buttonData.pointerCurrentRaycast.gameObject, leftButtonData.buttonData);

and when you look at Force AutoSelect() it's just this:

Code (CSharp):

1. protected virtual bool ForceAutoSelect() {
2.             return false;
3.         }

So I changed that to TRUE and suddenly the current raycasted target from my controller move events was actually working~ Hooray. One note though, it doesn't seem to parse through raycastable UI objects that don't have buttons or events on them so be extra sure you keep a tight house when it comes to "raycast target"s

Hope I save someone the pain I just had spending 5 hours to realize I have to flip an easily overlooked bool.

 

How did you manage to have the Image object (the virtual cursor) have the same position as the mouse?

 

Did you ever figure out how to do this with this script?
Thanks!

 

I tried the VirtualInputModule and it isn't doing anything. I created a script to move the virtual mouse and nothing interacts with it.

 

It's probably no longer compatible with the latest versions of Unity. I haven't touched that code in a very long time. If anyone has the fix for it, feel free.

 

I am just trying to add controller support, to a pc game. This requires some form of cursor moveable via controller that can interact with Unity's UI.

 

It appears to be semi-compatible? I'm not sure what the exact behavior was in the past but if I set

Cursor.lockState = CursorLockMode.Locked;

, it doesn't trigger certain pointer events. The click event still works but

IPointerExitHandler

and

IPointerEnterHandler

don't trigger (I haven't tested others). I have to set the mouse state to Confined to get them to work but in-editor but Confined doesn't function in editor, only builds.
I would very much like to be able to work this with a Locked cursor but unable to find if there's a way to make it happen.

 

This solution worked for me: https://answers.unity.com/questions/1262549/cursorlockstate-and-onpointerenter-not-working-tog.html

Code (CSharp):

1. private CursorLockMode _currentLockState = CursorLockMode.None;
2.  
3. public override void Process()
4.         {
5.             _currentLockState = Cursor.lockState;
6.             Cursor.lockState = CursorLockMode.None;
7.             bool usedEvent = SendUpdateEventToSelectedObject();
8.  
9.             if (eventSystem.sendNavigationEvents)
10.             {
11.                 if (!usedEvent)
12.                     usedEvent |= SendMoveEventToSelectedObject();
13.  
14.                 if (!usedEvent)
15.                     SendSubmitEventToSelectedObject();
16.             }
17.  
18.             ProcessMouseEvent();
19.             Cursor.lockState = _currentLockState;
20.         }

 

so now unity 2020.2.1. how to make virtual mouse cursor and make it work now?

 

Me too. How to make it?

 

This sample is working for me properly with PS4 wireless controller on Windows. It even shows how to move a soft cursor around.

 

@Lurking-Ninja Thx for reply, is this unity built-in package? can I just import it through package window?

 

Hi, yes, of course. It's a sample for the new input system. It's not a complete solution, but it's something to give you idea and a base to build your own on the top of it.

 

So I imported this package, then what? I run the game and connect nintendo switch pro controller to pc, but mouse cursor does not move by controller's L stick.

 

You need to set up the 'Cursor Graphic' and 'Cursor Transform' fields on the 'Virtual Mouse' component. Also, in order to work properly, you need to keep the Canvas Resize as constant size, i guess.

See the 'README' on the 'Gamepad Mouse Cursor' folder for more info

 

Where is Virtual Mouse component?
Where is [Input] package initially? There is none.
Where is Gamepad Mouser Cursor folder?

 

Eventually, How to mimic mouse over state of window system's real mouse?

 

I just want some 2d cursor texture moving by controller's Lstick and when it is over some UI element, calling mouse over state, but Virtual Mouse component is too complicated and over weighted, and mostly, there seems no Mouse Over mimic function in there.

 

I am interested in making this work with the new input system as well. I was given advice for the old input system and have been unable to wrap my mind around it (after I tried their advice and it failed because I have both old and new input systems enabled and am reliant on the new one - thus have that script in the scene). I have another script that creates the Vector2 coordinates and I need to send those to..(???) something to simulate a mouse.

In this example I have a trigger enter into a box with a collider. OnTriggerStay is called and updates the position of the cursor... This Box is sized the same size as the canvas and I am able to match the coordinates of the screen, similar to a mouse ( I compared it to the input from the mouse using Input System and they had the same ranges). So instead of doing all of the raycasting, since I have the coordinates, how do I send it to the canvas/event system?

Edit: I am going to try all the steps mentioned above, again, but initially I did not get this working.

 

We can simply override BaseInput with custom class. Works perfectly

Code (CSharp):

1.  
2. using UnityEngine;
3. using UnityEngine.EventSystems;
4.  
5. public class BaseInputCustom : BaseInput
6. {
7.     public override Vector2 mousePosition => GameCursor.position;
8.  
9.     protected override void OnEnable()
10.     {
11.         base.OnEnable();
12.         GetComponent<StandaloneInputModule>().inputOverride = this;
13.     }
14.  
15.     public override float GetAxisRaw(string axisName)
16.     {
17.         if (axisName == "Horizontal") return InputBuffer.x.dpad.x + InputBuffer.x.move.x;
18.         if (axisName == "Vertical") return InputBuffer.x.dpad.y + InputBuffer.x.move.y;
19.         return base.GetAxisRaw(axisName);
20.     }
21.  
22.     public override bool GetMouseButtonDown(int button)
23.     {
24.         if (button == 0)
25.             return InputBuffer.x.leftClick.isDown;
26.         return base.GetMouseButtonDown(button);
27.     }
28.     public override bool GetMouseButtonUp(int button)
29.     {
30.         if (button == 0)
31.             return InputBuffer.x.leftClick.isUp;
32.         return base.GetMouseButtonUp(button);
33.     }
34. }
35.