Input.Mouseposition is queued?

Hello,

I am trying to track down why my code is lagging so far behind the hardware cursor.

I have vsync off, I am hitting 60 fps, I am updating in LateUpdate, and my position is way off.

60 fps capture

2k fps capture

You can see the 2k version is great. The 60 should be good as well, but is way off.

The 60 fps recording isn't great. It doesn't do a fantastic job of showing the lag.

However, it behaves as though Input.Mouseposition has a queue of snapshots to process. It's very much behind. I can move, stop, wait, and watch the cube catch up. Even when the game is running at 60 fps.

Any help would be appreciated. This feels incredibly bad from a player perspective and is hurting my game's input.

Thank you

 

It would help if you actually showed some code then

 

Apologies, I didn't even think about that, because this test case is so basic. But, just in case, here is the code:

Code (CSharp):

1.  
2. private void LateUpdate()
3. {
4.         // ui
5.         //this.transform.position = Input.mousePosition;
6.  
7.         // world
8.         this.transform.position = Camera.main.ScreenToWorldPoint(Input.mousePosition);
9. }
10.  

I have tried 2 versions of this. "Ui" was making a Ui.Image track the cursor, and "world" is making a cube track the cursor. Both had the exact same results.

I probably should have worded my original post differently. I really don't think this is because of my code. I made a completely standalone test case for this, and reproed it with the most basic of setups.

Thanks

 

A UI cursor is tied to the game loop so it will lag. This is why games have a "hardware cursor" option.

 

If you run with a very low frame rate (e.g. 1) then it looks like it is always exactly 1 frame behind. I'm guessing that Unity is recording the mouse position at "the beginning" of a frame and then using that value for all calculations within the frame, but that if you cap the framerate, then "the beginning of the frame" actually means "the end of the previous frame" (i.e. before the wait rather than after it). Seems like maybe there's space for Unity to improve that, but I don't have enough technical knowledge in this area to say for sure.

If you track position on an event like OnPointerDown instead of every frame, it seems to have the "opposite problem" and takes the mouse position of the next frame (which makes sense, because that's the first frame in which you'd know at the start of the frame that the event has occurred).

You could try to "predict" the mouse position by extrapolating from the previous several frames to guess where it will be by the time the current frame actually gets drawn. (Obviously, this will only work if the user moves the mouse in a "smooth" way.)

You could also try to bypass Unity and get the cursor position in some C# way. I've never done that and I'm not sure what the options are or how cross-platform they would be.

 

I don't understand how this is an answer to my question. I am trying to figure out why there is so much lag. My tests show considerable lag with vsync off and hitting 60 fps.

 

@Antistone Have you experienced similar issues? Can you elaborate more on the OnPointerDown findings? Thank you

 

Like the name implies hardware cursors are hardware accelerated. Your operating system takes the position of the mouse and forwards it to the graphics card which overlays it on top of everything else. Hardware cursors will be moved as fast as Windows can receive the mouse's position which is generally hundreds (for normal mice) to a thousand (for a gaming mouse) of times per second.

By comparison Unity's software cursor is only moving as fast as the frame rate which is significantly lower than the rate at which the mouse is capable of reporting it. This is compounded by the fact that the game engine treats it as just another object rather than the special one that is the hardware cursor.

Edit: Just in case this helps we can put this into actual numbers. A normal mouse has a latency of around 4 milliseconds (250 reports to the OS/sec), your average gaming mouse has a latency of around 1 millisecond (1,000 per second), and a game running at 60 FPS has an effective latency of 16.67 milliseconds.

And there is no way around this limitation as there is no way to make a game run as fast as the hardware cursor can, so you have to choose do you want a fancy software cursor or do you have to live with the simple but fast hardware cursor.

 

@Ryiah This is exactly my point though.

I have created multiple test cases around this. You would expect, when running at 60 FPS for the GameObject to be, at most, 0.0166 seconds behind the display of the hardware cursor. But, that is not the case.

And, this gets even worse at lower FPS. At around 15 FPS, you would expect the GameObject to be, at most, 0.66 seconds behind. But it's significantly worse. When you stop moving the mouse, you can *watch* the GameObject slowly catch up. This shouldn't be the case. The display should be extremely choppy. You should not see a slow, smooth motion of the GameObject catching up to the hardware cursor.

Input.Mouseposition behaves as though it has a queue of previously sampled values that the tick loop is trying to process.

 

I just did a few minutes of testing immediately before writing that post.

In the OnPointerDown test, I made a script implementing IPointerDownHandler that would move a game object to PointerEventData.position, then I tried to move the mouse smoothly across the game window while clicking at a specific point. With the framerate capped to something low, the object sometimes got positioned approximately at the space where I thought I'd clicked, and sometimes got positioned noticeably after that point, but never noticeably before that point, so I believe it's taking the mouse position from the start of the frame following the mousedown.

I also did a test with IDragHandler but it didn't seem noticeably different from using LateUpdate.

With IDragHandler or LateUpdate and the frame rate set to 1, I did a test moving my mouse in a slow circle, and it looked to me like every time the position of the object updated, it was exactly 1 frame behind the mouse's current position. (With a bit of practice, I timed the circle so that the object always moved one quarter-circle at a time, and the mouse was always one quarter-circle ahead of it at the moment it moved.) I didn't see evidence of queuing for more than the space of a single frame.

But when testing at 10 fps or 60 fps I can't reliably tell the difference between individual frames. If there's queuing happening but only at higher framerates, I'd probably need a slow-motion video in order to tell for sure.

 

Thank you @Antistone for that detailed explanation. I appreciate it. I will need to investigate those handlers more.

 

I can confirm this is happening for me too. It is also true of Input.GetAxis("Mouse..."). Instead of reporting the latest position, it goes through a queue of samples. For me this results in the game behaving as if it received mouse input even though the user has stopped using the mouse altogether.

Has anyone figured out how to deal with this issue?

 

I'm seeing this too in 2019 LTS on macOS, very noticable int he editor, not sure if it exists in standalone builds.
my Framerate is very high, but there's still a significant amount of lag.

I am setting the position of the UI sprite which is my game-cursor in a normal update loop.

 

It's already been said. Hardware cursor is the only way to get a non-lagged cursor.

If you want a software cursor, LateUpdate might make it less laggy than Update.

You can see this in every game that has a fancy cursor. See Fallout 4/Skyrim's menus for example. It's laggy, unless they use some special engine code to get it working faster.

The Unity games Shadow Tactics/Desperados 3 use a hardware cursor for the main cursor, but have a graphic to the bottom-right and you can see it lags behind.

 

It's normal that it lags a bit at 60fps, but I've turned Vsync off and I don't see any speed increase compared to Vsync on... Why is this?

 

Let me sum this up for future readers.

Hardware cursor = almost no latency, directly updated when the mouse sends the info to the cpu. Entirely dependent on hardware. My gaming mouse has no latency, yet my air mouse has some latency.

Software cursor = always latency, has to sync with drawing and drawing is 1 or more frames behind, depending on video driver settings. You can't get away from this latency. This is also dependent on how often the mouse position is poled by the software, as it might not be at the beginning of each frame and might even be a buffered value.

 

It's always 1 frame behind.

What I don't understand is, how can the hardware cursor be that much faster if the monitor frame rate is fixed and the game is running at 2x or 10x the monitor refresh rate. That makes no sense.

Turns out, at least some people try to tell me, unity's game view doesn't actually render at the fps the stats gizmo displays. (I find that outrageous, btw.)

 

I still do not understand this issue.

I get the "recording the mouse position at the beginning of a frame" argument, but why the software mouse needs multiple frames to 'catch up' is beyond me.

Here's a video. As you can see, the OS Cursor has been still for multiple frames, while the Software cursor is still "playing back" previous positions. Why?

 

Try using Cursor.SetCursor, according to unity docs, it sets the appearance of the hardware pointer.

 

Same issue here, Unity 2019 and mouse position is lagging behind cursor.

 

It's not the mouse position that lags but the frames by which you view it. Use maxQueuedFrames to reduce lag.

 

Look at the video though, that seems to be wayyy more than 2 maxQueuedFrames (the default.)

 

Yeah I just saw the same issue when recording a tutorial video, I was using a 4k 60fps HDMI capture card and can clearly see the mouse pointer in Unity is 4 - 5 whole frames behind the desktop cursor.

Unity states the default value of maxQueuedFrames is 2, but I am seeing mouse lag of over 5 frames. I have not modified the value of maxQueuedFrames so it should be default.

Even in a simple scenes I see same mouse lag exists and am using a modern i9 win 10 system.

For a fps style game five frames of mouse lag is probably very noticeable

 

Yet it isn't, which shows Input.mousePosition isn't the issue. Every FPS/TPS controller I've made only has delay if I include damping.

I created a project to see if the lag is as bad as the gif posted by @noio. There is delay, seems appropriate. Using Update shows more of a delay than LateUpdate. When I move the mouse fast it gets pretty far behind but the system cursor is warping across the screen so it's to be expected.

The gif shows a problem, it's quite obvious that something isn't correct. If maxQueuedFrames is default, I have no idea what could cause that.

 

Yeh, I see. The extent of software lag is dependent on frame rate.

maxQueuedFrames

can be used to help reduce it. If the results are still unacceptable, use the hardware mouse a suggested in the post below yours.

 

How can I enable hardware cusor mode when trying to find the true mouse coordinates using

Mouse.current.position.ReadValue()

?

 

Hardware cursor is enabled by default. It can't be disabled really, only hidden so you can use the position to display an image where it should be, or for FPS just hidden.

You can set the hardware cursor at runtime with textures that are set to cursor in the import settings. Then you apply those using the Cursor API. https://docs.unity3d.com/ScriptReference/Cursor.html.

OR.

There is a setting to change it in the player settings, but if you're cursor has different sprites throughout game play use the cursor API.

 

I am not currently using the cursor API, would that give better latency?

I developed a simple function to translate mouse screen coordinates from

Mouse.current.position.ReadValue

to raycasted coordinates on a rectangle transform:

Code (CSharp):

1.  
2. public void Update() {
3.     if (entered)
4.     {
5.         var dV = Mouse.current.position.ReadValue() - (Vector2)rect.position; //The new input system
6.    
7.         var localRay = UICamera.transform.parent.InverseTransformPoint(UICamera.ViewportPointToRay(
8.             new Vector2(
9.                 (dV.x / rect.rect.width),// * imagePane.uvRect.width + imagePane.uvRect.x ,
10.                 (dV.y / rect.rect.height) * imagePane.uvRect.height + imagePane.uvRect.y
11.             )
12.         ).origin);
13.         localRay.z = 0;
14.      
15.         pointer.transform.localPosition = localRay;
16.     }
17. }
18.  

This produces nearly identical results as the gif shown above: https://forum.unity.com/threads/input-mouseposition-is-queued.748277/#post-6977156

Takes nearly 5 to 6 whole frames before Unity's mouse and Windows mouse coordinates agree.

Is there any way I can get more recent mouse coordinates?

 

The Cursor API is how you manipulate the hardware cursor, so there would be no software latency.

 

Think there's a misunderstanding happening here.

@CloudyVR just wants to get the freshest possible mouse coordinates, (maybe for "Shooting Things") and

Mouse.current.position.ReadValue

is the way for that.

The Cursor API just refers to how the cursor is rendered. That has no influence on whether or not you'll be getting fresher coordinates.

A Software Cursor is when you use the above

Mouse.current.position

to draw your own cursor inside of the game, and — as in my GIF above — the position of that Software Cursor will lag behind.

The Hardware Cursor is the "true" cursor that is positioned by the OS, but unfortunately, there's no way to read its coordinates, that's kind of the central issue in this thread. The only thing you can do is change the image used to draw it (with the Cursor API). But if you're doing raycasts (e.g shooting things) and other things where you need to read the coords, that's not really helpful.

Both a Software Cursor and "Shooting things" need fresh mouse coordinates. But only in case of the Software Cursor can the problem be solved by .. not using a Software Cursor (i.e. change the appearance of the Hardware Cursor using the Cursor API).

 

I'm not sure what would cause such a delay in your gif, I do not see this in any of my projects. The delay becomes greater the lower the frame rate, which is expected. With a good frame rate it shouldn't be ~12 frames behind like your gif shows.

I would submit a bug report as it will give Unity your hardware specs and editor settings to see if there is an issue somewhere. It's supposed to be 2 frames behind MAX and it's not, it could be a multitude of things that can cause it.

Using the Cursor API will hide that slight delay and the average reaction time will not notice the delay(33.33ms at default.) Unless the frame rate is low of course, but you get that with any game.

 

Yes, in practice however you wouldn't want to use both hardware and software mice simultaneously. The issue here is the comparison between the two. From a player's perspective, raycast and crosshair positions will match (dependent on

maxQueuedFrames

latency as mentioned above) when reading from the mouse position during the same frame.

Compute coordinates from delta values which are inherently framerate independent:

Code (CSharp):

1. public float MouseSpeed;
2.  
3. void Awake ()
4. {
5.     // NB This can lower framerate. See docs
6.     QualitySettings.maxQueuedFrames = 0;
7.  
8.     // Lock and hide the hardware mouse
9.     Cursor.lockState = CursorLockMode.Locked;
10.     Cursor.visible = false;
11.  
12.     // Initialise the position at screen center
13.     MousePosition = new Vector3(Screen.width / 2f, Screen.height / 2f, 0f);
14. }
15.  
16. void Update ()
17. {
18.     MousePosition.x += Input.GetAxisRaw("Mouse X") * MouseSpeed;
19.     MousePosition.y += Input.GetAxisRaw("Mouse Y") * MouseSpeed;
20. }

 

I found the solution, just make the mouse world position calculation in LateUpdate() and see the result