Time.deltaTime Not Constant: VSync CameraFollow and Jitter

I think part of the trick is that even if it drops from 60fps to 30fps and each deltaTime is 2x it will still look bad. Not jittery (velocity will be constant)... but it looks bad. Nothing you can do about it.

But to me (not an expert either) the trick is that you don't know the deltaTime will run over the allowed 16.67ms until mid-way through the calculations. But before you start the GPU/update calcs you must assume a deltaTime ahead of time (to increment positions, animations, particle system, etc.). I think that's the big point you missed above. So if you assume deltaTime is gonna take 16.67ms and the GPU & Update loop and stuff takes 10ms to complete then you are OK. So you have 10ms calcs, 6.67ms of idle time, and you run at a constant 60fps. But then lets say you are half-way through your 10ms calculations and another program interrupts and consumes 8ms of time. Well now you "missed" your monitor frame and it will be applied 33.33ms from the last push to the monitor. But you are halfway through your calculations (which used a deltaTime of 16.67ms) so the velocity for that delayed frame to frame won't be right. There's no way to knowing in advance if an external application is going to interrupt your update loop and cause a delay. I think?

Now unity may be able do fancy stuff by looking at past history of total process time and adapt. If total process time takes lets say >80% of monitorInterval then it may bump things up to 2 frames per update loop (33.33ms instead of 16.67ms). This is fine as long as process demands change gradually, but won't account for external app spike demands or Unity spike demands (i.e. an explosion that releases a lot of particles and drastically changes the GPU load from 1 frame to the next).

In otherwords there's no sure way to know if it will take <16.67ms to do the total process GPU/update calcs *before* doing them. So handling dropped frames will never be perfect although I imagine some work can be done to minimize the effects.

But bottom line in my opinion if your dropping frames there's no way it'll ever look good. I don't care much about this scenario much (it needs to be addressed to be sure, but we can't expect it to look good). All the focus should be on the operating condition where we are not dropping frames (which is where most games should operate 99% of the time)... in which case Time.deltaTime exactly equals 1f/MonitorRefreshRate.

Just my take. I don't know much about this stuff.

 

For what it's worth, our team was just trying to understand what deltaTime represents as well.

+1 to everything on this thread, it seems totally broken WRT VSync.

 

https://docs.unity3d.com/ScriptReference/Time-smoothDeltaTime.html

 

I agree you can use Time.smoothDeltaTime improve one manifestation of the bug which is manually moved objects (i.e. objects moved by transform.position += velocity * Time.smoothDeltaTime). I did some testing a while back and don't remember the exact numbers but the % variation of smoothDeltaTime was something like 10% of deltaTime if I remember correctly. Time.smoothDeltaTime was much smoother that Time.deltaTime.

Unfortunately many core Unity components internally rely on Time.deltaTime and this cannot be changed to Time.smoothDeltaTime such as.
-Rigidbody interpolator
-Particle system
-Animations

Unity FogBugz response Feb 20th, 2018
"We plan on fixing this jitter problem by revising VSync algorithm. It is currently under investigation and we will let you know once a solution is released.".

I completely agree with Unity this is the proper way to address the bug because it should fix all manifestations of it (manually moved objects, rigidbody interpolated objects, particle systems, animations, + more?)

The bottom line in my opinion is that with VSync enabled and no dropped frames Time.deltaTime should exactly equal the monitor refresh rate. Anything else will cause some level of visual jitter and I would consider a bug.

 

We've got the same issue here. conditions/senarios(not 100% reproducible):

• We're using 2017.1.0.p5, on Win10/Win7 x64.
• We limit Apllication.targetFrameRate to 30 FPS.
• Unity editor is probably running too long, like in days or such.
• the Time.deltaTime is obviously messed up, especially in empty/small scenes without much CPU/GPU loads, we output the log and found Time.deltaTime drop to 0.01s (100FPS) once a while, while it should be around 0.033s
• Character movement jitters in small scenes, while in big scenes with heavy load, it is not even noticeable.
• If frame limit changed to 60 FPS on the fly, then it becomes OK immediately, probably due to matched VSync without extra manual sleep/block.
• Close Unity editor and restart, everything is OK again.

So I'm guessing it's the Sleep issue: whether the parameter used for sleep is messed up(precision issue or bad calculation), or, less possibly, Windows has problem on scheduling threads/processes. Some of my colleague have the same issue, I'm trying to reproduce it since I restart Unity Editor yesterday and the problem doesn't show up for now.

 

Thanks. If you can figure out what turns this issue on/off I will let Unity know. They emailed me a little bit ago...

"
Mark,

We have contacted our issue-tracker team and they responded that we only generate issue-tracker links for bugs. Since your case is marked as a feature request, we cannot send you any link to track the progress of the case.

We will not forget about this case and we will surely let you know about any resolution of it.
"

So looks like there is no issue# for it, but they are going to try and fix it. So any information we can provide them would be helpful. Let me know if you can turn the issue on/off with repro steps and we'll let Unity know.

Thanks.

 

That's correct, but they also have feedback ( https://feedback.unity3d.com ), where feature requests can be voted on. I don't think Unity themselves actually add anything there, but you could make an entry, so we can vote on it.

 

Hmm.. This also implies that it will only show up in Unity 2018+.

 

Thanks, I posted the idea.
https://feedback.unity3d.com/sugges...h-vsync-is-not-constant-causing-visual-jitter

 

I'm struggling to understand something though. If this issue shows up with a blank new project and a blank scene shouldn't everybody have this problem?

But only a few people have had the problem and found this thread... instead of everybody. Why is that? Is it Windows specific? Is there something different about our hardware or background application loads? Is it too hard to notice or track down root cause (the Time.deltaTime variation)?

 

In my case that Time.deltaTime variation is bad, really bad. When I set up scene with nothing but a sprite moving up and down from the bottom to the top of the screen there is an ugly stutter every second or so. When I set up the same scene in Godot, the movement is perfectly smooth. I really don't know what to say, I have tried everything to remove the stutter, but nothing helped.

 

It's a fundamental issue that affects everything. Animations, Rigidbody interpolated motion, Update interval, etc. are all based off of Time.deltaTime and will all jitter.

Are you manually moving your sprites (like position += direction * Time.deltaTime)? ***If*** you are manually moving your sprites you can create your own filtered/constant CustomTime.deltaTime (i.e. take a 1 second average). This will give perfectly smooth motion... but will have issues if you run into dropped frames (but what method won't). You can try this.

 

I've seen dozens of posts over the years complaining about the same, it's not just this thread. I'm happy Unity is finally looking into it.

I think you've hit the core of the issue, that the Update time is not necessarily the same as the frame time, and that with vsync enabled Unity should report the delta time as a multiple of the screen update frequency, i.e. always 1/60 or 2/60 or 3/60 etc. I'm not sure if there even is a way to solve this when running without vsync though.

 

If you can find those other posts and link the here that may help. Not sure how we draw more attention to the issue.

 

Someone pointed me to this thread after ranting wildly that I too could not attain smooth movement in my game. We are making a mobile title and capping the framerate at 30. Vsync is off and we have had nothing but problems with Time.FixedUpdate and Delta time in trying to get smooth motion. At 30 frames per second it should not be hard to get smooth motion. We are using Rigidbody and physic update and it causes what I call Judder. You will get smooth motion for about 2-3 seconds and then a large hiccup, or you get choppy microjudder. Its noticeable on higher end games with higher frame counts as well but its much easier to hide. You can see it in Firewatch, and other Unity made titles as well. Our game is top down like Zullars example and it is most notable in this viewpoint. I am extremely dismayed that Unity, a industry standard, has this low level of a problem and its not getting the attention it deserves. This should have been fixed YEARS ago.
Here is an example of the problem.

 

I did a bit of experimentation with this myself, what I'm doing is moving a cube with constant speed, changing its color every frame and not clearing the camera. With an even frame rate, this should produce lines with an equal width.



Moving with a fixed timestep is the only way I can see produces the correct results, with smooth deltatime being the closest of the other options I've tried.

I've attached the code as well if anyone else wants to try.

 

I may be wrong, but it seems obvious to me that what deltaTime represents is in fact the time between the update calls, and not necessarily the time between the actual frames being displayed. And since we are not dealing with real time systems, an update call might be delayed by the operating system, resulting in a wrong calculation of the delta.

When running with a variable framerate, I don't believe there is any real solution to this problem, it's just inherit to how renderloops work. However, traditionally, if you wanted to avoid this problem you'd usually go for a fixed timestep, usually by enabling VSync.

The problem then as far as I can tell is that Unity does not solve this correctly, enabling VSync does not actually give us a fixed timestep, which would enable smooth motion.

 

done lots of testing on this, and the smoothest motion achieved here has been by turning Physics2D.autoSimulate off, not using deltaTime (or any of it's variants), and running System.Diagnostics.Stopwatch to calculate our own deltaTime instead- and using that value for Physics2D.Simulate() and all of our own delta-time based calculations.

this is still not locked to vsync and does vary each frame- but appears to jitter much less. We have tried things like smoothing that value, or clamping it to 1/60 or 1/30 when near vsync but those tricks don't work as good as hoped.

 

This is exactly the issue. With VSync enabled and low loading (a blank project with a blank scene) and no dropped frames the Time.deltaTime should be exactly constant... equal to monitor frame rate.

 

Thanks for the tips. This looks like it will work for any manually moved objects (transform.position += velocity * smoothedDeltaTime). Have you ran into any problems using a non-constant physics timestep? (Having different Time.deltaTime and Time.fixedDeltaTime's to deal with and talk between creates headeaches... but I kinda get the reason Unity separated the two... Update vs. FixedUpdate for physics).

However some unity components internally rely on Time.deltaTime and unfortunately they cannot be fixed in this way.
-Animator
-Particle system
-Rigidbody interpolater

 

You are absolutely correct.

On our project, we use our own procedural animation and particle effects, so we do not use Unity's Animator or Particle system. Early testing of the particle system in the 5.x cycle showed that start/stop was not performant enough for our uses so we just rolled our own. Perhaps modern particles are better?

Also, we are stepping physics exactly once a frame, at a variable deltaTime, using Physics2D.Simulate(deltaTime). We set fixedDeltaTime each frame before we do that, as a few physics things (like spring joints) seemed to get squirrely when we did not:

Time.fixedDeltaTime = customDeltaTime;
Physics2D.Simulate(customDeltaTime);
AllOtherStuff.Step(customDeltaTime);

We are simulating exactly the same timestep that we are rendering. This means interpolation/extrapolation does nothing but add error to the result, so all rigidbody interpolation / extrapolation is turned off.

As for your question about having problems with non-fixed timestep- box 2d has been handling like a champ, for our project at least. It is not deterministic, but ridiculous collision reactions are few and far between.

 

July 25th, 2018

I poked Unity QA and they got back to me.

"Mark,

The case remained a long time with our product managers who had to decide when this issue should be looked at. Today it was passed on to our graphics developers. From now on, progress on this case should be faster.

We hope to bring more news to you soon.

Best regards,
Tomas
Unity QA Team"

This is good news!

 

Indeed, hopefully it means it'll be resolved soon!

 

I agree. This sounds like it would work.

The reason they likely have a short frame following a long frame to "make up" for the error is likely to keep timers and time integrators from creeping.

But in general I really could care less what algorithm is used to handle Time.deltaTime when there are dropped frames... the game will look jittery and bad regardless of what method is used. I think all of the focus should be to get smooth constant Time.deltaTime for the 99% of the time when we are operating without dropped frames (which is where most people design their program to operate).

 

I'm not going to claim that I know what I'm talking about, but have a read at this https://medium.com/@alen.ladavac/the-elusive-frame-timing-168f899aec92
I think the reasons explained at the end of this article are the cause of this problem

 

This is an awesome contribution. The article matches what I'm observing perfectly, and it even gives the issue a name: "Heartbeat stutter".

Unfortunately, the conclusion after reading the article above is that the solution wouldn't work because there's no reliable way of knowing when "hardware" frames have been dropped.

Maybe this explains why deltaTime is not constant? It would be a sort of average of the frame creation rate from the CPU side.

 

Excerpt from the article linked by @BakeMyCake:

What happens here is that the game measures what it thinks is start of each frame, and those frame times sometimes oscillate due to various factors, especially on a busy multitasking system like a PC. [...] But due to the asynchronous nature of GPU operation, the GPU actually does make it in time for 60 fps on every single frame in this sequence.

This is what we see as a stutter — animation generated for a varying frame rate (heartbeat) being displayed at actual correct fixed frame rate.

So, basically, there’s no problem whatsoever — everything is running smoothly, it’s just that the game doesn’t know it.​

This is a very good explanation of why Time.deltaTime is not constant, nor necessarily matches the frame rate. It's just the time measured at the beginning of each Update - not the actual rendering time, which remains unknown even at the display driver level.

 

did some revisiting of frametime quantization, and had some very promising smooth results with a simple running total algorithm such as:

private float actualTimeElapsed = 0f;
private float runningTimeElapsed = 0f;
...


actualTimeElapsed += measuredDeltaTime;
float scaledDeltaTime = actualTimeElapsed - runningTimeElapsed;
if (scaledDeltaTime > (1f / 20))
{
// too slow... uhh just let it go
}
else if (scaledDeltaTime > (1f / 40))
{
scaledDeltaTime = (1f / 30);
}
else if (scaledDeltaTime > (1f / 80))
{
scaledDeltaTime = (1f / 60);
}
else
{
// too fast... just let it go...
}
runningTimeElapsed += scaledDeltaTime;

// keeping the elapsed values within a reasonable range is an exercise for the reader

 

^^ we've been testing a simplified version of that running total frametime quantization algorithm (using time remaining rather than 2 time elapsed counters) for several hours across several platforms and it is generally a great improvement in jitter reduction, especially when your framerate is hovering right around 60 or 30. it actually makes gameplay feel slicker and smoother, too. probably gonna ship it.

 

Has anybody seen an issue# or any dev comments on timeframe for addressing this issue?

I've gotten some emails from Unity QA (They've been very helpful and responsive) but not sure how to track this issue.

Thanks.

 

Interesting. Have you found a way to inject your smoothed Time.deltaTime into things like Update( ), Animations, or ParticleSystem that seem to internally rely on Time.deltaTime?

 

We are not using interpolation or extrapolation for rigidbodies, as we are running physics just once a frame at the quantized deltaTime (which works really nicely). This is really the key. The game is running way smoother than ever with vsync on, and that cheesy deltaTime quantization algorithm from above.

We run our own update manager... so it calls our update functions once a frame with whatever deltaTime we tell it. We do not use Update() anywhere as it has a lot of overhead. even with Update(), you could still just read a cached quantized deltaTime and use that in your Update functions...

We also do not use Unity's Animations or Particles, we use our own homegrown procedural systems that use whatever deltaTime we tell it. Even if you did use Unity's stuff, you might be able to modulate Time.timeScale each frame to trick those guys into behaving "good enough"... sounds a bit hairy though.

 

It's should be obvious that that running total frame time quantize code up there is NTSC specific, probably will look a bit choppy on PAL or whathaveyou that is not based off 60hz.

 

Oh yeah, Time.captureFramerate may be of help in getting your particles and animations to sync with a non-unity driven clock: https://docs.unity3d.com/ScriptReference/Time-captureFramerate.html - i haven't tried it, but it might be better than calculating and setting Time.timeScale each frame.

 

Cool, thanks! So you're running physics with variable time step, right? Something like "Physics.Simulate(scaledDeltaTime)"?

 

yep- exactly.

 

I've just verified that when Time.captureFramerate is non-zero then Time.deltaTime = 1 / Time.captureFramerate.

In this case the Update cycles are performed as fast as possible. If VSync enabled then the Update rate will match the refresh rate. Frame-skip may happen if some Update cycle takes too much time so it misses its frame.

 

did a bunch of graphing of delta times and quantized delta times and remaining time and learned some stuff.

found out Time.unscaledDeltaTime is actually pretty good to use as a base for quantization. works about as good as our own high-performance timer did.

we are now also using QualitySettings.vSyncCount and Screen.currentResolution.refreshRate to determine the quantization band. although refreshRate is not very accurate (it's an int, what does it do with 29.97hz, etc?).

we add a bit of fudge on our remaining time calculation to cover that refreshRate error. basically we just scale remaining time by a constant. the fudge allows time to drift slightly, so it is not valid for online games, etc. however, it also lets us only let an unquantized frame through when it really matters, and not when the error builds up in time remaining. this means buttery smooth gameplay.

 

does anyone know how to get a more accurate refresh rate for the current display?

 

Wouldn't that be the average value of Time.unscaledDeltaTime? Other than that, I don't think that there's a proper way to get it in Unity.

VR devices report their refresh rate as float:
https://docs.unity3d.com/ScriptReference/XR.XRDevice-refreshRate.html

Using a native plugin you may use get the refresh rate using the Windows API:
https://stackoverflow.com/questions...-refresh-rate-not-the-rounded-number#18902024

Other platforms may have their own methods, but then you should develop native plugins for every target platform.

 

Hey Edy- turns out an average of Time.unscaledDeltaTime like you said works pretty good. throwing out obviously bad out-of-range values makes it work even better. it takes a few hundred frames to start getting accurate, but that's the best we have so far.

with this refresh rate calculation implemented, that fudge factor i mentioned before is no longer necessary (which is good, fudge was causing some noise).

everything looks really nice, even more smooth and buttery than before. correction frames are very few and far between, even on the nexus 5. the one we test with reports refreshRate 60, but it's really somewhere between 58-59hz. it is a good problem device to test jitter correction on.

 

Hey, guys. Just bumped into this thread. I'm trying to figure out the reason of extreme stutter in my games as well as some other Unity games. I've got it only on one setup (2 monitors, one of them is g-sync). All other PCs (at work, my friends, etc.) runs this test smoothly. Compiled your example and got this:



You can see an extreme variation (1000%).

Some more observations:

1. Running the same test with -force-opengl gives me a stable 120 fps.
2. Running this in a window and alt-tabbing (loosing focus and making it run in a background, but window is still visible) gives me extra smooth 60 fps. But just when I click the window to focus it, it goes to jitter again.



The last observation is extremely curious. It CAN run smooth on DirectX, but can do it only if the window has no focus.

Can this be somehow related to everything said here?

 

I think that fits into the "heartbeat stutter" problem I quoted previously in this thread. The problem is not that DirectX can run smooth. Of course it can. The problem is the Unity application wrongly figuring out the frame rendering time (Time.deltaTime). Due to the multi-tasking nature of the GPUs there's no way to know when a specific frame has been actually displayed in the screen. I recommend you to read the article on the heartbeat stutter I quoted previously:

https://medium.com/@alen.ladavac/the-elusive-frame-timing-168f899aec92

Why the application runs smoothly when the window has no focus? This my own guess, but probably because DirectX disables some GPU optimizations in background applications. These optimizations might be the responsible of asynchronously receiving and presenting the frames. Thus, when disabled, frames are actually presented synchronously and the application can have a better measure of the frame rendering time. Hence, Time.deltaTime shows more precise values in background.

As a possible workaround, I'd try disabling as much GPU optimizations as possible in the graphics adapter settings. If the result improves, then you may re-enable them progressively and maybe find out the specific settings that allows applications to read a better frame rendering time.

 

Time.deltaTime, Time.unscaledDeltaTime, etc. are roughly related to how long it's been since last time Update was called. as OP mentioned, that has very little to do with how long your monitor displayed the previous image, or more importantly, how long the monitor will display the next image. Time.*Time is kind-of the wrong thing to use if you are trying to not jitter.

@soulburner In your case, are you running with vsync on?

 

Part of the issue is many built-in Unity components rely on the Time.deltaTime/delta time between Update()'s which varies drastically and causes jitter.
-Animations
-Particle Systems
-Co-routines
-GUI
-Rigidbody Interpolator
-Any other motion/animation controlled within the Update( ) loop.

These will all jitter due to the internal reliance on Update intervals Time.deltaTime which varies drastically.

Unless you are willing to overhaul the whole Update/Time.deltaTime system like @herb_nice I don't think there's much can be done until Unity fixes this.

I believe the fix should be:

1: With VSync enabled Time.deltaTime should exactly equal the monitor update interval (i.e. 60Hz --> 16.67ms). This no dropped frames condition is where most programs will operate 99% of the time. This small change will make everything look smooth.

2: In the rare case where the system is overloaded and frames are dropped I really don't care much what Unity does. When you overload the system and drop framerate things will look bad no matter what you do. Probably the best solution is for the Time.deltaTime to be an exact multiple of the monitor update interval. You could remember how long the last frame took to complete processing (as a predictor of the current frame) and then round up to the nearest whole frame interval and push to the monitor accordingly. i.e. if the frame takes 20ms to complete rendering (exceeding the 16.67ms monitor interval) then Time.deltaTime should be set to 33.33ms and you push to the monitor every 2 intervals or 30Hz. If the processing load changes drastically from one frame to the next, or background apps cause loading variation, then there will still be jitter... but I do not believe there is anything that can be done about this. Some adjustments may be needed to prevent the time integral from creeping. Again I really don't care much about how this condition is handled because I expect it to look bad when you drop framerate.

Has anybody seen any Unity dev posts or issue tracker on this?

 

You may force Time.deltaTime to have a specific value with Time.captureFramerate.

 

we sure put forth a lot of effort to work around fancy systems just to try to make our videogames play smooth like an old-fashioned IRQ driven gameboy.

 

* i did some testing with this- and it seems to only correct Time.deltaTime and NOT Time.unscaledDeltaTime. be careful if you use both.

 

That's not really true any more (=> in the last 10 years?). Most (all?) TVs these days run in 60hz as well, even in Europe.

 

perhaps worrying about PAL is showing my age, but 60hz is not a safe thing to assume if you are trying to correct jitter. Screen.currentResolution.refreshRate is also not safe to use, as it is innacurate enough to cause visible error... most devices run closer to 59.97hz and that difference is enough to cause periodic noise. an easy to test example here is the nexus 5 we have that claims 60hz, but actually runs much closer to 58hz.