Time.deltaTime Not Constant: VSync CameraFollow and Jitter

This Unity Time.deltaTime jitter is really unfortunate. I'm trying to record trailers and it's really starting to cause me problems.

Also having issues with Youtube's compression... it's really destroying quality of videos in dark areas. Like at t=13s the structure's video compression makes it look horrible until lightning strikes.

And also having issues with TextMeshPro.characterSpacing being jittery (a linear rate of increase of TextMeshPro.characterSpacing causes a nonlinear jittery text position).

And don't even get me started on video capture!

Nothing is working properly!!! Grarrrbblblerfff!

 

Don't you just use Time.captureFramerate for recording video anyway?

--Eric

 

I haven't used that. I'll look into it. Thanks.

 

I've considered modulating Time.captureFramerate as an alternative to my heavy-handed methods of getting around Time.deltaTime issues, but haven't yet had the gumption to fully explore it. In theory it _should_ be possible.

 

Note that Time.captureFramerate disables VSync in some systems. It works opening the Editor in OpenGL mode, but in DirectX VSync gets disabled when Time.captureFramerate is different than 0.

 

I got my hopes up when I saw this appear on 2019.2 roadmap:

but then I also read 2019.2.0 alpha release notes and found this:

Why only Android? Is this coming to desktop as well?

Ah, it's been discussed here: https://forum.unity.com/threads/whats-the-optimized-frame-pacing-feature.636847/#post-4291420

 

Anybody seen any updates on this issue?

 

Don't want to rant, but this thread is like 3 years old already. Are there any news?

I've just tested 2019.1.5f1 and it's still like it always was. It's enough to put a cube with simple constant-speed movement (not even moving the camera whatsoever) and it jitters like crazy. You can update it in update() / fixedupdate() / lateupdate(), you can lerp or dump it, you can update it in fixedupdate() with custom interpolation in update() to generate intermediate frames.

Really, makes no difference. Most commercial and successful games made using Unity do the same. At least the one i know of and have been playing - like City Skylines for example. Jitter everywhere.

I've tried the Jobs + ECS + Burst and it's the same. Sadly, cause the overall performance is amazing, but every few seconds there's a hiccup.

Not to mention i have ~72 frames per second on a 60 Hz monitor with VSync turned on, no idea why. Last time i've used Unity VSync worked ok, well… At least it produced framerates corresponding to the monitor refresh rate.

 

The issue goes way past 3 years though, I recently tested this on Unity 3, Unity 4 and Unity 5 and they all got this issue. So apparently at least every PC version we still have access to have had this issue.

 

Latest word from Unity side seems this:

https://blogs.unity3d.com/2019/06/03/precise-framerates-in-unity/
https://github.com/Unity-Technologies/GenLockProofOfConcept

I haven't tested it (won't waste more time on this issue), but for what I've read, it looks like targeted to capture and record video sequences to disk rather than realtime rendering. And I can't see how all that could improve the Time.deltaTime values.

 

i can also confirm 2019.1.5f1 this is aparrant. i have also tested with burst compiled ecs jobs. This is a deal breaker as the jitter affects the games aesthetics. I was thinking of purchasing an AMD graphics card and checking on that, anyone with amd have these issue? (currently with nvidia tested on 2 diff machines )

 

unity's native jitter occurs on all platforms, on all video cards. hardware is not the issue. it is a flaw in unity itself.

 

After some more testing & tinkering i've managed to get mostly stable framerate. Only in build, though. In editor gameplay still jitters.

What i did is i've changed Fullscreen Mode from Fullscreen Window to Exclusive Fullscreen in Project Settings -> Player -> Resolution and Presentation. Then File -> Build and Run.

I've tested it using MSI Afterburner and i get constant 60 FPS and smooth movement - at least using Lightweight RP & some simple, testing scene with cubes and stuff.

I still think there's something wrong in Unity, though. Fullscreen Window is the default for a reason, i suppose. And Exclusive Fullscreen comes with its own quirks. Screen flickering when alt+tabbing or minimizing being the most obvious. But they're minor compared to jittery movement.

There must have been some changes, i think, because from what i remember in the past i've tried everything - all available fullscreen modes in combination with every possible and recommended movement "techniques" too - with no luck.

So i suggest you to try to play with the fullscreen modes, maybe somthing will work for you.

Yeah, i've tested it some more and it seems the FPS in the stats in game window is mostly BS. Real framerate measured using afterburner was totally different.

That makes sense, as from what i remember i've always had trouble to get perfectly smooth movement / animations. But everywhere i've tried to find any solution, everybody said it was mine fault.

Actually it feels quite good, to find out all these years everybody was wrong and i was right

It may be targeted for capture and record, but in fact could be useful for games. In theory it may be possible
to override all the faulty sync unity code. For now i'll stick to my simple solution, but i imagine problems will be back once the scenes get more complex.

So that's at least something to try

Yeah, i don't think either it's related to hardware. It's more likely because of the way Unity calculates deltatime and/or how it handles synchronization with refresh rate.

But i'm in the process of upgrading my GPU for an AMD one anyway, so i'll post here if i find any differences.​

 

...at the cost of experiencing visual tearing. These solutions involve disabling VSync. So it's still a no-go as solution for realtime rendering. Problem is how Time.deltaTime is calculated with VSync enabled.

 

Agree with this. How Time.deltaTime is calculated for VSync needs to be fixed.

 

The core issue appears to be that main loop just doesn't run on consistently at same duration vs rendering that's locked to vsync. What Unity does now is that they clock the difference between playerloops and they do the timing as the first step on the loop. This is how most game engines measure deltatime btw, it's nothing out of ordinary. If you measure the deltatime yourself on Update, it will match quite close to what Unity reports on Time.deltaTime and includes the similar jittering.

So the real issue appears to be more of that main loop is actually running at uneven pace, not necessarily the way they measure the deltaTime. Jittering on the deltaTime is more of a symptom of the underlying issue. It doesn't really matter in the end what the real cause is though, I'd be fine if they could get more accurate timings directly from render thread etc

 

I agree with you that the real root cause of everything is the time variation between the start of the Update loops. But this is likely not fixable. Time.deltaTime on the other hand is probably fixable.

I've noticed that background applications increase the amount of Time.deltaTime variation & jitter. This makes me suspect that external loading (Windows processes, other applications, etc.) are the cause of pushing back the start of Unity's Update loop. Since Time.deltaTime is currently calculated as the time delta between the start of Update loops... this causes Time.deltaTime variation leading to visual jitter.

Unfortunately there is likely nothing Unity can do about the fact that external processes have taken over the CPU momentarily and push back the start of the Unity Update loop. But what Unity *can* do is adjust the way Time.deltaTime is calculated... so that even if the time difference between the start of Update loops varies slightly the Time.deltaTime is still constant.

 

I guess you're right. Initially i've been thinking about implementing some sort of custom vsync handling, but it's probably pretty much impossible, and even if, it would be fighting against the engine. Oh, well.

Agree, most engines have this problems - Unity, Love2D, Godot from what i can recall. On the other hand in Unreal somehow it's less of an issue (at least on Windows). I wonder if they managed to solve this or if this is some sort of creative trickery. Maybe that's why there's an motion blur by default.

 

Unreal always feels butter smooth to me and their deltatime measurement works the same way as on Unity, it just seems it runs the main loop on more consistent manner there.

 

It's been discussed up there already but here's the core issue: if vsync is on, deltaTime needs to be quantized to the duration a frame is displayed (when possible) to avoid jitter. Unity does not do this.

I'm repeating myself a bit here, but it is possible to avoid jitter in Unity by not using any Unity systems that use deltaTime internally (animations, particles, interpolation, extrapolation, etc) and also not using FixedUpdate... Calculating your own quantized deltatime and driving your own systems with it works fine. This is what I have resorted to.

If you can keep your game's framerate stable, and if you keep it simple, you can also get jitter-free physics with autoSimulate off, supplying your quantized deltaTime to Simulate().

 

Same here. Basically that's why i've never believed in all that "you're doing it wrong" which seems to be the most common response for any jitter-related problems in Unity over the net. Because i've been doing the same in Unreal and everything worked fine. Unreal is an overkill for my small projects, tho.

Yeah, i've read the whole thread. And while your solution seems doable it means throwing away a good portion of the engine features. This may work for some games, but what if you really need skinned animations and other non-trivial stuff that's already there in the engine? Reimplementing it on your own? Again, doable. But then what's the point of using Unity? At this point you could write your own engine - or, more realistically - migrate to a different one that don't have such issues.

I get your point, but don't think it's a viable solution for most people. Not to mention there's a lot of Unity users that don't have the skills required for implementing their own low-level systems. I mean, let's be realistic - time is money - that's why we use Unity. To focus on making games, not implementing IK/FK systems on our own.

/ edit:

Also… If that's so simple (relatively at least, compared to all these big projects they're focusing on) i wonder why they don't fix it? I mean, they have like 1k developers out there. They could throw a bunch of them for a few days and it would be done.

Other systems seems to work fine - it's just the input delta time, and once fixed they shouldn't produce any jitter.

And such a simple thing is taking them like a decade already. I really don't get it.

 

Most of these features support simulating on their own.
Like Physics can be simulated by your own deltaTime. As well as ParticleSystem.

I'm not saying that Unity shouldn't fix this. But as a workaround that should work.

 

By no means im saying this can't be a workaround, i'm definietly gonna try because fixing some of the problems is still better than fixing none of them. But i'm more than sure sooner or later i will hit some features you can't drive using your custom delta time.

And guessing from @herb_nice post character animation can be one of those.

 

Because.. it's not that simple I've done this myself, not exactly how herb described but averaged prefiltered deltatimes from timespan x and it does smooth out the result substantially. But this is a hack, not a real solution to underlying problem. It would need ton of failsafes and more hacky checks to actually work without breaking when framerates fluctuate again.

Also would want to point out that in this thread we've focused mainly on V-Sync and it's easiest to measure the issue with it. But it's also possible it still affects non-vsynced setups too, and possibly adaptive sync/freesync/g-sync displays as well, it's just a lot harder to measure these as the adaptive sync displays update rate varies by design. I still don't have one on my end so can't visually estimate it either but the hack mentioned above would definitely not work when v-sync is disabled / when you allow variable update rate on display (but you could still get wrong game loop timings for it).

 

True, but i'm not saying it would be easy for a single person. They're a fairly big company after all! Compare this to other stuff they're doing: whole DOTS stack, 3 new renderers (hybrid probably reuses a lot from LW & HD, but still), node-based editors for shaders & vfx, gpu lightmapper, maintainging support for an awful lot of platforms & other possibly less epic but still time-consuming enough projects.

I'd say proper vsync / adaptive sync handling seems rather small compared to all of that. Especially considering they have a lot - hardly to imagine they don't - developers with a lot of expertise in low/os/driver/hardware/etc -level programming.

I guess they don't see enough ROI in fixing this. Oddly enough, most people over the net can't even see this jitter. Like it was something beyond theirs perception. At least that's what they're saying. /edit: Which btw seems true, otherwise this thread would have 1500 pages, not 5.

 

I don't get it either. It looks like S*** and it is a simple thing to fix. Either let us override deltaTime globally, or quantize unscaledDeltaTime in the engine. The part that actually does stuff in my quantization code is well under 100 lines.

Because the game is mostly done and both options are a pain in the ass. I've written engines before, it's not something to take lightly. Porting sucks.

I have no need for Unity's features that depend on deltaTime because I have my own custom procedural animation and particles, which lend the game a unique feel (and are fun to code, really). So it's Unity as a simple multiplatform foundation really. I know most people do not approach it that way- if I do need to use fancy features I will certainly consider different engines for future projects.

 

You could release it, as an open source or just throw it in the depths of the net. Could be a great help for many

And btw, @rizu raised a valid point about variable framerate synchronization. Did you test that, how your code works with freesync / gsync, maybe?

I'd say it's a rather specific situation. Don't think many Unity users write their own systems like that. But sounds cool, anyway

That's a valid usecase too, i guess. Some time ago i've been looking for something more of a framework than full blown game engine with good multiplatform support, and there's not much to choose from, really. And tbh didn't see anything that can compete with Unity in terms of supported platforms count. That's a great thing to have all these options. So i ended up going back to Unity.

But now i'm back fighting with the jitter (officially) "nobody" can see. And the most anoying part is nobody at Unity cares. They don't even care to specify any ETA, write some BS about how their engineers are already working on it, or anything really.

How on earth we are supposed to use their software if they don't care to provide any ETA on when such a critical thing will be fixed.

/ edit: What i mean is i feel like this thread is already labeled as /dev/null internally and we're all just wasting time writing here. Hope that's not true, but... well, it looks like they have more important things to do.

 

This is out of Unity's reach to fix, they would have to work with Microsoft and they not neccessarilly see this issue as needed (xbox..)

 

So give us workarounds, give us hacks. Games aren't perfect applications and the engine doesn't have to be either. It's been shown this can be addressed in some form, so please Unity, address it.

 

I made a quick deltatime logging test on Unity 2019.1.7f1 and Unreal Engine 4.22.2 (so by the time I write this, latest releases from both engines). I only wrote deltatime on preallocated 1D float array that I looped manually and only wrote the results out on application quit, I also enabled vsync on both engines and only had empty scenes rendering on both, no extra scripts beyond my oversimplified logging script. I timed this for 30 seconds on both engines standalone shipping builds from which logged the last 11 seconds from (so gave both engines time to warm up and ignored the last second from following chart, just in case the shutdown would have affected it). So in the graphs, you see total of 10 seconds of measurements, one measurement per frame:

Unity's standalone build with Exclusive Fullscreen mode:


Unity's standalone build with Fullscreen Window (as expected this is tad worse than exclusive mode):


And finally UE 4.22.2 shipping build with Fullscreen mode:


As you can see from these graphs, the issue in Unity is more of the deltatimes jumping around the target but almost never AT the target rate. In Unreal, things are even more hectic but it's more like looking at the noise, their deltatime variance is also bigger.

For some reason UE4 frame updates still feel smooth where Unity's constantly feel like it jitters, maybe UE4 got some ordered chaos going on here..

 

my hacky quantization method automatically turns off when vsync is turned off, as there is no frame duration reference without vsync. it just falls back to crappy normal unity jitter mode.

quantization also briefly disables itself if there are a few out-of-quantization-range frames close to each other, turning back on when performance is smooth again (ie, no out-of-range frames for a while). other than injecting a frame when running total gets way out of whack (inserting a frame at quantizedDeltatime * 2), that's pretty much all there is to my solution and is a marked improvement over vanilla... it is certainly not perfect but it does work.

 

So @Edy linked this from the reddit and I've just read through pages of comments on all this.

Can someone summarize what the problem is and if there actually is one?

@rizu you seem to know what you're talking about, is there actually a problem, or is this just result of approximation and guesswork w/ regard feeding the gpu with frame data being a one way process until the adaptive stuff is widespread?

 

@frosted Do note that I don't think any of the people on this thread know what the issue is for sure as we don't have all the data for it. But there are things we can speculate with:

This was linked here earlier as it tries to get to the bottom of this as well: https://medium.com/@alen.ladavac/the-elusive-frame-timing-168f899aec92

The core issue is that there's a constant variance on the deltatimes that are not in perfect sync with the rendering when we have VSync enabled (so rendering is synced to the monitor refresh) and this causes visible stuttering in smoothly panning scenes or any camera movement that would need to be smooth. Of course it also affects everything but since camera is usually moving through the scene, it's the part that affects the whole frame then and is therefore more noticeable. It's impossible to make butter smooth scrolling for example with the default setup as it will inevitably jjitter visibly every now and then. It's not a big difference but enough to be noticed if you are sensitive to microstuttering, some may not notice this at all.

Even if it bugs you or not, it does affect other things like for example in my case, it messes up post processing effects that rely on velocity changes. Since the deltatimes keep jumping back and forth, this means the post processing effects that rely on motion vectors get constantly big back and forth motion vectors / pixel velocities and this can completely ruin the effect (I've seen this happen on motion blur + when I tried to make velocity based effect, it was pointless as the movement jittering just caused huge flickering on the effect).

Do note that we don't really know how much the actual display updating varies on the gpu/display here. To get to the truth on that, I think one should try to log that data too (I don't know if you could measure this on sw somehow or if you'd need to connect the display cable to oscilloscope for this). It's totally possible that the display sync signal jumps around as well (I assume it varies a bit but don't know by how much). However since we can only measure deltatime post update and gpu's render few frames ahead to have a small buffer, the times we get will always be so much behind that correcting the movement with those lagging times would also show as stuttering. In the big picture, whether the gpu/monitor refresh varies doesn't really change much as we can't react fast enough for it + it would still screw up the motion vectors.

Therefore best we can do is to keep the deltatimes the engine systems use close to the avg refresh target which is actually very stable value if you measure it for bit longer duration. If the display driving varies after this, we've at least minimized the jitter from gameplay code so it doesn't multiply with the potential display variance when VSync is in use.

 

@frosted this post also summarizes the problem in a more technical way, including a picture showing the varying Time.deltaTime values:

https://forum.unity.com/threads/tim...afollow-and-jitter.430339/page-4#post-4173886

Just adding that as for what I've read so far, we could take for granted that the display update rates are totally stable (i.e. 59.94 Hz, 60 Hz, etc).

 

Thing is, I've read a good amount of the posts here and checked out some of the videos linked, etc.

The discussion is so scattershot that after reviewing the material here I'm not sure there actually is a problem.

The talk from the Inside devs was focused on performance optimization and never mentioned anything related to heartbeat stutter for example. They referred to stutter as slow update process dropping frames, not some inherent flaw in unity's deltaTime calculation.

The UE4 sample with giant variation to deltaTime further hints at this potentially being a non issue and not the source of problem.

Since most of the alternative testing being done is based on subjective assessment of the visual result, you can have a placebo effect biasing the reported results. "Everything is better when I use ...."

Here's an example of how Unity runs currently:

Frame1:
FixedUpdate: 2ms
Update: 20ms (time.deltaTime = 18ms)
WaitForVBlank 10ms

Frame2:
FixedUpdate: 2ms
FixedUpdate: 2ms
Update: 10ms (time.deltaTime = 34ms)
WaitForVBlank 4ms

Frame3:
Update: 10ms (time.deltaTime = 14ms)


Like the above strikes me as correct and deltaTime should vary (including being under 16ms at times). This is also, afaik, how delta time works in Unity.

So what exactly is the problem? You'd prefer deltatime be locked at multiples of 1/refresh?

 

The issue is that we can clearly see the jittering on camera movement with Unity standalone builds + same jittering screws up motion vectors. Like mentioned, some people are more annoyed by this and some don't even notice. But you definitely don't see as visible stuttering on Unreal despite the timings tell their deltatime move around even more than in Unity. Their deltatime pattern is totally different though.

 

But that jitter is limited to vsynch being off right?

The unreal build visual test - is that with taa running? (i believe taa would naturally smooth out micro stutter)

 

Everything you see in this thread is written with assumption that VSync is enabled.

I don't use TAA on Unreal as the artifacts it brings are not acceptable in my use cases

 

I actually made a movement test project in past for Unity where I had just constantly moving bars next to each other, it was easy to spot the jittering with that. I guess I could refresh it and visualize the actual issue with it + show how it works with the current deltatime hacks we have to help this issue.

 

Do we have a low complexity scene that can reproduce the problem in a semi reliable fashion?

 

Yes. If you open a blank new Unity project, turn on VSync, and create a cube and move it at a constant speed you will see jitter due to the Time.deltaTime variation. If you want a side-by-side comparison make 2 cubes and during the Update loop call...
CubeA: Transform.position += velocity * Time.deltaTime //This is jittery
CubeB: Transform.position += velocity * (1/60f) //This is smooth. Adjust to whatever your monitor refresh interval is

The amount of jitter is somewhat dependent on the user too. I notice a difference between...
-Computer hardware (My laptop and my desktop and my friends desktop all showed different amount jitter)
-Based on the amount of external background Windows applications running
-Editor vs. Build

 

Well, everything stutters in the editor, there isn't even true vsync there afaik.

 

Correct. That kind of misunderstanding is why I'm kinda skeptical of a lot of this discussion.

 

Many of us have the same issue.

Unity has replicated the issue ^

Before you start accusing others of misunderstanding test two things
-2 Cubes side by side with above script
-Plot the Time.deltaTime variation using script in previous posts.

 

Screen.currentResolution.refreshRate would also give the rounded value (it's an int).

 

But yeah, the issue IS real, just some people are not as sensitive to this. But once you see it and get annoyed by it, it'll pain you a lot Also worth noting that there are games that will not suffer from this by design as you need either smooth camera movement or slowly moving objects etc on your scene to easily spot this. Games with panning camera shots are going to suffer from this where say, fast paced top down shooter where the camera doesn't constantly move on each player step may be fine for most.

 

Hah. It's funny I didn't notice for a long time but now it drives me nuts. My wife doesn't notice.

Having my camera attached to the jittery object also amplifies the issue.

There is also a very large difference between hardware. I took the same .exe and tested on 3 different computers and saw very different amounts of jitter. I think that's part of the reason some notice more than others as well.

 

The problem is, exactly where is that micro stutter and under what circumstances.

In editor, since there's no vsynch - sure - but stand alone build in exclusive fullscreen with vsynch, I would expect there to be far far less. Without the waits for vblank, your update cycles are going to be kinda random in duration and will fluctuate with various noise.

If you're in a fixed framerate environment (vsynch), fixed frame movement should generally be smoother than non fixed frame. Time.deltaTime will, and should, produce more jagged movement than a constant value on a per frame basis, especially under load.

What deltaTime represents is an attempt to synchronize against actual player time regardless of variance in framerate. Framerate independence.

There are two separate questions:
1 - is micro stutter a problem.
2 - is deltaTime wrong or inadequate, and is that wrongness causing issue.

I am not focused on #1. I'm focused mostly on #2.

 

#1 is an issue, that's why this thread exists and what people are annoyed by, I know I came here because I've been annoyed by this jittering from the Unity 4 days and only recently discovered that deltatimes vary this much even on exclusive fullscreen with vsync enabled (you can see my graphs on this few posts up).

#2 this can be debated, but debating it doesn't really fix the underlying issue in any way. The deltatime we get is actual delta between the main loops we get, so that's technically correct. But since everything on the engine depends on it, including users own code, it makes it an issue. There is Time.smoothDeltaTime that tries to avg the deltatime a bit but in my experience, it's nowhere near enough to hide the jittering, we'd need some stronger smoothing algo for it + way to apply it to the engines own systems.

 

@frosted the problem does exist. It's real, and Unity suffers it:

https://medium.com/@alen.ladavac/the-elusive-frame-timing-168f899aec92
(long but easy to read. Really REALLY worth reading)​

From the comments to:
https://twitter.com/VehiclePhysics/status/1136628705030279168

"I tried to make a steady speed endless runner and found it would glitch regularly no matter what I did, so I moved it to GameMaker instead!"

"I've gotten so used to it now that the little stutter is one way I can usually tell when a game was made in Unity "​

From:
https://www.reddit.com/r/Unity3D/co..._in_unity_maybe_but_at_the/erjrkxr/?context=3

"I really, really wish Unity would focus on these unsexy old fundamental problems, instead of just adding flashy features."​

Above in this thread:

"But now i'm back fighting with the jitter (officially) "nobody" can see. And the most anoying part is nobody at Unity cares. "​

Other threads:

https://forum.unity.com/threads/impossible-to-get-perfectly-smooth-motion-with-unity.141417/
(started 2012, last post: Sunday).

https://forum.unity.com/threads/linear-movement-stuttering.142754/
(started 2012, last post: 2018)​

"those poor guys have to battle a lot of jittery stutter deniers"
(this thread, page 4)​

....
(going to give up on all this soon)