Time.deltaTime Not Constant: VSync CameraFollow and Jitter

Ok, though you must understand that this is not a very user friendly solution. We are expected to scan every single alpha/beta release note going forward (Possibly for the next year or more, since timelines are uncertain on the fixes) until we find something that indicates this frame timing issue is 'fixed' on one of the platforms that we are interesting. We won't know how the fix will be worded in the release notes or what issue # to look out for. Another problem is that we have no idea what the internal state of the issue is apart from 'acknowledged' and 'fixed'. That is to say that if one of the issues for a specific platform is put on the back burner or deemed unfixable, we would have no way of knowing so. Say for example frame timing issues on OpenGL2 on Android is deemed unfixable/wont fix or put on hold, will you tell us here?

Sorry I don't mean to give you a hard time here and I really appreciate you taking the time to reply here to us.

Edit:
You can already see above the confusion this causes. @hellstorm had no idea what the fix will be listed as. Did he miss it? Was it not fixed? And when it eventually gets listed, how is he to be sure it wont be listed as something obscure? Its too fuzzy and introduces a lot of uncertainty and confusion.

 

I did some basic timing testing for this using old method I used to verify the DT noise being somewhat the same in past from Unity 3.x to 2019.1. I now tested this again using 2019.1.14f1, 2019.2.21f1, 2019.3.10f1, 2020.1.0b5 and 2020.2.0a7.
My test scene is just blank project where I store values into large preallocated array and this array is serialized into file after measurements are done. I use IL2CPP and incremental GC on player settings and still using the built-in renderer with 64-bit Windows build forced to use always DX11.

My results were bit surprising. I actually didn't see notable difference on this test between 2020.1 and 2020.2:


In fact even 2019.3 gave exactly same looking deltatime consistency. BUT... it's WAY better now on all these three than it ever was before. I redid the 2019.1 test I did last time a year ago and also threw in 2019.2 to the mix to see where this changed and can tell that pretty much every Unity version between Unity 3.x and 2019.2 has similar behavior as 2019.2 and anything at 2019.3 or newer has similar behavior as 2019.3 on this graph:

Do note the different scale on these two graphs.. The one between 2020.1 and 2020.2 fluctuates mainly around 16.66ms like it should where 2019.2 and older practically overshoots this value every time and there's some offset above and beyond this target always.

I also did a test on DX12 using 2020.2 and it looked just like the one I did on DX11, so no major change there. This hints that the change I've seen was from some other major change that happened between 2019.2 and 2019.3.

It also hints that the fix mentioned on this thread may have not landed on 2020.2.0a7 yet, or that I'm missing some toggle to enable it or it simply only works on SRPs (and I've only tested this on built-in). I'm actually wondering about the first option most as it's really not mentioned on a7 final notes at all.

@Tautvydas-Zilys can you tell which 2020.2 release is supposed to have the DX11 fix? Also any idea what happened between 2019.2 and 2019.3 that would explain what I've seen here?

 

Ah I missed this comment. Well it makes sense I didn't see the difference yet . Still curious about the 2019.2->2019.3 change though.

edit: I also realize that empty scene isn't ideal test case for this but I'll do further testing with HDRP with more populated scene once this feat actually lands

 

I only used empty scene for the timing but I don't think I had that big of a dips in past - at least in notable amounts - with actually populated scenes when using exclusive fullscreen. I should have mentioned that all the timings were done in that mode as well. When running windowed, timings are absolutely horrible I hope the upcoming fix will help especially the latter case (when running game in windowed state).

 

thanks for the graph! do you have a up to date version of this script you could share ?


cant wait to see what it looks like with the soon to be released fix heh

 

I've been thinking of the difference between 2019.2 and older vs 2019.3 and newer. It basically boils down to some change that was done that affected playerloop on 2019.3 but it might be just some happy coincidence that is result of some other work (I really like how it now jumps around the vsync target already, instead of overshooting it every single time). Quickly going through the things introduced on 2019.3 that could have made impact on playloop gave me these suspects:

- Baselib update: This is my first guess. More about it here: https://unity.com/releases/2019-3/platforms#internal-baselib

- Profiling improvements: maybe some code was restructured that affected the DT measurement point? Afaik DT has always been measured among the first things on playerloop so wonder if there could have been something that has made the new iteration occasionally start sooner or later.

- Upgrade from PhysX 3.4 to PhysX 4.1 (but then again PhysX has been updated numerous times in past and structural difference between 3.4 and 4.1 isn't that major. It's actually quite small jump, nothing like jump from 2.8 was and the old DT behavior existed with Unity versions that used all PhysX 2.8, PhysX 3.3 and PhysX 3.4).

- AssetDatabase Pipeline v2: seems unlikely candidate but this has required structural changes still.

I don't think it can be from incremental GC unless they've made some last minute change to it on 2019.3. I've used it on my testing even on 2019.1 (ever since it got introduced).

BTW I tested yesterday 2020.2.0a7 on actual HDRP project with some content in it and surely the DT variance got tad bigger scale but it's still jumping nicely around the vsync target like it should. I can imagine the upcoming change will make this better though so fingers crossed here

I can clean it up a bit and paste a link here, yes. But it's just super simple thing that only collects data at runtime, saves it to file at end and I just plot the values manually using spreadsheet

 

sounds like it would be great as a way to capture this for VR without much overhead! (and oh boy does this timing issue crop up in vr)

 

Does this fix apply to FixedDeltaTime for physics steps also?

 

FixedDeltaTime never fluctuated. Fixed is always going to return the same value as what you put in Time Manager, so no it will not change. Try printing this value to see.

But the timing of when FixedUpdate might take place Unity-side, could potentially have different timings, @Tautvydas-Zilys ?

 

I can see physics interpolation getting benefits (smoother visuals) for more consistent delta times, but it wouldn't affect fixedupdate directly.

 

That depends on if Unity wants to use the new method of deltaTime to also time it's internal calls to FixedUpdate, invokes etc - some clarification would be nice.

Doesn't sound to me it's affected so far (for good or bad)... need clarification as I hinted above. Maybe hints aren't enough.

>Interpolation
>Timing of when things get called

Are these also improved?

 

How I understood this is that they swapped the whole DT timing for DX11 in this case, which would logically mean everything using deltatime in the engine would be affected. I don't know if the physics interpolation used deltaTime or smoothedDeltaTime but both should see improvements on this if that's the case. Do note that interpolation itself isn't done on fixedupdate but for the next update.

Also even if there was some possibility to run two different timing methods, nothing would prevent you from implementing physics interpolation yourself with the improved delta time manually

 

Obviously but one would prefer improvements to be baseline and that's what I'm asking Unity.

 

That's pretty important info for bug hunters.

 

a8 is now out

I must say, my initial impressions are... wow!


This is a bit bad graph in a way that I realized that the 2019.2 result is from empty scene and a7 and a8 are both from populated scene which used to stress the deltatime a bit more. So 2019.2 result should be even worse in this comparison but the a7 vs a8 comparison is accurate.

 

Rock solid, well done devs!

 

I had to look a few times before I realised the blue line wasn't made by you as some kind of marker.

 

Edit: This posts oddity only happen if you have mismatching desktop monitor set, in my case I had 144Hz and 60Hz monitors. With all displays set on same refresh rate, new DX11 deltatime is rock solid on Windowed mode as well.

Did some further testing:
Basically both DX11 Exclusive Fullscreen and Fullscreen Window are now spot on but Windowed mode is bit funky:


As you can see from this, a8's windowed mode is "solid" but it's always above and beyond the target. On a7 the values are in the same ballpark but there's a lot of noise. I don't know what's the technical reason why these windowed modes don't get the same result as fullscreen ones.

@Tautvydas-Zilys any comment on what I'm seeing on windowed mode?

For ref, here are some timings I logged from windowed mode:

0.0138902
0.013888
0.0208355
0.013887
0.0208334
0.0138903
0.0138906
0.0208333
0.0138898
0.0208331

And some from exclusive fullscreen:

0.0166667
0.0166659
0.0166698
0.0166638
0.0166683
0.0166656
0.0166662
0.0166671
0.0166668
0.0166667

 

I'll still do further testing just to rule out I'm not missing anything. I've tried windowed mode on the stock resolution it offers + manually forcing it to 1920 x 1080. I'm on Windows 10 Pro with latest 1909 platform update.

Edit: I'm also testing this on bleeding edge HDRP atm.

Edit 2: I'm also running desktop with two 60Hz monitors and one 144Hz monitor. Maybe this causes some confusion (the windowed app does TRY to match the monitors refresh rate it sits at though).

 

Heh, I was just going to write that I narrowed this down to the monitor setup. If I have 144Hz monitor as primary display, DTs go off. If I have 60Hz monitor as primary or just disable 144Hz one, DT's are spot on. Also if I force the 144Hz monitor to run at 60Hz, everything is fine as well.

G-Sync settings on the 144Hz monitor doesn't seem to make any impact. I've tried turning it fully off, turning it on only on fullscreen and also on fullscreen + windowed, all give same results.

 

Ah ok, so I guess no repro project needed This does explain what happens here though.

 

While it does suck it breaks, its good to know why. I don't know if many players would actually play the game on this type of setup either (or in bordered windowed mode in general) but now I can at least put some warning in case the player does try to setup it like this I suppose (or just run my own DT filtering on such case but it's not ideal for animations etc that use engine's own DT).

 

You guys are promoting doing more than just games these days with Unity. Might not always be so niche.

 

Why did he put that blue line in there ... oh nice

 

I don't think the detecting is super hard as long as vsync is enabled. If you look at the DT's here:

0.0138902
0.013888
0.0208355
0.013887
0.0208334
0.0138903
0.0138906
0.0208333
0.0138898
0.0208331

there are always bunch of DTs that are way lower than what the monitor supports (in this case it shouldn't go that much below 16.6s but here we have values like 13.8s which are impossible with vsync on 60Hz monitor - taken that you can somehow reliably detect the monitor the app window is at. Do note that even if you run g-sync etc variable refresh rate monitor, it'll allow lower frame rate but it'll still cap it to the monitors max Hz if you have vsync enabled for the app.

If you just naively avg those 10 delta times, you get a value of 16.667ms. Simple avg will not take obvious frame drops into account though but I'm pretty sure the frame drops could be identifiable as they are not likely to be among the two super stable groups you get for delta times, in my example these stable values are all super close to 13.9ms and 20ms values.

So one approach could be to just always take avg from the two DT groups if one detects such pattern. Like have just thresholds to see when things fall into these two values and only avg them for the user + make special path for obvious dropped/less frequently updated frames.

 

While it wouldn't actually update the visuals at steady 16.66ms in this case, current approach leads into situations where you can have like double the offset making the jitter twice worse than it would otherwise be. If Unity reported steady 16.66ms DT in situation like this and monitor refresh jumps between two values: 13.9ms and 20ms, you can quickly see that in case of using 16.66ms as basis, the offset is never notably bigger than 3.33ms.

If you use values as is (so in this case either 13.9ms or 20ms whichever happens to arrive) and since the DTs are reported from past (so it'll not reflect specifically how long the next frame update will take), you sometimes hit the right value (smooth) and sometimes you'll double the error, getting 6.66ms jitter. Jitter is more random too than the constant 3.33ms offset on the first case, leading to visually less pleasing result.

So I don't know. Just playing with DTs will not have perfect result either way. I guess you can make it tad better but not perfect unless you can somehow trick the windows compositor to deal with this differently.

For game simulation intent alone, stable 16.66ms DT is always way better than the jumping value though.

 

Grand SCIENCE! GO GO GO

 

Looking at the thread here's what I've gathered for the fix:

DX11 - released on 2020.2.0a8
DX12, Switch - released on 2020.2.0a12
PS4, Xbox One - released on 2020.2.0a15
Metal - released on 2020.2.0a16
OpenGL, Vulkan, VR - likely postponed to 2021.1

Not looking for super specific estimates (as I know there will not be such here) but would love to know if DX12 and VR are getting investigated in short term or if they are going to be a thing only on 2021+ dev cycles. Mainly trying to set expectations right here.

edit: updated DX12 state
eidt: updated console states

 

Thanks for working on this issue. Seems like you guys had to dig really deep down into core Unity code to squash it.

 

Just found this thread after hours spent trying to get 100% smooth 72fps on my Quest game, without success... Well, I'm partially relived to see it's a known issue.
I'm currently still using 2019.3.0f3 because I found the balance of things that work for me and I won't risk updating for minor things, but I will sure update to 2020 if the stutter is gone or greatly reduced. Is this already fixed for OSX editor and Oculus Quest target?

 

Currently, none of your mentioned targets are supported.

I'd expect Quest to get the fix through Vulkan in the future. As for editor part... they are almost done with metal support but I frankly don't even know anything about the editor support here as I always do this sort of testing only on standalone (there's always just been too much overhead and extra factors in the editor to do any smoothness testing there).

Anyway for the metal, vulkan and VR:

 

A12 is out now: https://unity3d.com/unity/alpha/2020.2.0a12

I already tested it and getting similar results now for deltatime on DX12 as with DX11 fix

 

I'd like to note that I do have an issue with one of my projects where I can't close the project gracefully by ALT+F4 when HDRP + DX12 is in use (works fine on DX11) and it also occasionally froze after toggling few times between fullscreen and windowed mode (alt+enter) but I can't tell yet if it's related to this change at all or if it's some other DX12 oddity.

I can't repro it on a new HD template so will need to investigate it further.

 

Looking good so far.

I did test Windows DX11 VR using Valve's new OpenVR plugin (had to apply some additional fixes for the plugin to make it work) just for the sake of making sure it's not somehow fixed by the earlier DX11 deltatime fix but can confirm it's still behaving like prior to the fix (which I guess was expected as @Tautvydas-Zilys hasn't mentioned VR being addressed yet).