Showcase Motion Interpolation Solution (to eliminate FixedUpdate() stutter)

This post is useful for those who are interested in a custom solution for motion-interpolation in Unity. If you are new to the topic of fixed timestep loops and motion-interpolation, here is an introduction to that:
https://gafferongames.com/post/fix_your_timestep/

And here is the complementary thread that shows how to make fixed timestep logic more responsive and potentially even more performant:
https://forum.unity.com/threads/sta...motion-interpolation-global-solution.1547513/

And this here documents how FixedUpdate() is timed in Unity:
https://docs.unity3d.com/Manual/TimeFrameManagement.html

The goal is to achieve stutter-free visuals for deterministic game logic that runs in FixedUpdate(). This
TransformInterpolator script does just that:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. /// <summary>
6. /// How to use TransformInterpolator properly:
7. /// 0. Make sure the gameobject executes its mechanics (transform-manipulations)
8. /// in FixedUpdate().
9. /// 1. Make sure VSYNC is enabled.
10. /// 2. Set the execution order for this script BEFORE all the other scripts
11. /// that execute mechanics.
12. /// 3. Attach (and enable) this component to every gameobject that you want to interpolate
13. /// (including the camera).
14. /// </summary>
15. public class TransformInterpolator : MonoBehaviour
16. {
17.    private struct TransformData
18.    {
19.        public Vector3    position;
20.        public Vector3    scale;
21.        public Quaternion rotation;
22.    }
23.  
24. //Init prevTransformData to interpolate from the correct state in the first frame the interpolation becomes active. This can occur when the object is spawned/instantiated.
25.    void OnEnable()    
26.     {
27.         prevTransformData.position = transform.localPosition;
28.         prevTransformData.rotation = transform.localRotation;
29.         prevTransformData.scale    = transform.localScale;
30.         isTransformInterpolated    = false;
31.     }
32.  
33.    void FixedUpdate()
34.    {
35.       //Reset transform to its supposed current state just once after each Update/Drawing.
36.        if (isTransformInterpolated)
37.        {
38.            transform.localPosition = transformData.position;
39.            transform.localRotation = transformData.rotation;
40.            transform.localScale    = transformData.scale;
41.  
42.            isTransformInterpolated = false;
43.        }
44.  
45.        //Cache current transform state as previous
46.        //(becomes "previous" by the next transform-manipulation
47.        //in FixedUpdate() of another component).
48.        prevTransformData.position = transform.localPosition;
49.        prevTransformData.rotation = transform.localRotation;
50.        prevTransformData.scale    = transform.localScale;
51.    }
52.  
53.    void LateUpdate()   //Interpolate in Update() or LateUpdate().
54.    {
55.        //Cache the updated transform so that it can be restored in
56.        //FixedUpdate() after drawing.
57.        if (!isTransformInterpolated)
58.        {
59.            transformData.position = transform.localPosition;
60.            transformData.rotation = transform.localRotation;
61.            transformData.scale    = transform.localScale;
62.  
63.            //This promise matches the execution that follows after that.
64.            isTransformInterpolated = true;
65.        }
66.  
67.        //(Time.time - Time.fixedTime) is the "unprocessed" time according to documentation.
68.        float interpolationAlpha = (Time.time - Time.fixedTime) / Time.fixedDeltaTime;
69.  
70.        //Interpolate transform:
71.        transform.localPosition = Vector3.Lerp(prevTransformData.position,
72.        transformData.position, interpolationAlpha);
73.        transform.localRotation = Quaternion.Slerp(prevTransformData.rotation,
74.        transformData.rotation, interpolationAlpha);
75.        transform.localScale = Vector3.Lerp(prevTransformData.scale,
76.        transformData.scale, interpolationAlpha);
77.    }
78.  
79.    private TransformData transformData;
80.    private TransformData prevTransformData;
81.    private bool isTransformInterpolated;
82. }

The alternative to custom motion-interpolation is Rigidbody.interpolation, that is natively provided by Unity. Here are the pros and cons of custom motion-interpolation so that you can make your decision.

Custom motion-interpolation pros:

• You can turn off the TransformInterpolator for offscreen objects to achieve significantly better performance than doing so by using Rigidbody.
• Transform is in sync with its state (not so otherwise: changes to Transform and Rigidbody.position are not immediate when moved by RigidBody.MovePosition()).

Custom motion-interpolation cons:

• Less performant than Rigidbody.interpolation in general

Performance tips:

• Don't interpolate Transform.localScale if it remains constant, as most objects don't grow in size. You might gain about 10% performance just by doing that if you have many objects on the screen. Engine calls can add up and get expensive.
  
• Turn off interpolation for offscreen objects.
• If the processing budget is too tight, it can still be Ok to just interpolate the camera and the main character/s. Running the TransformInterpolator for few objects won't be the bottleneck.

 

You're over looking an important point: Fixed update should be used for physics only, nothing more.

If you want your own loop that runs at a fixed interval, introduce your own manager.

 

What would be the benefit of your own manager?

 

Well... you can make it do whatever you want in whatever manner you want.

FixedUpdate works to the beat of the physics engine. It usually happens at your fixed timestep, but can happen at different rates depending on what's happening. It should not be used for stuff that happens every n amount of time.

 

FixedUpdate() works to the beat of its scheduler (the physics engine just runs in it). A custom implementation doesn't change that principle (you cannot scatter updates equidistant in time in a single threaded architecture here), except for the amount of "fixed updates" to execute in a given frame. Here is where stability can be improved. But this is just an implementation detail that doesn't affect the intended use case of FixedUpdate(), that is running logic in fixed time steps.

The only thing I see is this: The update-local events have to be mapped/wrapped to work in FixedUpdate() in general, if you don't want to miss them (like GetKeyDown). But this is not a big deal to implement that. Other than that, FixedUpdate() is adequate to be used for any logic that you want to be deterministic.

 

False!!!

FixedUpdate == Physics stuff only. I've seen this in writing by many of Unity's devs here.

Fixed update may run at your specified time step but there are situations where it doesn't and your intention to use it to be 'deterministic' will fall apart.

 

The intention of a fixed update loop is still to run logic in fixed time steps. Again, it doesn't mean that they are perfectly distributed on the timeline. They aren't, as calls to fixed update happen in bursts at the beginning of a frame. This imperfection is actually a requirement for the interpolation to work correctly since processing logic has to be finished before rendering the frame.

The claim about deterministic part is also misguided. The distribution of fixed updates in time doesn't affect determinism. It doesn't matter how you schedule to execute the same logic, the result will be the same.

 

I don't quite understand the point of the video, or what you're solving. You're not supposed to move objects in FixedUpdate, you allow the physics engine to move it for you, and I'm pretty sure it already interpolates that movement for your frame rate. The motions look jittery because it's not supposed to be moved there, it wasn't designed like that. Is there something that I'm missing?

 

If you are relying on physics, then this topic is not for you. This is for games which rely on custom mechanics that you want to be deterministic (so running in Update() and scaling movement with deltaTime is out of the question), as pointed out in the beginning. Yes, you could roll your own manager, as spiney suggested, but I would actually like to utilize Unity's architecture instead of working against it. So this is why running the logic in FixedUpdate() can be a sensible thing to do.

So, assuming you want to run the logic in FixedUpdate(), how can you then make it look smooth? This is what the TransformInterpolator is designed to solve. You can get an introduction to this topic here: https://gafferongames.com/post/fix_your_timestep/

 

What do you mean?

Whenever I've used my own managers, they just run in Update. If I need to spread out the logic then I just use a simple timer, that, would you believe, also runs in Update.

Then it's trivial to make a system for objects to subscribe to this manager.

 

You manually update components. Your code calls the update functions, right?

 

Could you give an example of some mechanic you wanted to be deterministic? The leading reason you want games to be deterministic is so that if you give the exact same inputs, the game would result in the same outcome, but the problem is that by desyncing your game from the Update loop, you cannot make that happen, as the game does not correctly receive input outside of Update.

 

Well, more so I just introduce my own Update equivalent, usually defined by an interface. The manager just gets ticked by Update(), and calls this sub-Update call on everything subscribed to it.

 

Yes, this is what I mean by "working against the engine". Unity has an underlying framework that exposes and calls functions you are meant to just implement. You are sidestepping this infrastructure and roll your own manager to handle the gamelogic. In short, you are sidestepping Unity's flow of doing things. It might still work well for your particular needs, but this is something that is not done by default, but by necessity.

From my current Unity knowledge, I only see a strong reason to sidestep Unity's flow when you need more control over the order of execution than being able to change the script execution order is offering you.

 

Actually no. This is a misleading way to look at it because input is an external influence, outside of game's control. You don't expect the input to be deterministic, but the mechanical response to a received input. The mechanical possibility space is expected to remain invariant. So there are two major points why you want this determinism.

1. Reproduceable edge cases the player can rely on
2. Consistency in the result of accumulative mechanics (like walking from place A to B)

For the following examples, let's assume vsync is enabled so that framerate is synced with monitor's refresh rate (it still won't change the problem when it is off), and we are running the game in a variable time step loop.

An example for point 1. can be an object like a box. And the player discovered that he has just the right jump height to jump on it. The player expects this action to be reproduceable, certainly not dependent on monitor's refresh rate, and the noise in deltaTime.

The same is true for point 2. Let's say you let a character walk from point A to B, and acceleration is involved. Then ideally you expect the player to end up in the same spot after the same time, independent of your refresh rate. But this won't be the case because your movement speed will vary a bit. In fact, the problem is even amplified by the common practice of normalizing speed in games (the "speed * deltaTime" way of things: an easy but rather coarse way of integrating). Take a Quake-like game and let it run in a variable time step loop (Update()) like that. Consider one player playing it on a 60Hz screen. The other one on a 144Hz screen. They would effectively play two different games, not just because one feels more responsive than the other, but also because the movement mechanics will effectively differ. The actions one player can pull off might not even be possible for the other player. The expectation of an invariant possibilty space is broken.

It shows how important determinism is and why I actually would want it in any game. Simply because I want everyone to play the same game when they actually play the same program. It should ideally not depend on noise in deltaTime and the refresh rate.

So the common practice of doing these things, including arguing against FixedUpdate(), is not because it is the correct way to go about this stuff, but because this way is easier for inexperienced developers and works "well enough". Different views on the "well enough" part is what makes a world of difference and divides developers.

 

In the words of the docs: "The fixed time step system steps forward at a pre-defined amount each step, and is not linked to the visual frame updates. It is more commonly associated with the physics system, which runs at the rate specified by the fixed time step size, but you can also execute your own code each fixed time step if necessary."

I've never seen info about fixed update breaking and running out of step. Do you have any examples?

 

This is often a source of confusion: FixedUpdate() will not be called at a fixed time interval measurable by your wrist watch: It is called when a fixed amount of time has passed in-game.

Say that a frame took 0.5 seconds to render. If your fixed timestep is set to 0.02, FixedUpdate() will be called 0.5/0.02 = 25 times in a row at the start of the next frame. You won't get 25 evenly spaced calls throughout the next few frames.

This is an over simplification, since FixedUpdate uses a time accumulator to determine how much time it needs to simulate (and it can become too much, in which case you face the dreaded spiral of death) but you get the idea. So depending on what our understanding of "running out of step" is, we might be talking about completely different things or about the exact same stuff.

Using FixedUpdate() for things other than physics is justified when a variable time delta is not good enough, for instance when implementing your own kinematic character controller. RetroMasters82's needs are quite common.

 

I just want to add that this page in the docs has a really nice diagram explaining how FixedUpdate works.

 

Btw, addressing just the algorithmic stability (not performance related aspects) of Unity's fixed update loop:

Let's consider a scenario where FixedUpdate() runs 10% faster than Update(), for simplicity's sake. Then the optimal sequence of executed fixed updates per frame would look like this:

...111211111111121111111112111...

But in Unity (and any engine using this common practice) you would often find parts like this instead, given enough noise in deltaTime:

...112110111211120202111112111...

Both update sequences have the same amount of updates, the update rate is the same. Ony the distribution varies, and it is easy to see which one is more consistent and preferable. In the latter, you can see the notorious "20202"-stutter cycles gamers love to hate. But the actual part causing an unnecessary decrease in responsiveness (responsiveness is inversely proportional to the variance of updates for a timed input) is the "isolated" zero (...11011...). And a zero shouldn't ideally occur at all here since fixed update is running faster, but it does.

The instability is purely algorithmic. So it is within engine's control to improve the fixed update loop. I've been only baffled why the gaming industry doesn't tackle it on a fundamental level (from what I have seen so far). It bothered me so much that I actually dedicated an entire Bachelor-thesis to improve fixed update stability. The fortunate result: there are pretty simple and elegant solutions which improve stability dramatically.

Motion Interpolation and fixed updates go hand in hand, but they are actually two orthogonal topics.
So I will probably have to prepare some stuff for easier digestion and open a new thread on fixed update loops in the hope that the respective tech guys at Unity listen. Otherwise, I possibly found a backdoor to inject stable fixed update loops into Unity by manipulating fixedDeltaTime on a frame by frame basis to make it dance in a stable manner. It's an elegant hack, but still a hack.

 

A nice GDC talk on how 343 did the whole fixed/variable update for Halo Infinite, the talk is titled "One Frame in Halo Infinite":

 

Nice that Halo values deterministic mechanics. That's a good overview of the engine's architecture. Unfortunately, this format is too coarse to go into detail how the actual fixed update loop is implemented. Its implementation might well be identical to Unity, plus possibly a snapping feature for 60/120Hz.

To address one major point (minute 7:05-9:22) related to the fixed update loop:
I miss an argument why he wants to catch up to "real world time" after an execution delay occurs. In the context of multiplayer, I can see a point. Otherwise, I would discourage this philosophy because, as pointed out, it adds another delay on top of the first one, plus you are risking a further spiral of death. It's a pretty bold claim saying that even professional players had a hard time to detect the delay In general, as long as it is not really necessary to do so, trying to catch up to the real world time at any given moment adds more instability than benefit.

 

Btw. for a serious deep dive into fixed update loops, I dedicated an entire Bachelor-Thesis to their stability (but it also includes interpolation):
https://drive.google.com/file/d/1VajBHqmsajsU3Gp1ZYFhAPhavFQJC7wL/view?usp=sharing

It essentially reveals why "everyone" is implementing fixed time step loops wrong and offers better (and also simple) solutions. Now I know the paper is hard to read and is in need for a tldr-version. But to get the gist, just understand the graphical instability examples. Then you will understand the "Hysteresis Loop". It's almost as easy to implement as the "Standard Loop" (the one Unity is using). But it is significantly more stable and has no caveats.

In addition, we should not use/rely on raw deltaTime values but on a moving average (take around 30-60 samples and everyone is happy). A moving average doesn't introduce a drift to the wall clock (as shown in the paper), and reduces noise significantly. (Also, the data type float is fine for deltaTime, but time/fixedTime should better be double since it accumulates over time. With float, you will get more noise and loss of information in longer play sesssions.)

I have compiled this post into the first one so that all the info on my part is in the first post. Hopefully it will find some use

 

The only thing that bugs me is that we get no native hookFunction that is called before Fixed Update each frame.

Because of that, Input which is sampled in Update, is sampled AFTER the Physics update of the current renderframe. This sucks because the input would already be ready for the taking.

The only method of getting a reliable hook is Start(). Start() is called before the physics update, even if the current frame has none. So I coded my own Tickmanager that instantiates a lightweight component at the end of the frame. The Start of that component will be triggered in the next frame before physics and update and that is my input sampling moment.
If i now press a jump button to set a jumpflag or similar, most of the time the jump will be started a frame earlier than with sampling it in Update.

 

This reminds me of https://forum.unity.com/threads/smo...iminating-hiccup-sources.162694/#post-1149041

You make bold claims in this thread, maybe i should read your thesis...

 

I would like to have a confirmation on that from Unity staff. I would assume Input is sampled before any game logic updates (that includes FixedUpdate()). And if that is not the case, regarding your workaround, make then sure you are really sampling the input and not just asking for the already polled one in the previous frame.

 

Well, do that and science will disagree with you.

So far Unity only focused on reducing noise in time sampling, but they haven't focused on reducing the influence of noise. And that will keep biting you back as noise cannot be completely eliminated, unstable systems will still remain unstable as they are sensitive to even tiny amounts of noise.

To illustrate my point by an actual real-world example:
Here is an example of Unity runningFixedUpdate() (LTS 2021.3.8f1), just recorded on my machine :
I recorded a (fixed) update sequence for around 5 minutes. deltaTime was relatively consistent (but not perfect!) for each frame.

Setup: (fixed update rate: 59.9Hz, refresh rate: 60Hz):

recorded partial fixed update sequence: ...11110111121011111...

We can all agree that "2" is an unnecessary catch-up here (which again is inducing the following "0" to counter balance that) as there is nothing to catch-up according to the given frequency ratio. This wrong catch-up occurs every 15 seconds on average. This is pretty frequent and can throw off gamers intent more than it needs to, as the possible variance of a timed (or continous) input unnecessarily increased by one whole fixed update.

Here is how that part is handled by a stable fixed update loop (the Hysteresis Loop, for example):

..11110111111111111...

It does exactly what you would expect, just dropping a fixed update once in a while. And it does it consistently without breaking a sweat. Zero errors, no matter how often I try. But this is actually expected when you get to the research paper, as this setup only demands stability for loop length = 1. Any stable fixed update loop can guarantee that.

 

Disagree? Sorry, i didn't mean to challenge you, it was just an observation. Bold claims are promising, especially when someone took the time to put his preceding thoughts into academic writing.

Yes, it is sampled at the very beginning of the frame. I was refering to the common implementation: Setting input variables like movement vectors or jump flags that are used in fixed update to drive things. If you set those variables in Update, you have already missed the current frame's Fixed Update in which they could have been used. Furthermore, If you set them in FixedUpdate, you might totally miss keyup/down inputs in frames that have no physics cycle.
Hence a Pre-FixedUpdate hook should be available in my opinion.

 

I see. But you actually don't need the Start()-hack to achieve the same effect. My suggestion would be to just cache the necessary input (like keyup/down) in Update(). And then in FixedUpdate() just ask for the current or the cached input. The input wrapper does something like this, to illustrate the idea:

Code (CSharp):

1. bool isKeyDown(KeyCode key) { return Input.GetKeyDown(key) || isCachedKeyDown(key);}

 

Oh yes, I am aware about the edge cases. Here is also no feature for spawning/teleporting. In general, in a clean architecture, edge cases are not the responsibility of motion interpolation. So this solution is purely for objects that are in motion. You handle edge cases by extending this solution to your liking and putting control systems on top of it. It is also supposed to show how simple the actual core of motion interpolation can work in Unity, as I know there are peole who struggle to come with that. That might give them a good jump start.

I have much childing/unchilding going on in my current Unity game. I now also found an elegant and robust way to inject my fixed update loops into Unity without sidestepping Unity's architecture. So chances are I will present the total package in future; how to achieve a stable FixedUpdate(), plus smooth visuals, plus possible features to handle edge cases for interpolation. I hope that will encourage more people to give determinism a chance.

 

I don't fully understand the requirements for true determinism, but the bigger hurdle may be to convert people to use fixed point math since I believe PhysX and Box2D aren't truly deterministic, and I believe using Unity's built in colliders and raycasts cause floating point errors as well. I could be completely wrong about it since I only have a very shallow understanding of how it works, but it may be hard to get people to switch out of so much built-in functionality.

 

There is a standard for floating points. It is not "totally deterministic" because it allows for different rounding modes. So the same rounding mode has to be set when the game is running. But many games can get away without that. For 2D games, like platformers, often just using a consistent subset of float/double can be sufficient. And 3D games that tend to be more continuous/analog in nature, like many FPS games, will be experienced as practically deterministic, even with varying rounding modes.

So there are cases where total determinism is just an academic virtue, but doesn't necessarily add real value. I would still evaluate the requirements on a case by case basis, though. You would need total determinism, for example, if you need to trigger a preset physical event that has to snowball into a deterministic outcome. Or you might want a replay system that solely relies on input recordings, and you want to go absolutely sure that a noticeably wrong outcome can never occur (though you can still never be sure when cosmic rays flip a bit, haha).

The much bigger issue than rounding modes is actually the scaling of motion in variable time step. Especially the common "speed * deltaTime"- practice, since it is a pretty coarse way of integrating. So the logic of the same program can quickly and noticeably diverge when you run it on different systems/setups. And as we know, that main problem is automatically solved by the fixed time step.

 

I just want to say thank you so much for the TransformInterpolator script. I implemented a moving platform that uses Animations to move the platform in fixed update (as the character is a rigidbody, I want the updates to occur at a very specific time in the physics update loop) via SampleAnimation, but was having a hard time smoothing it. This script saved me a ton of work, set the script execution order, and it just works. Thank you!!

 

Hey, that's great to hear. It's nice that the script can be already leveraged to make you (and your players!) happy. I will soon make a thread demonstrating how to implement a stable fixed timestep loop inside of Unity and update this thread in one go, to show how to handle possible edge cases.

 

@RetroMasters82 thanks for your script, man, very useful !
I also read your thesis as this problem has been bugging me for a while.

A little note to others using the script: it works out of the box, but by design, it introduces a 'one fixed frame' delay on your animation (because the script is performing interpolation using the last two known positions, and not extrapolation, which is a bit trickier to get right).
By 'one fixed frame' delay, i mean 1/50 s (if you left the physics update rate to its default value). In my use case I don't see how this could cause problems, but it may be different in some very specific use cases...

 

Digging deeper into the problem, your solution works perfectly for transforms being moved in fixedupdate, but not so well for characters. Here is my use case: my character has animations, and a character controller to move its root. For some animations, the root is moved via the character controller in fixedupdate, for others, I use the root motion delta time value as input to character controller.move, and this happens in the OnAnimatorMove callback. The tricky part here is that fixedupdate and that callback happen at different times in the frame. I guess I will need to handle the interpolation in late update differently for that case. Not very obvious, but I’ll see if I can figure out a clean way to handle both cases with a modified version of your interpolator. So far what I did is disable interpolation when using pnanimatormove. It’s not entirely correct but it’s not too bad either.

 

Let me stress, the solution is designed to interpolate transforms that are moved in fixedupdate only, nowhere else. Any other use case is wrong semantics. Looks like OnAnimatorMove() is called after Update() in your case. So you should set the animation mode to use fixedupdate. Otherwise, for every transform that is manipulated in the Update()/LateUpdate() cycle (that includes the ones governed by the animation system that internally interpolates), you have to disable the interpolator script.

Hope it helps

Btw, in order to get extrapolation right, you have to predict the unpredictable

 

I'm building a Tennis Ball game, and was having a lot of suttering motion on my ball. Worked like a charm. Thank you for sharing this script.

 

Great script! Thank you. But for some odd reason it works only once. If i disable then re enable the object the interpolation doesnt work anymore.
Fixed, it. I had a slightly different way of using it.

 

Glad it helped. Just curious to know the way you wanted to use it initially.

 

Cool script! This line helped the most...
float interpolationAlpha = (Time.time - Time.fixedTime) / Time.fixedDeltaTime;
FYI, I simplified your script to something like below. I'm testing now but so far it works in Unity...

Code (CSharp):

1.  
2. // Place this on an invisible object being tracked
3. void FixedUpdate(){
4.                 prevTransformData.position = transformData.position;
5.                 prevTransformData.rotation = transformData.rotation;
6.                 transformData.position = transform.position;
7.                 transformData.rotation = transform.rotation;
8.                 fixedTime = Time.fixedTime;
9. }
10. // Place this on the visible tracker
11. void Update(){
12.                 float interpolationAlpha = (Time.time - fixedTime) / Time.fixedDeltaTime;
13.                 transform.localPosition = Vector3.Lerp(prevTransformData.position, transformData.position, interpolationAlpha);
14.                 transform.localRotation = Quaternion.Slerp(prevTransformData.rotation, transformData.rotation, interpolationAlpha);
15. }

 

I guess you execute the modified TransformInterpolator after the mechanics. Know that this simplification does not preserve determinism, as there is no reset of the transform to its "fixed" state.