Question Which interpolation to use for predicted physics

When running the physics simulation in the prediction loop, in nonideal networking conditions I can observe still jitter and micro teleporting for predicted entities.

I assume it has to do with not choosing the correct way to interpolate the predicted entities:

- using PhysicsBody Smoothing (None, Interpolated, Extrapolated)
- using Translation and Rotation Variants with SmoothingAction.Clamp, SmoothingAction.Interpolate,
SmoothingAction.InterpolateAndExtrapolate
- using GhostPredictionSmoothingSystem and Registering `DefaultTranslateSmoothingAction` (and maybe also adding one for rotation?)

As there are many different combinations and possibilities, I'm not sure which combination should be used in which circumstance.

Another question related to this is how to set the number of ticks the server will run at (apart from setting physics Ticks Per Sim Tick).

I'm currently testing with a Quest 2 (at 72fps) headset, and the physics predictions work, but not well enough (in terms of smoothness) to be useable in VR, under normal conditions (around 50-70ms of latency).

Any advice would be appreciated.

 

Thank you @timjohansson,

This cleared up a lot of questions I had.

Now the local prediction and physics interaction work already relatively well (still see some 1 frame jumps that aren't smoothed out from time to time.). But the main issue is for remote players.

If I locally grab a physics-based object (by pinning it to the hand with a joint or by adjusting the physics velocities every frame) it works as expected, but a remote player using the same behaves very differently. It's extremely jittery, in a way that can't be really explained by wrong predictions.

I tried both, with a Remote Predicted Player, and without adding the cost fields, but in either case, it behaved the same.

I am currently running the Simulation at the same tick rate as the framerate.

Might I be missing something that the remote player interactions behave so differently compared to the owner-predicted one?

Also is there a way to smooth the motion of the predicted entities, as it seems the interpolation or extrapolation of the physics parts isn't enough to smooth out small jitters even for owner predicted entities?

 

Thank you @timjohansson that helped a lot.

Indeed switching it to be predicted (and my better understanding of the prediction loop and not making some wrong assumptions), made it work incredibly well.

A lot better than I was expecting, especially with joints, etc. it is indeed looking very promising.

 

@timjohansson

I'm currently looking at an issue where I'm seeing the occasional "1-frame jump" mentioned above on an interpolated+predicted kinematic moving body.

I made a little test for debugging. My interpolated+predicted kinematic body moves with a constant velocity along the z axis, and I've tried to log the following values on every variable frame right after the TransformSystemGroup where interpolation is applied:

• LocalToWorld.Position.z (interpPos)
• PhysicsGraphicalInterpolationBuffer.PreviousTransform.pos.z (prevPos)
• Translation.Value.z (currPos)

here's what I'm getting:

Spoiler: More info

Here's what my test rigidbody ghost looks like (the "initial linear velocity" is what makes it move):


And here's my debug.Log code:

Code (CSharp):

1.  
2.     [UpdateInWorld(TargetWorld.Client)]
3.     [UpdateInGroup(typeof(FixedStepSimulationSystemGroup))]
4.     [UpdateAfter(typeof(BufferInterpolatedRigidBodiesMotion))]
5.     [UpdateAfter(typeof(ExportPhysicsWorld))]
6.     public partial class InterpolationDebugPredictionSystem : SystemBase
7.     {
8.         protected override void OnUpdate()
9.         {
10.             GhostPredictionSystemGroup ghostPredictionSystemGroup = World.GetOrCreateSystem<GhostPredictionSystemGroup>();
11.             uint tick = ghostPredictionSystemGroup.PredictingTick;
12.  
13.             Entities.ForEach((in Translation translation, in InterpolationDebug interpDebug, in PhysicsGraphicalInterpolationBuffer interpBuffer) =>
14.             {
15.                 UnityEngine.Debug.Log("PHYSICS PREDICTION UPDATE - tick:" + tick + " - prevPos:" + interpBuffer.PreviousTransform.pos.z + " - currPos:" + translation.Value.z);
16.             }).Schedule();
17.         }
18.     }
19.  
20.     [UpdateInWorld(TargetWorld.Client)]
21.     [UpdateInGroup(typeof(SimulationSystemGroup))]
22.     [UpdateAfter(typeof(TransformSystemGroup))]
23.     public partial class InterpolationDebugVariableSystem : SystemBase
24.     {
25.         protected override void OnUpdate()
26.         {
27.             float mostRecentFixedTime = (float)World.GetOrCreateSystem<RecordMostRecentFixedTime>().MostRecentElapsedTime;
28.             float timeAhead = (float)Time.ElapsedTime - mostRecentFixedTime;
29.  
30.             Entities.ForEach((in LocalToWorld ltw, in Translation translation, in InterpolationDebug interpDebug, in PhysicsGraphicalInterpolationBuffer interpBuffer) =>
31.             {
32.                 UnityEngine.Debug.Log("<color=#00FF00> VARIABLE UPDATE - timeAhead:" + timeAhead + " - interpPos:" + ltw.Position.z + " - prevPos:" + interpBuffer.PreviousTransform.pos.z + " - currPos:" + translation.Value.z + "</color>");
33.             }).Schedule();
34.         }
35.     }
36.  

There is something fishy going on in the two highlighted lines here: On one frame the "interpPos" is exactly the same value as "currPos", and then on the next frame, the "interpPos" goes back a bit. I think this is where the jitter or "1-frame jump" is happening. The "timeAhead" value is the same as the one calculated by the
SmoothRigidBodiesGraphicalMotion system; it's "Time.ElapsedTime -
m_RecordMostRecentFixedTime.MostRecentElapsedTime". It kinda looks like something triggers a variable update at a "Time.ElapsedTime" that is exactly the same as the elapsedTime of the latest FixedUpdate, so the "timeAhead" value is 0, and the interpolation code decides to skip interpolation when elapsedTime is 0. This problem doesn't happen in offline mode, because the "elapsedTime" of a variableUpdate will pretty much never perfectly coincide with the elapsedTime of a fixedUpdate in practice.

In short: I believe this interpolation jitter happens because once in a while, something in NetCode sets a "Time.ElapsedTime" for the variable update that is exactly the same as the elapsed time of the last physics prediction update (or of the latest tick). After a bit more digging, I've discovered that this happens in "GhostPredictionSystemGroup" when "ClientSimulationSystemGroup.serverTickFraction == 1". Now we'd have to find out if there is a good reason for this, and if it can be changed. Looks like the serverTickFraction is set to 1 in "ClientSimulationSystemGroup", with the comment: "// If the tick is within +/- 5% of a frame from matching a tick - just use the actual tick instead". What was the original intention for this bit of code?

But aside from all this, I have some vague doubts about the way we save the "previous position" for interpolation during the physics prediction group. Shouldn't it be saved only once before all of the several physics prediction updates on any given frame are processed? Basically it would make sure that interpolation works from whatever transform was last rendered to whatever transform is up-to-date in simulation, regardless of if there have been prediction corrections or not. I could be wrong though

 

@timjohansson
Too much info in my last post for it to be easily digestible, so I'll sum things up:

I've solved the interpolation jitter issue by removing all the lines of code in ClientSimulationSystemGroup.cs under the "// If the tick is within +/- 5% of a frame from matching a tick - just use the actual tick instead" comment. This is what was happening;

• This +/- 5% rounding code was setting the ClientSimulationSystemGroup.serverTickFraction to 1
• When serverTickFraction is 1, the GhostPredictionSystemGroup sets a Time.ElapsedTime that is exactly the same as the elapsedTime of the latest full tick (or of the latest physics prediction update)
• When the elapsedTime is the same as the one of the latest physics update, the SmoothRigidBodiesGraphicalMotion calculates a timeAhead of 0, and when this value is 0, it skips interpolation and directly applies the transform of bodies in simulation

This is what causes a 1-frame jitter once in a while. Just to be clear; I don't think making SmoothRigidBodiesGraphicalMotion use the "previous transform" when timeAhead is 0 would fully fix the issue. The fix would be about making sure Time.ElapsedTime is always 100% correct, because the interpolation code needs that accuracy to behave properly in any case. Any rounding of the time will also make all movement feel less fluid

 

I'm using interpolate, yes.

My impression is that both parts can cause issues:

• The interpolation skip when timeahead is 0 causes the most blatant jitter
• The 5% rounding causes all movement (not just interpolation) to be slightly less fluid than it should be, due to accelerating/decelerating time-based movement only on certain frames. Visually, it kinda would have a similar effect to a framerate that has spikes once in a while (although they would be spikes of only 5%, so I'm probably exaggerating the impact this actually has in practice)

However, since you're saying the timescale is constantly adjusted to keep up with server, then I suppose the tiny accelerations/decelerations of movement might be inescapable even if we removed the rounding, and so I'd agree that only the interpolation system would need a fix

I'd find it useful to have the rounding % configurable though