Resolved How to fix jittering when client-only object lerps toward a NetworkObject's position

EDIT: The jitter is considerably less in a standalone build. This might just be due to using the Unity Editor. It still happens in the standalone build but much less so. It might be because I'm taxing my CPU by running two instances of the build (host and client), though I'd really prefer to eliminate the jitter completely.

EDIT 2: I moved the code to LateUpdate and that seems to fix it completely. I guess I needed to allow NetworkTransform to do its interpolation code first before I make use of the transform.position.

I'm making a space racing game. The player prefabs have a NetworkRigidbody and NetworkTransform. So far, Netcode for GameObjects is working correctly.

But the way I set it up, the camera prefab is entirely separate from the vehicle, it is just told which vehicle to follow. This camera prefab is not a Network Object. Each client has one camera prefab, and it is told to follow the player prefab that the client owns.

It doesn't parent itself to the vehicle prefab, rather it just keeps updating its position to the vehicle's transform.position, with some lerping to create a rubber-band effect (when you accelerate, the camera stays behind for a while, then catches up).

It works fine with a host (no jitter), but for a client, there is very pronounced jittering on the camera transform. The player vehicle transform itself is fine, thanks to NetworkTransform, the only problem is on the camera transform. When I use the Unity Editor as a client, I can see in the Scene View that it's only the camera transform that jitters.

When I turn off the lerping in the camera's code (for the rubber-band effect), the jitter is eliminated, but I need that effect (client requests it as a feature).

I don't want to turn the camera prefab into a Network Object because all it does is copy the position of the player vehicle Network Object anyway, it seems redundant and a waste of bandwidth.

I also have some client-only visual effects that follow the player position smoothly using lerp. They exhibit the same problem.

Here is my code for the camera's rubber-band effect, it is called in Update of the camera's MonoBehaviour:

Code (CSharp):

1. // Smoothly follows the Ship's speed, this is what we use for cam position elasticity.
2.  
3. // This is the rigidbody.velocity.magnitude that was transmitted from the server
4. // (all physics simulation happens on the server).
5. // Naturally this changes as the ship accelerates and decelerates.
6. float shipSpeed = _shipPhysics.Speed;
7.  
8. // Lerp towards that speed
9. // _trackedSpeedSmooth is set to 2
10. _trackedSpeed = Mathf.Lerp(_trackedSpeed, shipSpeed, _trackedSpeedSmooth * Time.deltaTime);
11.  
12. // Express the tracked speed in a 0.0 to 1.0 range.
13. // 1.0 means we've finally caught up to the actual current speed.
14. // (even though the way we use Lerp for the _trackedSpeed is that janky way to use it,
15. // it actually looks like it manages to reach the final value,
16. // because speedPercent eventually does reach 1.0 when I test it)
17. float speedPercent = shipSpeed > 0 ? Mathf.Clamp01(_trackedSpeed / shipSpeed) : 1;
18.  
19. // ---------------------
20.  
21. Vector3 shipCurrentPosition = _ship.position; // this is the player Ship transform.position
22.  
23. // The Follow-Behind Position is a Vector3,
24. // it serves as the position that is always behind the Ship.
25. // This is more evident when the ship is at high speeds.
26. // _followSmoothSpeed is set to 5
27. _followBehindPosition = Vector3.Lerp(_followBehindPosition, shipCurrentPosition,
28.    _followSmoothSpeed * Time.deltaTime);
29.  
30. // We move between the Follow-Behind Position and actual current Ship Position,
31. // based on the speedPercent we calculated earlier.
32. // That means any sudden increase in speed will make the camera move near the
33. // Follow-Behind Position, making it fall behind, and then after a short while,
34. // it catches up to its usual position since speedPercent goes back to 1.0.
35. // This _rigTransform is the camera's transform.
36. _rigTransform.position = Vector3.Lerp(_followBehindPosition, shipCurrentPosition,
37.    speedPercent);