Question Stuttering with rigidbody + Cinemachine

Having difficulties here with some odd stutter/jitter problems. I've seen other related threads, but the way they've described their problems seem different from mine in some ways, so I hope I can get some help with this.

My camera is following a shoulder object that's child to a rigidbody. Brain is set to smart update, late blend (any other combination results in the player stuttering rather than the world).

I'm getting input in Update, and rotations (rotating player on Y, shoulder on X) and force are applied in FixedUpdate.

The result is a smooth player, but stuttering/jittering world.

 

I'm not entirely sure as to what counts as smooth player movement in this context, then. I'm using AddForce and getting direction from Unity's Input System. The rigidbody is set to interpolate, as well.

I'll try fiddling with the input actions, and maybe try finding somewhere to implement averages in the movement force I guess, but I was lead to believe that the systems I was using in this context were doing that stuff behind the scenes already. Any additional clarification would be appreciated.

Edit: I should add, I guess, that the character seems to be moving smoothly from the scene view, too. Not just the game camera.
Edit 2: Also, when just moving with my WASD input, everything is smooth. It's only when moving the camera and the player at the same time that the world seems to jitter.

 

Hmm. Okay, so I packaged up and sent an example project over that replicates the issue that I'm having.

Since then, though, I was fiddling with it a bit more. All jitter seems to go away when I set the brain to Fixed Update and turn off my rigidbody's interpolation. I'm worried about the future consequences of having no interpolation on the player character. If I have to turn it back on later, I'd just run into the same problem I'm having now.

Smart Update on the brain seems to think that Late Update would be best, so I'd like to understand this before I just move on with what seems like a bandaid fix.