Question Game view jittery depending on selected GameObject in Hierarchy?

I have a odd issue with my game being jittery depending on which gameobject I have selected in the Hierarchy :/ After spending a few hours on Google trying to narrow down my issue, I have come up with no solution besides eliminating what ISN'T the issue.

First I thought it might be Cinemachine that was the issue, or a Interpolation issue on my player. But after realizing that my interpolation on my Player is set to Interpolate, and that the "playback jitterness" completely disappear if I select my background element in the Hierarchy, I have come to the conclusion that it might be a performance issue...?

I just find it extremely odd that if I select my Player, the game gets extremely jittery, but if I then select my BG, everything is silky smooth? :/

 

EDIT:
With my Player selected, if I "minimize" all my components (the arrow on the left) in the Inspector, my game view is smooth...

No one else seem to have the same issue when Googling it...

Would love for a person to help me figure out why I now need to worry about my Inspector components hogging up all my performance...