Scene view open next to game view makes player stuck.

I have a player that moves by applying a force to it. that works perfectly fine unless I have scene view open while running the game. then the player only moves if I double the forces applied to the player. I checked if the forces are somehow smaller, but they're the same.

Edit: When I jump then he's not stuck in any axis while in the air.

Edit: I didn't multiply by Time.deltaTime and the framerate probably dropped a bit when I had the second window open.