Question One Gamepad controls multiple objects in Local Multiplayer

Hey forum,

using Unity 2020.3.35f1
using Input System 1.3.0

so I'm currently porting a project which has local multiplayer. I need to migrate it to the new Input System and while I do make progress I stumbled over a problem regarding local multiplayer.

In the local multiplayer scene all the "player" objects are already there, they are just disabled depending on the previous setup-screen settings the players choose. I'd like to not fiddle around with that since that would a lot of work to restructure. What I'm getting at: I do not want to setup a PlayerInputManager to join players via button press and spawn the objects.

Afaik the "Join Manually" behaviour of the PlayerInputManager should be able to recognize any PlayerInputs in the scene when available, right?

As of now there are four player objects in the scene and depending on what playercount was setup the respective number of players will be enabled. All of the players have a PlayerInput Component attached.
There is a PlayerInputManager as well. When I bootup the scene the PlayerInputManager recognizes the PlayerInput Components and lists them in the Debug-Field in the Inspector.

All of the players have their script where the movement is handled. With simple code like this for example:

Code (CSharp):

1.        
2. if (playerInput.actions["Accelerate"].IsPressed())
3.  

But when I actually play the game with Gamepad#1 which should control Player1 all players move.
If I do the same on attached Gamepad#2 which should control Player2 all players move as well and so on and so on.

Did I miss anything? This seems like a trivial thing to solve, but I don't know what I missed. Is there a setting in the PlayerInput Component or the PlayerInputManager which needs to be setup in a specific way to handle this? Do I manually need to check for the playerinputs in code, if so, how do I do that?