Bug With 2 gamepads pulling a stick at the same time, first one to let go doesn't get cancelled callback

As title says, I have 2 gamepads connected. I'm using a custom Player Input "Input Actions" with "Generate C# Class" toggled.

I have a move Action type Value Control Type Vector2

I have a single instance of this player input class, I'm calling it systemPlayerInput and I'm getting all the input through it.

Code (CSharp):

1.  
2.             systemPlayerInput.Player.Move.Enable();
3.             systemPlayerInput.Player.Move.performed += MovePerformed;
4.             systemPlayerInput.Player.Move.canceled += MoveCancelled;

Based on the device ID that I got on the callbackcontext I discover which player I want to pass that input to, and I pass that on.

Code (CSharp):

1.  
2.     private void MovePerformed(InputAction.CallbackContext obj) {
3.         if (GetAgnController(obj.control.device.deviceId, out AgnosticController agnCtrl)) {
4.             Debug.Log($"{agnCtrl.index.ColorFour(Color.cyan,Color.blue,Color.green,Color.magenta)} Move performed {obj.control.device.deviceId}");
5.             agnCtrl.move = obj.ReadValue<Vector2>();
6.         }
7.     }
8.    
9.     private void MoveCancelled(InputAction.CallbackContext obj) {
10.         if (GetAgnController(obj.control.device.deviceId, out AgnosticController agnCtrl)) {
11.             Debug.Log($"{agnCtrl.index.ColorFour(Color.cyan,Color.blue,Color.green,Color.magenta)} Move cancelled {obj.control.device.deviceId}");
12.             agnCtrl.move = obj.ReadValue<Vector2>();
13.         }
14.     }

And that was working just fine until I plugged in 2 gamepads at the same time.

On the following screenshot, I was moving the stick on a first controller, then I started moving it with the second controller (green on the console debug logs, number 2). then I let go of controller green, no move cancel happened, then I moved controller 1 (blue) a bit more, then I let go of it and then that one gets cancelled. In this scenario green never gets the cancelled callback.

So turns out that I'm only getting the cancelled callback if I don't do my movement at the same time as another controller... Is there an easy fix for this? Or is the answer: "Don't do what you're doing and use the Player Manager we provide."