How can I ignore Enemy's camera\controllers with netcode?

I currently have two players in the server, each has it's own open XR rig (VR sensors) and a camera.
For some reason, even though in the NetworkObject of the camera each player only has ownership of it's own camera, whenever a new player enters they overwrite all the other cameras and everyone sees through that new camera. I have tried to implement in the code a statement that checks as a client which cameras you own and if you are not the owner it uses

Code (CSharp):

1. playerCamera.enable = false;

, but to my surprise whenever the client uses it for itself, it will also disable the camera for the player that should be using it.

I have the same issue with the VR sensors, whenever one client moves their hands it will move everyone's (Because one VR controller moves all the open XR rigs in the scene) even though they do not own the networkObject. I think that in Photon Engine there is an option to choose if some objects are setActive to a specific client. But I couldn't find a way to do that with netcode.