Third Party XRGrabInteractable & Pun2 - Gameobject sync issue

I have been trying to use XR Toolkit and Photon 2 Networking to create a multiplayer experience. When Player A grabs an object using XRGrabInteractable it looks fine. Player B, watching that interaction - vice versa.

When Player B grabs the object from the player A hands, for Player B object is in hand and for Player A object is still in the hand (When grabbed from either players hand, object is not syncing in network - both players think they are holding the object)

public class XRGrabNetworkInteractable : XRGrabInteractable

{

private PhotonView photonView;

private void Start()

{

photonView = GetComponent<PhotonView>();

photonView.OwnershipTransfer = OwnershipOption.Takeover;

}

protected override void OnSelectEntered(XRBaseInteractor interactor)

{

photonView.RequestOwnership();

base.OnSelectEntered(interactor);

}

}